Inspection method, inspection device, and program

Abstract

To provide an inspection method and the like capable of automatically determining whether an abnormal portion of a wiring pattern of a device leads to a defect.SOLUTION: An inspection method includes: an acquisition step (S10) of acquiring an abnormal portion image which is an image including an abnormal portion of a wiring pattern of a device obtained by image processing from an appearance inspection image of the device having a laminated structure having one or more layers in which patterns are formed; a generation step (S11) of generating a label image in which the abnormal portion is converted into a color label based on the layer in which the abnormal portion is located, from the abnormal portion image; and a determination step (S13) of determining whether the abnormal portion is a defect of the device by determining the overlap between the color label and the wiring pattern with no abnormal portion according to a predetermined decision tree by using a superimposed image obtained by superimposing the label image and a normal pattern image including a wiring pattern at a position corresponding to the abnormal portion of the device and without the abnormal portion.SELECTED DRAWING: Figure 20

Description

Technical Field

[0001] The present disclosure relates to an inspection method, an inspection apparatus, and a program.

Background Art

[0002] In the manufacturing process of semiconductor devices such as liquid crystal devices and organic ELs, pattern inspection by visual inspection is performed to detect defects in the semiconductor devices. Here, the pattern inspection is an inspection for detecting abnormal portions of a wiring pattern with respect to a wiring pattern that should originally exist.

[0003] For example, Patent Document 1 proposes a method capable of automatically performing pattern inspection of a substrate on which a pattern such as a wafer is formed. In Patent Document 1, a method capable of automatically performing pattern inspection of a substrate by using image processing is described.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] However, in the inspection method of Patent Document 1, although abnormal portions of a pattern can be accurately detected, there is a problem that the abnormal portions do not necessarily correspond to defects in the semiconductor device. For this reason, it has been necessary for an operator to visually determine whether or not the abnormal portions correspond to defects in the semiconductor device.

[0006] The present disclosure has been made in view of the above circumstances, and an object thereof is to provide an inspection method or the like capable of automatically determining whether or not an abnormal portion of a wiring pattern of a device corresponds to a defect.

Means for Solving the Problems

[0007] To achieve the above objective, an inspection method according to one embodiment of the present disclosure is an inspection method for a device having a laminated structure with one or more layers on which a pattern is formed, comprising: an acquisition step of acquiring an abnormal portion image which is an image including an abnormal portion of the wiring pattern of the device obtained by image processing from an external inspection image of the device; a generation step of generating a label image in which the abnormal portion is converted from the abnormal portion image into a color label based on the layer in which the abnormal portion is located; and a determination step of determining whether the abnormal portion is a defect of the device by using a superimposed image obtained by superimposing the label image and a normal pattern image which includes a wiring pattern without the abnormal portion at a position corresponding to the abnormal portion of the device, and determining whether the abnormal portion is a defect of the device by performing an overlap determination between the color label and the wiring pattern without the abnormal portion according to a predetermined decision tree.

[0008] This allows for automatic determination of whether or not an abnormal portion of the device's wiring pattern is linked to a defect.

[0009] Here, for example, the normal pattern image is a pre-prepared image that is the same scale as the image of the region containing the abnormal part from the visual inspection image, and in the determination step, when performing the overlap determination, it may be determined whether the abnormal part is short-circuiting the first wiring and the second wiring by performing an overlap determination between the color label and the first wiring and the second wiring included in the normal pattern image according to the decision tree.

[0010] Furthermore, for example, the normal pattern image is a pre-prepared image that is the same scale as the image of the region containing the abnormal part in the visual inspection image, and in the determination step, when performing the overlap determination, it may be determined whether the abnormal part has broken the first wiring by determining the overlap between the color label and the first wiring included in the normal pattern image according to the decision tree.

[0011] Furthermore, for example, the abnormal portion may be a pattern portion in which a portion of the formed pattern is missing a wiring pattern, or the abnormal portion may be a pattern portion in which a wiring pattern remains in a part of the formed pattern where a wiring pattern should not be formed.

[0012] Furthermore, for example, in the generation step, a trained model may be used to generate the label image from the abnormal part image. Prior to the generation step, the trained model may have been trained using training data consisting of abnormal part images of the device obtained by image processing from visual inspection images, and label images in which the abnormal part shown in the abnormal part image has been converted into different color labels depending on the material of the abnormal part shown in the abnormal part image.

[0013] Furthermore, for example, in the generation step, the label image may be generated by converting the abnormal portion from the abnormal portion image into a color label corresponding to the material of the abnormal portion as seen in the abnormal portion image, using image processing, as a color label based on the layer in which the abnormal portion is located.

[0014] Furthermore, for example, in the acquisition step, the abnormal part image is acquired by applying image processing using the background subtraction method to the visual inspection image of the device to generate the abnormal part image.

[0015] To achieve the above objective, an inspection apparatus according to one embodiment of the present disclosure is an inspection apparatus for a device having a laminated structure with one or more layers on which a pattern is formed, comprising: an image acquisition unit that acquires an abnormal portion image which is an image including an abnormal portion of the wiring pattern of the device obtained by image processing from an external inspection image of the device; a label image generation unit that generates a label image in which the abnormal portion is converted from the abnormal portion image into a color label based on the layer in which the abnormal portion is located; and a determination unit that determines whether or not the abnormal portion is a defect of the device by using a superimposed image obtained by superimposing the label image and a normal pattern image which includes a wiring pattern without the abnormal portion at a position corresponding to the abnormal portion of the device, according to a predetermined decision tree, to determine whether or not the abnormal portion is a defect of the device.

[0016] These general or specific embodiments may be implemented as devices, methods, integrated circuits, computer programs, or recording media such as computer-readable CD-ROMs, or as any combination of systems, methods, integrated circuits, computer programs, and recording media. [Effects of the Invention]

[0017] This disclosure provides an inspection method that can automatically determine whether an abnormal portion of a device's wiring pattern is linked to a defect. [Brief explanation of the drawing]

[0018] [Figure 1] Figure 1 is a diagram showing the schematic configuration of an inspection system including an inspection device according to an embodiment. [Figure 2] Figure 2 shows an example of a computer hardware configuration that implements the functions of the inspection device according to the embodiment using software. [Figure 3] Figure 3 is a block diagram showing an example of the functional configuration of an inspection device according to an embodiment. [Figure 4]FIG. 4 is a diagram showing an example of a pattern inspection image which is an appearance inspection image of the device according to the embodiment. [Figure 5] FIG. 5 is a diagram showing an example of an abnormal part image of the device according to the embodiment. [Figure 6] FIG. 6 is a diagram showing another example of the appearance inspection image of the device according to the embodiment. [Figure 7] FIG. 7 is a diagram showing an example of an abnormal part image obtained from the appearance inspection image shown in FIG. 6. [Figure 8] FIG. 8 is a diagram showing an example of a label image according to the embodiment. [Figure 9A] FIG. 9A is a diagram showing an example of a case where label images with different color labels are generated from an abnormal part image depending on the material of the abnormal part. [Figure 9B] FIG. 9B is a diagram showing an example of a case where label images with different color labels are generated from an abnormal part image depending on the material of the abnormal part. [Figure 9C] FIG. 9C is a diagram showing an example of a case where label images with different color labels are generated from an abnormal part image depending on the material of the abnormal part. [Figure 9D] FIG. 9D is a diagram showing an example of a case where label images with different color labels are generated from an abnormal part image depending on the material of the abnormal part. [Figure 10A] FIG. 10A is a diagram showing an example of an image pair prepared as teacher data according to the embodiment. [Figure 10B] FIG. 10B is a diagram conceptually showing a method of training a model using an image pair as shown in FIG. 10A as teacher data. [Figure 11] FIG. 11 is a diagram showing an example of a normal pattern image corresponding to the pattern inspection image according to the embodiment. [Figure 12] FIG. 12 is a diagram showing an example of an overlay image according to the embodiment. [Figure 13] FIG. 13 is a diagram for explaining an example of a phase detection method using a difference image according to the embodiment. <Figure 14A is a diagram illustrating the difference image according to the embodiment. [Figure 14B] Figure 14B shows an example of a histogram of a difference image according to the embodiment. [Figure 15] Figure 15 is a diagram illustrating an example of a phase detection method using a difference profile according to an embodiment. [Figure 16A] Figure 16A is a diagram illustrating the differential profile according to the embodiment. [Figure 16B] Figure 16B shows an example of a histogram of the difference profile according to the embodiment. [Figure 17] Figure 17 is a diagram showing an example of a flowchart illustrating a decision tree routine for performing a short circuit according to the embodiment. [Figure 18] Figure 18 shows an example of a superimposed image used in the overlap detection algorithm for short circuit detection. [Figure 19A] Figure 19A is a diagram illustrating the connections between wires that are determined to be short-circuited in the SL process according to the decision tree of the embodiment. [Figure 19B] Figure 19B is a diagram illustrating the connections between wires that are determined to be short-circuited in the SL process according to the decision tree of the embodiment. [Figure 19C] Figure 19C is a diagram illustrating the connections between wires that are determined to be short-circuited in the SL process according to the decision tree of the embodiment. [Figure 19D] Figure 19D is a diagram illustrating the connections between wires that are determined to be short-circuited in the SL process according to the decision tree of the embodiment. [Figure 19E] Figure 19E is a diagram illustrating the connections between wires that are determined to be short-circuited in the SL process according to the decision tree of the embodiment. [Figure 20] Figure 20 is a flowchart illustrating the operation overview of the inspection device in the embodiment. [Figure 21] Figure 21 is an explanatory diagram illustrating an example of the operation of the inspection device in the embodiment. [Figure 22]Figure 22 is a flowchart showing an example of the operation of overlap detection for the superimposed images shown in Figure 21. [Figure 23] Figure 23 is an explanatory diagram illustrating an example of the operation of the inspection device related to a modified example. [Figure 24] Figure 24 is a flowchart showing an example of the operation of overlap detection for the superimposed images shown in Figure 23. [Modes for carrying out the invention]

[0019] The embodiments of this disclosure will be described in detail below with reference to the drawings. Each embodiment described below is a specific example of this disclosure. The numerical values, shapes, materials, standards, components, arrangement and connection configurations of components, steps, and the order of steps shown in the following embodiments are examples only and are not intended to limit this disclosure. Furthermore, components in the following embodiments that are not described in the independent claims representing the highest-level concepts of this disclosure will be described as optional components. Also, the figures are not necessarily strictly accurate. In each figure, substantially identical components are denoted by the same reference numerals, and redundant explanations may be omitted or simplified.

[0020] (Embodiment) The following describes the inspection apparatus and other related devices according to this embodiment.

[0021] [1. Inspection System] The inspection device 10 according to this embodiment will be described below with reference to the figures.

[0022] Figure 1 is a diagram showing the schematic configuration of an inspection system including an inspection device 10 according to an embodiment.

[0023] The inspection system shown in Figure 1 comprises an inspection device 10, an imaging device 20, a stage 21, and a stage drive unit 22.

[0024] The inspection device 10 is a device that performs pattern inspection on a device 30, which has a laminated structure with one or more layers on which patterns are formed, and automatically determines whether or not an abnormal portion of the wiring pattern obtained by the pattern inspection is linked to a defect in the device 30. Here, the device 30 is a display device such as a liquid crystal device or an organic EL, or a semiconductor device, and is a device in the manufacturing process, that is, a device as an intermediate product (device wafer) before dicing. In other words, the device 30 is in the state of an intermediate product before dicing, and before dicing, devices 30 with the same pattern are manufactured in a state where they are lined up on a substrate (wafer). The wiring pattern contains one or more wirings. Further details of the configuration of the inspection device 10 will be described later.

[0025] The imaging device 20 images the inspection target area on the device 30 and is composed of, for example, a CCD (Charge Coupled Device) or CMOS (Complementary Metal-Oxide Semiconductor). More specifically, the imaging device 20 acquires an external inspection image of the device 30 by imaging the inspection target area on the device 30. The imaging device 20 is controlled by the inspection device 10, but may also be controlled by another computer.

[0026] Stage 21 holds device 30.

[0027] The stage drive unit 22 consists of a ball screw, guide rails, and a motor, and moves the stage 21 relative to the imaging device 20. The stage drive unit 22 is controlled by the inspection device 10, but it may also be controlled by another computer.

[0028] [1-1. Hardware configuration of inspection device 10] Before describing the functional configuration of the inspection device 10 according to this embodiment, an example of the hardware configuration of the inspection device 10 according to this embodiment will be explained using Figure 2.

[0029] Figure 2 shows an example of the hardware configuration of a computer 1000 that implements the functions of the inspection device 10 according to the embodiment using software.

[0030] As shown in Figure 2, computer 1000 is a computer equipped with an input device 1001, an output device 1002, a CPU 1003, internal storage 1004, RAM 1005, a GPU 1006, a reader 1007, a transceiver 1008, and a bus 1009. The input device 1001, output device 1002, CPU 1003, internal storage 1004, RAM 1005, reader 1007, and transceiver 1008 are connected by the bus 1009.

[0031] The input device 1001 is a user interface device such as an input button, touchpad, or touch panel display, and accepts user input. In addition to accepting user touch input, the input device 1001 may also be configured to accept voice input or remote control input.

[0032] The output device 1002 also serves as the input device 1001 and consists of a touchpad or touch panel display, which notifies the user of information that needs to be known.

[0033] The internal storage 1004 is flash memory or the like. The internal storage 1004 may also pre-store at least one of the following: a program for realizing the functions of the inspection device 10, and an application that utilizes the functional configuration of the inspection device 10. Furthermore, the internal storage 1004 may store a neural network model (generative model), acquired training data, parameters such as the intermediate layers of the model, a procedure for image processing including background subtraction, a procedure for performing overlap detection, and a decision tree used in overlap detection.

[0034] RAM1005 is Random Access Memory, used to store data during the execution of a program or application.

[0035] The GPU1006 is a Graphics Processing Unit that copies programs, applications, and data stored in the internal storage 1004 to dedicated RAM built into the GPU, and then performs image processing according to the instructions contained in those programs and applications.

[0036] The reader 1007 reads information from a recording medium such as a USB (Universal Serial Bus) memory. The reader 1007 reads the program or application from the recording medium on which the program or application is recorded and stores it in the built-in storage 1004.

[0037] The transceiver 1008 is a communication circuit for wireless or wired communication. The transceiver 1008 may, for example, communicate with a server device connected to a network and download programs and applications like the ones described above from the server device and store them in the internal storage 1004.

[0038] The CPU 1003 is a central processing unit that copies programs and applications stored in the internal storage 1004 to the RAM 1005, and then sequentially reads and executes the instructions contained in those programs and applications from the RAM 1005.

[0039] [1-2. Functional configuration of inspection device 10] Next, the functional configurations of the inspection device 10 according to this embodiment will be explained using Figure 3.

[0040] Figure 3 is a block diagram showing an example of the functional configuration of the inspection device 10 according to the embodiment.

[0041] As shown in Figure 3, the inspection device 10 comprises an image acquisition unit 101, a label image generation unit 102, a superimposed image generation unit 103, and a determination unit 104. Note that the superimposed image generation unit 103 is not essential for the inspection device 10; it may be externally provided as long as the inspection device 10 can utilize its results.

[0042] [1-2-1. Image acquisition unit 101] The image acquisition unit 101 acquires an abnormal area image, which is an image containing abnormal parts of the device's wiring pattern obtained from the device's external inspection image through image processing.

[0043] Figure 4 shows an example of a pattern inspection image 41, which is an external inspection image of the device 30 according to the embodiment. The example shown in Figure 4 shows an abnormal portion 31a of the wiring pattern indicating remaining pattern. The abnormal portion 31a of the wiring pattern indicating remaining pattern is a pattern portion in which a wiring pattern remains in a part of the pattern formed on the device where a wiring pattern should not be formed. Figure 5 shows an example of an abnormal portion image 42 of the device 30 according to the embodiment. The example shown in Figure 5 shows an abnormal portion 31b of the wiring pattern obtained by image processing.

[0044] In this embodiment, the image acquisition unit 101 acquires an external inspection image of the device 30 from the imaging device 20, such as the pattern inspection image 41 shown in Figure 4. The image acquisition unit 101 also applies image processing using background subtraction to the acquired external inspection image to generate an abnormal area image, which is an image that includes an abnormal portion of the wiring pattern of the device 30, such as the abnormal area image 42 shown in Figure 5.

[0045] Here, background subtraction is an image processing method that compares an observed image with a pre-acquired background image to extract objects present in the observed image that are not present in the background image. In this embodiment, the visual inspection image of device 30 is used as the observed image, and the normal pattern image is used as the background image, which is an image that includes a wiring pattern at a position corresponding to the abnormal part of device 30 and where the abnormal part is absent. The normal pattern image may be a cropped portion of another visual inspection image for the same layer of device 30 from which the visual inspection image was obtained. Alternatively, the normal pattern image may be a cropped portion of the visual inspection image from the first visual inspection of the same device 30 from which the visual inspection image was obtained.

[0046] The following describes an example of generating an abnormal area image using the background subtraction method, using Figures 6 and 7. Figure 6 is a diagram showing another example of an external inspection image of a device 30 according to the embodiment. Figure 6(a) shows a pattern inspection image 51 that includes an abnormal portion 32a of the wiring pattern, and Figure 6(b) shows a normal pattern image 52 without an abnormal portion of the wiring pattern. Figure 7 is a diagram showing an example of an abnormal area image 53 obtained from the external inspection image shown in Figure 6.

[0047] Specifically, the image acquisition unit 101 performs image processing using the background subtraction method, taking the difference between the pattern inspection image 51 shown in Figure 6(a) and the normal pattern image 52 shown in Figure 6(b), then compressing the gradation by half and adding a bias component 128. As a result, the image acquisition unit 101 can generate an abnormal area image 53 that includes the abnormal area 32b as shown in Figure 7. The abnormal area image 53 shown in Figure 7 is an image in which only the abnormal area 32a shown in Figure 6(a) has been extracted.

[0048] In this way, the image acquisition unit 101 can acquire an image of the abnormal area.

[0049] [1-2-2. Label image generation unit 102] The label image generation unit 102 generates a label image in which the abnormal area is converted into a color label based on the layer in which the abnormal area is located, from the abnormal area image acquired by the image acquisition unit 101. The color label based on the layer in which the abnormal area is located is a color label corresponding to the material of the abnormal area as seen in the abnormal area image. Examples of materials for the abnormal area include wiring material, semiconductor layer, and dust.

[0050] Figure 8 shows an example of a label image 43 according to the embodiment. The label image 43 shown in Figure 8 is an image in which the abnormal portion 31b of the abnormal portion image 42 shown in Figure 5 is converted (replaced) with a color label portion 31c corresponding to the material of the abnormal portion 31b.

[0051] In this embodiment, the label image generation unit 102 generates a label image 43, as shown in Figure 5, which includes a color label portion 31c obtained by converting the abnormal portion 31b contained in the abnormal portion image 42 into a color label corresponding to the material of the abnormal portion 31b reflected in the abnormal portion image 42, for example, as shown in Figure 5. The material of the abnormal portion 31b differs depending on the layer of the device 30, i.e., whether the abnormal portion 31b is located in the layer where the source layer of the device 30 is formed, the layer where the gate layer is formed, or the layer where polysilicon is formed. The material of the abnormal portion 31b can be determined by the color of the abnormal portion 31b reflected in the abnormal portion image 42.

[0052] Here, one method for generating label images is to use deep learning. As a method using deep learning, a generative model that generates images according to learned rules may be used. More specifically, the label image generation unit 102 may generate label images from abnormal part images using a trained model. In this case, the trained model should be trained using abnormal part images of the device obtained by image processing from visual inspection images, which are prepared as training data, and label images in which the abnormal part shown in the abnormal part image is replaced with a different color label depending on the material of the abnormal part shown in the abnormal part image.

[0053] As a result, the label image generation unit 102 can use a trained model to generate label images in which the abnormal parts contained in the abnormal image are converted (replaced) into different color labels depending on the material. Alternatively, the label image generation unit 102 may use a trained model to interpolate the missing parts of the abnormal area in the abnormal image due to background subtraction, and generate label images in which the abnormal areas are converted (replaced) into color labels.

[0054] The following examples, using Figures 9A to 9D, illustrate cases where a trained model generates label images with different color labels depending on the material of the abnormal area.

[0055] Figures 9A to 9D show an example of how label images with different color labels are generated from an image of an abnormal area, depending on the material of the abnormal area. In the figures, GL refers to the gate layer where the gate is formed, SL refers to the source layer where the source is formed, and Poly-Si refers to the layer where polysilicon is formed. Dust means that the material of the abnormal area consists of dust.

[0056] Figure 9A(a) shows an example of an abnormal area image 61 including an abnormal area 61a indicating the remaining GL pattern. Figure 9A(b) shows an example of a label image 62 including a color label area 62a that is differentiated by hatching depending on whether the material of the abnormal area 61a is a gate layer.

[0057] Furthermore, Figure 9B(a) shows an example of an abnormal area image 63 including an abnormal area 63a that indicates the remaining SL pattern. Figure 9B(b) shows an example of a label image 64 including a color label area 64a that is differentiated by hatching depending on whether the material of the abnormal area 63a is a source layer.

[0058] Figure 9C(a) shows an example of an abnormal area image 65, which includes an abnormal area 65a showing the remaining poly-Si pattern. Figure 9C(b) shows an example of a label image 66, which includes a color label area 66a that is differentiated by hatching depending on whether the material of the abnormal area 65a is the layer on which polysilicon is formed.

[0059] Figure 9D(a) shows an example of an abnormal area image 67 including an abnormal area 67a indicating dust. Figure 9D(b) shows an example of a label image 68 including a color label area 68a that is differentiated by hatching according to the case where the material of the abnormal area 67a consists of dust.

[0060] Next, we will explain an example of a model training method for generating label images from abnormal area images, using Figures 10A and 10B.

[0061] Figure 10A shows an example of an image pair prepared as training data according to the embodiment. Figure 10B is a conceptual diagram showing a method for training a model using an image pair like the one shown in Figure 10A as training data.

[0062] The image pairs prepared as training data shown in Figure 10A consist of an image of an abnormal area and its label image. Figure 10A shows an image pair consisting of an image of an abnormal area and a label image generated from the abnormal area image by transforming (replacing) the region of the abnormal area with a color label corresponding to the material, i.e., the color of the abnormal part of the abnormal area image. Note that the color fluctuates, but by training with abnormal area images that have fluctuating colors, the transformation takes this into account.

[0063] Furthermore, Figure 10A(a) shows, as an example, an image of an abnormal area including an abnormal portion showing the remaining SL pattern and a corresponding label image as ground truth data. Figure 10A(b) shows, as an example, an image of an abnormal area including an abnormal portion showing the remaining GL pattern and a corresponding label image as ground truth data. Figure 10A(c) shows, as an example, an image of an abnormal area including an abnormal portion consisting of dust and a corresponding label image as ground truth data. Figure 10A(d) shows, as an example, an image of an abnormal area including an abnormal portion showing the remaining Poly-Si pattern and a corresponding label image as ground truth data.

[0064] Furthermore, when training a model to generate label images from abnormal area images, first, multiple image pairs consisting of abnormal area images and label images are prepared, as shown in Figure 10A.

[0065] Next, as shown in Figure 10B, the neural network constituting the model is supervised training so that the input is an anomaly image and the output is a corresponding label image. This allows us to obtain a model that generates a label image when an anomaly image is input.

[0066] Figure 10B shows an example where the model is composed of a fully connected 4-layer neural network, but it does not have to be 4 layers or fully connected. It may also be composed of a neural network such as a GAN (Generative Adversarial Network) that performs adversarial generation learning of image pairs, such as pix2pix. In other words, any neural network configuration that can produce a model that generates a labeled image when an anomaly image is input is acceptable.

[0067] Furthermore, image processing may be used as a method for generating the label image. More specifically, the label image generation unit 102 may generate a label image from the abnormal area image by converting the abnormal area into a color label corresponding to the material of the abnormal area as seen in the abnormal area image using image processing.

[0068] In this embodiment, the label image generation unit 102 first converts the abnormal area image to, for example, the HSV color space, and then extracts the area of ​​the abnormal part that appears in the abnormal area image based on its color. The label image generation unit 102 then converts (replaces) the area of ​​the abnormal part with a color label that corresponds to the material of the abnormal part, i.e., a color represented by a hue, saturation, and brightness within a certain range, thereby generating a label image from the abnormal area image. It should be noted that the HSV color space is used because it makes it easy to handle color with a single parameter, and therefore the abnormal area image is converted to the HSV color space, but this is not the only option. The same processing may be performed after converting to the RGB color space.

[0069] In this way, the label image generation unit 102 can generate a label image from the abnormal area image acquired by the image acquisition unit 101.

[0070] [1-2-3. Overlay image generation unit 103] The superimposed image generation unit 103 generates a superimposed image by superimposing the label image generated by the label image generation unit 102 with the normal pattern image. The normal pattern image is an image that includes a wiring pattern at a position corresponding to the abnormal part of the device 30, and where the abnormal part is absent. In other words, the normal pattern image is a pre-prepared image that is the same scale as the image of the region containing the abnormal part in the visual inspection image. The normal pattern image may be an image obtained from the visual inspection image, or it may be a pre-prepared image that is the same scale as the visual inspection image and depicts only the parts necessary for inspection.

[0071] Figure 11 shows an example of a normal pattern image corresponding to a pattern inspection image according to the embodiment. Figure 11(a) shows the pattern inspection image 41 of Figure 4, and Figure 11(b) shows a normal pattern image 44, which is an image with the same pattern as the pattern inspection image 41 and without any abnormal parts. In Figure 11(b), in order to reduce the processing load of subsequent overlay processing, a binarized normal pattern image of the wiring portion is shown so that only the wiring portion represented by wiring 1 to wiring 6 is shown. The normal pattern image 44 does not have to be binarized.

[0072] Figure 12 shows an example of a superimposed image according to the embodiment. In the example shown in Figure 12, a superimposed image 45 is shown, which is obtained by superimposing the label image 43 shown in Figure 8 and the normal pattern image 44 shown in Figure 11(b).

[0073] In other words, in this embodiment, the superimposed image generation unit 103 acquires a label image 43, for example, shown in Figure 8, which is generated by the label image generation unit 102. The superimposed image generation unit 103 also acquires a normal pattern image 44 from the visual inspection image of the device 30 acquired by the imaging device 20, which is an image with the same pattern as the pattern inspection image 41 and free of abnormal parts, by cropping or the like. The superimposed image generation unit 103 then generates a superimposed image 45 by superimposing the acquired label image 43 and the normal pattern image 44.

[0074] The superimposed image generation unit 103 needs to superimpose the wiring patterns of the color label portion in the label image with the wiring patterns in the normal pattern image without error. The method for superimposing the normal pattern image and the label image without error will be described below.

[0075] In this embodiment, first, the superimposed image generation unit 103 detects the repeating pattern (layout pattern) of the pattern inspection image that is the basis of the label image, and the position within that repeating pattern. The repeating pattern (layout pattern) and the position within that repeating pattern will be referred to as the phase below.

[0076] Next, the superimposed image generation unit 103 adjusts the phase of the normal pattern image to match the phase of the pattern inspection image that is the source of the label image, and then performs image processing to superimpose the normal pattern image and the label image.

[0077] The following describes an example of a phase detection method for the pattern inspection image that forms the basis of the label image. Phase detection methods include, for example, methods using difference images and methods using difference profiles.

[0078] Figure 13 is a diagram illustrating an example of a phase detection method using difference images according to the embodiment. Figure 14A is a diagram illustrating a difference image according to the embodiment. Figure 14B is a diagram illustrating an example of a histogram of a difference image according to the embodiment.

[0079] Figure 13(a) shows an example of a reference image 71, which is an image of a predetermined region where repeating patterns are easily identifiable and which has been acquired in advance. Figure 13(b) shows an example of a pattern inspection image 72. The reference image 71 is assumed to be the image of phase 0 (reference position of the repeating pattern) of the pattern inspection image 72.

[0080] More specifically, the superimposed image generation unit 103 raster scans the reference image 71, for example, using the crop position 73a of the pattern inspection image 72 as the starting position, and detects the position in the pattern inspection image 72 where the pattern inspection image 72 at the crop position 73a and the reference image 71 perfectly overlap. When the superimposed image generation unit 103 detects the position in the pattern inspection image 72 where the reference image 71 perfectly overlaps, it detects that position in the pattern inspection image 72 as having phase 0.

[0081] More specifically, the superimposed image generation unit 103 generates a difference image between the reference image 71 shown in, for example, Figure 14A(a) and the cropped image 73 shown in, for example, Figure 14A(b), which is cropped from the pattern inspection image 72 at the crop position 73a. The superimposed image generation unit 103 then calculates a histogram of the difference image at the raster-scanned crop position 73a, for example, as shown in Figure 14B, and determines that the crop position 73a where the variance of the calculated histogram is minimized is phase 0. This is because the crop position 73a where the variance of the difference image histogram is minimized can be considered the position of the pattern inspection image 72 where the cropped image 73 at that crop position 73a and the reference image 71 coincide (overlap perfectly).

[0082] Figure 15 is a diagram illustrating an example of a phase detection method using a difference profile according to the embodiment. Figure 16A is a diagram illustrating a difference profile according to the embodiment, and Figure 16B is a diagram illustrating an example of a histogram of a difference profile according to the embodiment.

[0083] Figure 15(a) shows an example of the luminance profile 71a of the X component of the reference image 71. Figure 15(b) shows an example of the luminance profile 72a of the X component of the pattern inspection image 72. The same applies to the luminance profile of the Y component, so only the luminance profile of the X component will be explained here.

[0084] More specifically, the superimposed image generation unit 103 performs a line scan of the luminance profile 71a of the X component of the reference image 71, using, for example, the crop position 74a of the luminance profile 72a of the X component of the pattern inspection image 72 as the starting position. The superimposed image generation unit 103 detects the position on the luminance profile 72a where the luminance profile 72a at the crop position 74a and the luminance profile 71a of the reference image 71 perfectly overlap. When the superimposed image generation unit 103 detects the position on the luminance profile 72a where the luminance profile 71a of the reference image 71 perfectly overlaps, it detects the position of the pattern inspection image 72 as having a phase of 0 in the X direction.

[0085] More specifically, the superimposed image generation unit 103 generates a difference profile between, for example, the luminance profile 71a of the reference image 71 shown in Figure 16A(a) and, for example, the crop profile 74 shown in Figure 16A(b). For example, Figure 16A(c) shows an example of a difference profile. Furthermore, for example, the crop profile 74 shown in Figure 16A(b) is the luminance profile of the X component cropped from the luminance profile 72a of the pattern inspection image 72 at the crop position 74a.

[0086] The superimposed image generation unit 103 then calculates a histogram of the difference profile at the line-scanned crop position 74a, as shown in Figure 16B, and determines that the crop position 74a with the minimum variance of the calculated histogram is the phase 0 of the X component. This is because the crop position 74a with the minimum variance of the difference profile histogram can be considered to be the position in the X direction of the pattern inspection image 72 where the crop profile 74 of that crop position 74a and the luminance profile 71a of the reference image 71 approximately coincide (overlap perfectly).

[0087] In this way, the superimposed image generation unit 103 can generate a superimposed image by superimposing the label image and the corresponding normal pattern image.

[0088] [1-2-4. Judgment section 104] The determination unit 104 uses the superimposed image generated by the superimposed image generation unit 103 to determine whether the abnormal portion is a device defect by determining the overlap between the color label portion and the wiring pattern without abnormal portions according to a predetermined decision tree. Here, when performing the overlap determination, the determination unit 104 may also determine whether the abnormal portion is short-circuiting the first and second wiring by determining the overlap between the color label and the first and second wiring included in the normal pattern image according to the decision tree. Alternatively, when performing the overlap determination, the determination unit 104 may also determine whether the abnormal portion is breaking the first wiring by determining the overlap between the color label and the first wiring included in the normal pattern image according to the decision tree.

[0089] In this embodiment, the determination unit 104 uses the superimposed image 45, for example shown in Figure 12, generated by the superimposed image generation unit 103, to determine whether the color label portion bridges the wiring, according to a predetermined decision tree. This allows the determination unit 104 to determine whether the color label portion is short-circuiting the wiring. If the color label portion is short-circuiting the wiring, the determination unit 104 determines that the abnormal portion is a device defect, and if the color label portion is not short-circuiting the wiring, it determines that the abnormal portion is not a device defect.

[0090] Figure 17 is a diagram showing an example of a flowchart illustrating a decision tree routine for performing a short circuit according to the embodiment. Figure 17 shows an example of a decision tree routine that performs a short circuit by performing an overlap detection using the superimposed image 45 shown in Figure 12.

[0091] In Figure 17, first, the determination unit 104 determines whether the abnormal portion 31b corresponding to the color label portion 31c is short-circuiting wiring 1 and wiring 2 (S1). More specifically, the determination unit 104 determines whether the color label portion 31c corresponding to the abnormal portion 31b is bridging wiring 1 and wiring 2 in the superimposed image 45 shown in Figure 12. This allows the determination unit 104 to determine whether the abnormal portion 31b is short-circuiting wiring 1 and wiring 2.

[0092] In step S1, if the abnormal portion 31b does not short-circuit wiring 1 and wiring 2 (no in step S1), the determination unit 104 determines whether the abnormal portion 31b is short-circuiting wiring 2 and wiring 3 (S2). More specifically, the determination unit 104 determines whether the abnormal portion 31b is short-circuiting wiring 2 and wiring 3 by making an overlapping determination in the superimposed image 45 shown in Figure 12 to see if the color label portion 31c is bridging wiring 2 and wiring 3.

[0093] In step S2, if the abnormal portion 31b does not short-circuit wiring 2 and wiring 3 (no in step S2), the determination unit 104 determines whether the abnormal portion 31b is short-circuiting wiring 3 and wiring 4 (S3). More specifically, the determination unit 104 determines whether the abnormal portion 31b is short-circuiting wiring 3 and wiring 4 by making an overlapping determination in the superimposed image 45 shown in Figure 12 to see if the color label portion 31c is bridging wiring 3 and wiring 4.

[0094] The determination unit 104 performs this determination for each wiring interval, and if the abnormal portion 31b does not short-circuit the wiring interval in all steps, it determines that the abnormal portion 31b is not a defect in the device 30 (S4).

[0095] On the other hand, if in steps S1, S2, S3, etc., the abnormal part 31b is causing a short circuit between the wires, it is determined that the abnormal part 31b is a defect in the device 30 (S5).

[0096] The overlap detection process used to determine short circuits may also be expressed using an algorithm. Below is an example of an algorithm for determining short circuits.

[0097] Figure 18 shows an example of a superimposed image used in an overlap detection algorithm for short circuit detection. The superimposed image 45a shown in Figure 18 is a simplified image in which wires 1 to 4 of the superimposed image 45 shown in Figure 12 are reduced to just two wires, wire 1 and wire 2.

[0098] The determination unit 104 performs the overlap detection algorithm described in 1) to 5) below on the superimposed image 45a shown in Figure 18.

[0099] 1) First, the determination unit 104 converts the regions (images) of wiring pattern 1, wiring pattern 2, and color label portion 31c in the superimposed image 45a into a two-dimensional array of numerical values, etc., represented by grayscale. Next, it fills the portion of the two-dimensional array corresponding to wiring pattern 1 with the numerical value "1", the portion corresponding to wiring pattern 2 with the numerical value "2", and the portion corresponding to color label portion 31c with the numerical value "3". Furthermore, it fills the portion corresponding to the area without a pattern with the numerical value "0". In this way, the determination unit 104 can replace the regions of the superimposed image 45a with a simple numerical array.

[0100] 2) Next, the determination unit 104 replaces the numerical value "1" in array 1 with the numerical value "5". This indicates that the area of ​​array 1 corresponding to the area of ​​wiring pattern 1 has been checked.

[0101] 3) Next, the determination unit 104, as PASS1, checks all the numerical values ​​"5" in the wiring 1 from the top left to the bottom right of the superimposed image 45a shown in Figure 18, and if the numerical values ​​"2" or "3" exist in the eight locations above, below, left, and right of the value, it replaces those locations with the numerical value "5".

[0102] 4) Next, the determination unit 104 determines that if the location corresponding to the number "2" filled in 1) has been replaced with the number "5", then wiring 1 and wiring 2 are short-circuited by the color label portion 31c.

[0103] 5) Then, the determination unit 104 performs the operations in 3) and 4) as follows: PASS2 from the bottom right to the top left of the superimposed image 45a, PASS3 from the bottom left to the top right of the superimposed image 45a, and PASS4 from the top right to the bottom left of the superimposed image 45a.

[0104] In addition, the above explanation described how to determine whether an abnormal portion causes a short circuit between wires by overlap detection according to a predetermined decision tree. However, the decision tree can be predetermined according to the structure of device 30, and can be predetermined based on the combination of wires that should be short-circuited in the structure of device 30.

[0105] Figures 19A to 19E are diagrams illustrating the connections between wires that are determined to be short-circuited in the SL (Source Layer) process according to a decision tree in the embodiment. The SL process is a manufacturing process before the completion of the device 30, in which the source layer is formed and the wiring etc. in the source layer is formed. Figures 19A to 19E show an example of the layer structure of the device 30 in the SL process. In the figure, the device 30 is formed in the order of a glass substrate, a gate insulating film, and an interlayer insulating film. Polysilicon is formed in some areas of the glass substrate and the gate insulating film, and the gate, source, and drain are formed on the gate insulating film and the interlayer insulating film on the polysilicon. Furthermore, the initialization power line INI, enable line EN, and power line Vcc are formed on the gate insulating film, and the reference voltage control line REF, reference power line Vref, scan line WS, and power line Vcc are formed on the interlayer insulating film.

[0106] In the SL process, the device 30 having such a layered structure will be short-circuited between the SL layers, for example, between the REF-Vref wiring shown in (i) of Figure 19A, between the REF-WS wiring shown in (ii) of Figure 19B, and between the WS-Vcc wiring shown in (iii) of Figure 19C. Furthermore, in the SL process, the device 30 will be short-circuited between the GL (Gate Layer) layers, for example, between the EN-Vcc wiring shown in (iv) of Figure 19D, and between the INI-EN wiring shown in (v) of Figure 19E.

[0107] In this way, a decision tree used for short circuit detection can be determined based on the combination of wirings determined by the structure of device 30.

[0108] Although it was explained that short circuit detection between the wirings of device 30 is performed in the SL process, this is not the only option. Short circuit detection between the wirings of device 30 may also be performed in the GL process. In this case, the decision tree used for short circuit detection should be determined according to the combination of wirings determined by the structure of device 30 in the GL process.

[0109] [2. Operation of the inspection device 10] An example of the operation of the inspection device 10 configured as described above will be explained below.

[0110] Figure 20 is a flowchart illustrating the operation overview of the inspection device 10 in the embodiment. Figure 21 is an explanatory diagram illustrating an example of the operation of the inspection device 10 in the embodiment.

[0111] First, the inspection device 10 acquires an abnormal area image including the abnormal portion (S10). More specifically, the inspection device 10 acquires an abnormal area image, which is an image including the abnormal portion of the wiring pattern of the device 30 obtained by image processing from the visual inspection image of the device 30. For example, the inspection device 10 acquires a pattern inspection image 81, which includes the abnormal portion 81a of the remaining pattern shown in Figure 21(a), as the visual inspection image of the device 30 in the SL process. The inspection device 10 applies image processing using the background subtraction method to the acquired pattern inspection image 81 to generate and acquire an abnormal area image 82, which includes the abnormal portion 82a shown in Figure 21(b).

[0112] Next, the inspection device 10 generates a label image (S11). More specifically, the inspection device 10 generates a label image in which the abnormal portion is converted into a color label based on the layer in which the abnormal portion is located, from the abnormal portion image acquired in step S10. For example, the inspection device 10 generates a label image 83 shown in Figure 21(c) from an abnormal portion image 82 showing the abnormal portion 82a shown in Figure 21(b), which includes a color label portion 83a in which the abnormal portion 82a is converted into a color label corresponding to the material of the abnormal portion 82a. As mentioned above, there are two methods for generating the label image 83: one using a trained model such as a generative model composed of a neural network, and another using image processing based on the color of the abnormal portion 82a.

[0113] Next, the inspection device 10 generates an image by superimposing the label image and the normal pattern image (S12). More specifically, the inspection device 10 generates a superimposed image by superimposing the label image generated in step S11 with a normal pattern image that includes a wiring pattern corresponding to the abnormal part of the device 30, but without the abnormal part. For example, the inspection device 10 detects the phase of the pattern inspection image 81 that includes the abnormal part 81a which is the basis of the label image 83 shown in Figure 21(c), and adjusts the phase of the normal pattern image 84 shown in Figure 21(d) to match the detected phase. The phases of the normal pattern image 84 and the pattern inspection image 81 refer to the repeating pattern (layout pattern) in the normal pattern image 84 and the pattern inspection image 81, and the position within that repeating pattern, as described above. Then, the inspection device 10 generates the superimposed image 85 shown in Figure 21(e) by performing image processing to superimpose the phase-matched (adjusted) normal pattern image 84 and the label image 83.

[0114] Next, the inspection device 10 determines, according to the decision tree, whether the abnormal portion is a device defect from the superimposed image generated in step S12 (S13). More specifically, the inspection device 10 uses the superimposed image generated in step S12 to determine whether the abnormal portion is a device defect by determining the overlap between the color label portion and the wiring pattern without the abnormal portion, according to a predetermined decision tree. Below, an example of the operation of short-circuit detection by overlap detection on the superimposed image 85 shown in Figure 21 will be explained using Figure 22.

[0115] Figure 22 is a flowchart illustrating an example of overlap detection operation for the superimposed image 85 shown in Figure 21. Figure 22 shows an example of overlap detection operation for the device 30 in the SL process. In the example shown in Figure 21, the color label portion 83a is represented by hatching that indicates the color corresponding to the case where the material of the abnormal portion 82a is the source layer. In Figure 22, this color label portion 83a is referred to as the SL label.

[0116] In steps S131, S132, S133, etc. in Figure 22, the inspection device 10 determines whether the SL label bridges between the wirings. The device 30 in the SL process has signal lines Sig, reference voltage control lines ref, power lines Vcc, initialization power lines Ini, etc. Step S131 shows an example of determining whether the SL label bridges between the Sig-ref wirings or if there is overlap. Similarly, steps S132 and S133 show examples of determining whether the SL label bridges between the Sig-Vcc wirings and between the Sig-ini wirings or if there is overlap. If the SL label does not bridge between the wirings in steps S131, S132, S133, etc. (no in steps S131, S132, and S133), the inspection device 10 determines OK, that is, that the abnormal portion 82a is not a defect in the device 30 (step S134).

[0117] On the other hand, if the SL label bridges between wirings in steps S131, S132, S133, etc. (yes in steps S131, S132, S133), it may be determined that NG, i.e., the abnormal portion 82a, is a defect in the device 30 (step S137). In the example shown in Figure 22, laser repair is possible in certain cases even when the SL label bridges between wirings, so an example of operation including the determination of whether laser repair is possible is shown. That is, the area of ​​the SL label is a predetermined size of αμm 2 If the above conditions are not met and the SL label is not bridging over the 3 wiring (no in steps S135 and S136), it is determined that laser repair is possible (step S138). This prevents the abnormal portion 82a from becoming a defect in the device 30.

[0118] [3. Effects, etc.] According to the inspection device 10 of this embodiment, an abnormal area image is obtained (or generated) by extracting only the abnormal parts from the pattern inspection image using background subtraction image processing, and a label image is generated from the abnormal area image by converting the abnormal parts into color label parts that are color-coded according to their material. Furthermore, according to the inspection device 10 of this embodiment, a normal pattern image and a label image are superimposed, and by determining whether the abnormal parts are causing defects in the device (product) by determining the overlap between the normal wiring pattern and the color label parts of the label image according to a decision tree.

[0119] In this way, the inspection device 10 of this embodiment can automatically determine whether an abnormal portion of the wiring pattern of a device having a laminated structure with one or more layers on which patterns are formed is linked to a defect. In other words, the inspection device 10 of this embodiment can automatically determine whether a pattern abnormality detected by pattern inspection constitutes a product defect.

[0120] In this embodiment, the inspection device 10 and the like can be used to generate a label image by using a pre-trained model such as a generative model composed of a neural network, or by using image processing based on the color of the abnormal area.

[0121] In the method using a pre-trained model, a neural network model that excels at recognition and completion can be used to automatically generate label images with high accuracy. In this case, the inspection device 10 of this embodiment generates label images using a model that excels at recognition and completion, while entrusting simple overlap detection, which the model is not good at, to image processing, and using a decision tree with clear rules to determine whether the abnormal portion leads to a product defect. As a result, the method has high judgment accuracy and clear judgment rules, making it possible to automatically determine whether the abnormal portion of the wiring pattern of the device is linked to a defect.

[0122] Furthermore, unless the manufacturing process changes significantly, the recognition and correction of abnormal parts are independent of the product type. Therefore, if a decision tree is prepared for each product type that handles the determination of whether the abnormal part leads to a product defect or not, it has the effect of eliminating the need to train the most difficult part, the model, for each product type. In other words, the inspection device 10 of this embodiment can be easily deployed for multiple product types, and it is possible to automatically determine whether the abnormal part of the wiring pattern of a device in multiple product types leads to a defect or not.

[0123] Furthermore, in methods that use image processing based on the color of abnormal areas, the label image is generated by utilizing the property that a specific "material" appears at a specific location, thus minimizing the impact of color variations due to process fluctuations.

[0124] (modified version) In the above embodiment, the abnormal portion was described as a portion where the pattern remains, but this is not the only case. The abnormal portion may be a portion where the pattern is missing. Here, an abnormal portion with a missing pattern is a pattern portion in which part of the formed wiring pattern is missing.

[0125] The operation of the inspection device 10 when the abnormal area is a pattern missing is as shown in Figure 20, except that the decision tree used for overlap detection is a break detection rather than a short circuit detection. Below, an example of the operation of the inspection device 10 when the abnormal area is a pattern missing will be explained using Figure 20. The following explanation will focus on the differences from the embodiment described above.

[0126] Figure 23 is an explanatory diagram illustrating an example of the operation of the inspection device 10 in a modified form.

[0127] In Figure 20, first, the inspection device 10 acquires an abnormal part image including the abnormal portion (S10). In this modified example, for example, the inspection device 10 acquires a pattern inspection image 91 including the abnormal portion 91a of the pattern defect shown in Figure 23(a) as an appearance inspection image of the device 30 in the SL process. The inspection device 10 applies image processing using the background subtraction method to the acquired pattern inspection image 91 to generate and acquire an abnormal part image 92 including the abnormal portion 92a shown in Figure 23(b).

[0128] Next, the inspection device 10 generates a label image (S11). In this modified example, for example, the inspection device 10 generates a label image 93 shown in Figure 23(c) from the abnormal part image 92, which includes the abnormal part 92a shown in Figure 23(b), and includes a color label portion 93a obtained by converting the abnormal part 92a into a color label corresponding to the material of the abnormal part 92a. As mentioned above, there are two methods for generating the label image 93: one using a trained model such as a generative model composed of a neural network, and another using image processing based on the color of the abnormal part 92a.

[0129] Next, the inspection device 10 generates an image by superimposing the label image and the normal pattern image (S12). In this modified example, for example, the inspection device 10 detects the phase of the pattern inspection image 91, which includes the abnormal portion 91a that forms the basis of the label image 93 shown in Figure 23(c), and adjusts the phase of the normal pattern image 94 shown in Figure 23(d) to match the detected phase. Then, the inspection device 10 generates the superimposed image 95 shown in Figure 23(e) by performing image processing to superimpose the phase-matched (adjusted) normal pattern image 94 and the label image 93.

[0130] Next, the inspection device 10 determines, according to the decision tree, whether the abnormal portion is a device defect from the superimposed image generated in step S12 (S13). More specifically, the inspection device 10 uses the superimposed image generated in step S12 to determine whether the abnormal portion is a device defect by determining the overlap between the color label portion and the wiring pattern without the abnormal portion according to a predetermined decision tree. Below, an example of the operation of wire break detection by overlap detection on the superimposed image 95 shown in Figure 23 will be explained using Figure 24.

[0131] Figure 24 is a flowchart illustrating an example of overlap detection operation for the superimposed image 95 shown in Figure 23. Figure 24 shows an example of overlap detection operation for the device 30 in the SL process. In the example shown in Figure 23, the color label portion 93a is represented by hatching that indicates the color corresponding to the case where the material of the abnormal portion 92a is the source layer. In Figure 24, this color label portion 93a is referred to as the SL label. If the material of the abnormal portion is the gate layer, the color label portion is represented by hatching that indicates the color corresponding to the material of the abnormal portion being the gate layer. In this case, the color label portion is referred to as the GL label. Similarly, if the material of the abnormal portion is a contact hole, the color label portion is represented by hatching that indicates the color corresponding to the material of the abnormal portion being the contact hole. In this case, the color label portion is referred to as the CH label.

[0132] In Figure 24, the inspection device 10 first determines whether the GL label is interfering with the Sig line (signal line wiring) by overlapping (S231).

[0133] In step S231, if it is determined that the GL label is interrupting the Sig line (signal line wiring) (yes in step S231), it is determined that the abnormal part is a defect in device 30, i.e., S232. Since this is an overlap check for device 30 in the SL process, a break in the wiring in the GL layer formed below the SL layer is irreparable, so it is determined to be NG.

[0134] Furthermore, in step S233, the inspection device 10 determines whether the SL label is interrupting the Sig line. If it is determined in step S233 that the SL label is interrupting the Sig line (yes in step S233), then the area of ​​the SL label is determined to be a predetermined size of αμm 2 Determine if the above is true (S234). If, in step S233, it is determined that the SL label is interrupting the Sig line (yes in step S233), it may be determined that NG, i.e., the abnormal portion 92a is a defect in the device 30.

[0135] In step S234, if the area of ​​the SL label is greater than or equal to a predetermined size (yes in step S234), it is determined that NG, i.e., the abnormal portion 92a is a defect in the device 30 (S232). On the other hand, in step S234, if the area of ​​the SL label is not greater than or equal to a predetermined size (no in step S234), it is determined that it can be repaired by complex modification (S235). This prevents the abnormal portion 92a from becoming a defect in the device 30. This is because, even if the SL label is interrupting the wiring, if the area of ​​the SL label is smaller than a predetermined size, the SL label is in the SL layer and can be repaired using the complex metal. In the example operation shown in Figure 24, an exception is made when determining NG, in which case it is further determined whether or not complex repair is possible.

[0136] In step S236, the inspection device 10 determines whether the CH label is located within the Sig line. If the CH label is located within the Sig line in step S236 (yes in step S236), it is determined that NG, i.e., the abnormal portion 92a, is a defect in the device 30 (S232).

[0137] Here, if the judgment result in steps S231, S233, S236, etc. is no (no in steps S231, S233, S236, etc.), the inspection device 10 determines OK, that is, that the abnormal part 92a is not a defect in the device 30 (step S237).

[0138] As described above, the inspection device 10 of this modified example can automatically determine whether an abnormal portion of the wiring pattern of a device having a laminated structure with one or more layers on which patterns are formed constitutes a defect, even if that abnormal portion is a pattern gap. In other words, it can automatically determine whether a pattern abnormality detected by pattern inspection constitutes a product defect.

[0139] (Other embodiments) The inspection apparatus and determination method related to this disclosure have been described above based on each embodiment, but this disclosure is not limited to these embodiments. As long as they do not deviate from the spirit of this disclosure, various modifications that a person skilled in the art can conceive of will be applied to each embodiment, and other forms constructed by combining some of the components of each embodiment are also included within the scope of this disclosure.

[0140] Furthermore, the following forms may also be included within the scope of one or more aspects of this disclosure.

[0141] (1) Some of the components constituting the above determination device may be a computer system consisting of a microprocessor, ROM, RAM, GPU, hard disk unit, display unit, keyboard, mouse, etc. A computer program is stored in the RAM or hard disk unit. The microprocessor achieves its function by operating in accordance with the computer program. Here, the computer program is composed of a combination of multiple instruction codes that indicate commands to the computer in order to achieve a predetermined function.

[0142] (2) Some of the components constituting the above determination device may be made up of a single system LSI (Large Scale Integration). The system LSI is a multi-functional LSI manufactured by integrating multiple components onto a single chip, and specifically, it is a computer system that includes a microprocessor, ROM, RAM, GPU, etc. A computer program is stored in the RAM. The system LSI achieves its function by operating the microprocessor or the GPU in accordance with the computer program.

[0143] (3) Some of the components constituting the determination device described above may consist of an IC card or a standalone module that can be attached to or removed from each device. The IC card or module is a computer system consisting of a microprocessor, ROM, RAM, GPU, etc. The IC card or module may include the above-mentioned multi-functional LSI. The IC card or module achieves its function by operating the microprocessor or GPU according to a computer program. The IC card or module may be tamper-resistant.

[0144] (4) Furthermore, some of the components constituting the above determination device may be the computer program or the digital signal recorded on a recording medium that can be read by a computer, such as a flexible disk, hard disk, CD-ROM, MO, DVD, DVD-ROM, DVD-RAM, BD (Blu-ray® Disc), semiconductor memory, etc. Alternatively, the digital signal may be recorded on one of these recording media.

[0145] Furthermore, some of the components constituting the above-described determination device may transmit the computer program or the digital signal via telecommunications lines, wireless or wired communication lines, networks such as the Internet, data broadcasting, etc.

[0146] (5) The disclosure may also be the methods described above. Alternatively, it may be a computer program that implements these methods using a computer, or a digital signal consisting of the computer program.

[0147] (6) The Disclosure also relates to a computer system comprising a microprocessor, a GPU, and memory, wherein the memory stores the computer program, and the microprocessor or the GPU operates in accordance with the computer program.

[0148] (7) Alternatively, the program or the digital signal may be carried out by another independent computer system by recording it on the recording medium and transferring it, or by transferring the program or the digital signal via the network or the like.

[0149] (8) In addition, some of the components that make up the above determination device may be performed on a cloud or server device.

[0150] (9) The above embodiments and the above modifications may be combined. [Industrial applicability]

[0151] This disclosure can be used in inspection methods, inspection devices, and programs that can automatically determine whether abnormal parts of wiring patterns found through visual inspection in the manufacturing process of devices such as liquid crystal devices and organic EL displays are related to product defects. [Explanation of symbols]

[0152] 10 Inspection equipment 20 Imaging device 21 stages 22 Stage drive unit 30 devices 31a, 31b, 32a, 32b, 61a, 63a, 65a, 67a, 81a, 82a, 91a, 92a Abnormal part 31c, 62a, 64a, 66a, 68a, 83a, 93a Color label portion Pattern inspection images 41, 51, 72, 81, 91 Images of abnormal areas: 42, 53, 61, 63, 65, 67, 82, 92 Label images 43, 62, 64, 66, 68, 83, 93 44, 52, 84, 94 Normal pattern images 45, 45a, 85, 95 superimposed images 71 Reference Images 71a, 72a Brightness Profiles 73 cropped images 73a, 74a Crop position 74 Crop Profiles 101 Image acquisition unit 102 Label Image Generation Unit 103 Overlay image generation unit 104 Judgment section 1000 computers 1001 Input Device 1002 Output device 1003 CPU 1004 Internal Storage 1005 RAM 1006 GPU 1007 Reader 1008 Transceiver 1009 Bus

Claims

1. A method for inspecting a device having a laminated structure with one or more layers on which a pattern is formed, An acquisition step of obtaining an abnormal part image, which is an image including an abnormal part of the wiring pattern of the device obtained by image processing from an external inspection image of the device; A generation step of generating a label image in which the abnormal portion is converted from the abnormal portion image into a color label based on the layer in which the abnormal portion is located, The method includes a determination step of determining whether the abnormal portion is a defect in the device by using a superimposed image obtained by superimposing the label image and a normal pattern image that includes a wiring pattern without the abnormal portion at a position corresponding to the abnormal portion of the device, and performing an overlap determination between the color label and the wiring pattern without the abnormal portion according to a predetermined decision tree, thereby determining whether the abnormal portion is a defect in the device. In the acquisition step described above, The abnormal part image is obtained by applying image processing using the background subtraction method to the visual inspection image of the device and generating the abnormal part image. In the above generation step, Using a pre-trained model, which is a neural network, the label image is generated from the abnormal image. Prior to the aforementioned generation step, The aforementioned trained model is trained using image processing of abnormal parts of the device, which are prepared as training data, and label images in which the abnormal parts shown in the abnormal part image are converted into different color labels depending on the material of the abnormal part shown in the abnormal part image. In the above generation step, To compensate for the loss of the abnormal portion due to background difference in the aforementioned abnormal portion image, Testing method.

2. The normal pattern image is a pre-prepared image that is the same scale as the image of the region containing the abnormal part from the visual inspection image. In the determination step, when performing the overlap determination, the system determines whether the abnormal portion is short-circuiting the first and second wiring by determining the overlap between the color label and the first and second wiring included in the normal pattern image according to the decision tree. The inspection method according to claim 1.

3. The normal pattern image is a pre-prepared image that is the same scale as the image of the region containing the abnormal part from the visual inspection image. In the determination step, when performing the overlap determination, the system determines whether the abnormal portion is causing a break in the first wiring by determining whether the color label and the first wiring included in the normal pattern image overlap according to the decision tree. The inspection method according to claim 1.

4. The abnormal portion is a pattern portion in which part of the formed wiring pattern is missing. The inspection method according to claim 1 or 3.

5. The abnormal portion is a pattern portion in which a wiring pattern remains in a part of the formed pattern where a wiring pattern should not be formed. The inspection method according to claim 1 or 2.

6. In the above generation step, From the image of the abnormal part, the label image is generated by converting the abnormal part into a color label corresponding to the material of the abnormal part as seen in the image of the abnormal part, based on the layer in which the abnormal part is located, using image processing. The inspection method according to any one of claims 1 to 5.

7. An inspection apparatus for a device having a laminated structure with one or more layers on which a pattern is formed, An image acquisition unit acquires an abnormal portion image, which is an image including an abnormal portion of the wiring pattern of the device obtained by image processing from an external inspection image of the device. A label image generation unit generates a label image in which the abnormal portion is converted from the abnormal portion image into a color label based on the layer in which the abnormal portion is located, The device comprises a determination unit that determines whether the abnormal portion is a defect in the device by using a superimposed image obtained by superimposing the label image and a normal pattern image that includes a wiring pattern without the abnormal portion at a position corresponding to the abnormal portion of the device, and performing an overlap determination between the color label and the wiring pattern without the abnormal portion according to a predetermined decision tree, thereby determining whether the abnormal portion is a defect in the device. The image acquisition unit, The abnormal part image is obtained by applying image processing using the background subtraction method to the visual inspection image of the device and generating the abnormal part image. The label image generation unit, Using a pre-trained model, which is a neural network, the label image is generated from the abnormal image. The aforementioned trained model is trained using image processing of abnormal parts of the device, which are prepared as training data, and label images in which the abnormal parts shown in the abnormal part image are converted into different color labels depending on the material of the abnormal part shown in the abnormal part image. The label image generation unit, To compensate for the loss of the abnormal portion due to background difference in the aforementioned abnormal portion image, Inspection device.

8. A program for causing a computer to perform an inspection method for a device having a stacked structure with one or more layers on which a pattern is formed, The aforementioned inspection method is An acquisition step of obtaining an abnormal part image, which is an image including an abnormal part of the wiring pattern of the device obtained by image processing from an external inspection image of the device; A generation step of generating a label image in which the abnormal portion is converted from the abnormal portion image into a color label based on the layer in which the abnormal portion is located, The method includes a determination step of determining whether the abnormal portion is a defect in the device by using a superimposed image obtained by superimposing the label image and a normal pattern image that includes a wiring pattern without the abnormal portion at a position corresponding to the abnormal portion of the device, and performing an overlap determination between the color label and the wiring pattern without the abnormal portion according to a predetermined decision tree, thereby determining whether the abnormal portion is a defect in the device. In the acquisition step described above, The abnormal part image is obtained by applying image processing using the background subtraction method to the visual inspection image of the device and generating the abnormal part image. In the above generation step, Using a pre-trained model, which is a neural network, the label image is generated from the abnormal image. Prior to the aforementioned generation step, The aforementioned trained model is trained using image processing of abnormal parts of the device, which are prepared as training data, and label images in which the abnormal parts shown in the abnormal part image are converted into different color labels depending on the material of the abnormal part shown in the abnormal part image. In the above generation step, To compensate for the loss of the abnormal portion due to background difference in the aforementioned abnormal portion image, program.

Images