Image processing device, defect detection system, image processing method, and image processing program

Abstract

To provide an image processing device and the like which generate learning data for generating a highly accurate defect detection model by machine learning to improve defect detection accuracy.SOLUTION: An image processing device 30 comprises: an image acquisition unit 31 which acquires an image in RGB colors obtained by imaging a region including a defective region of a sample; a color space conversion unit 32 which converts an RGB color space into an HSV color space for the acquired image; a range setting unit 33 which sets a range of a value of each of HSV for the defective region in the converted image; and a first color conversion unit 34 which converts a pixel within the set range of the value of each of HSV into first color information whose color contrast value is equal to or greater than a threshold relative to a background color in the image.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an image processing apparatus, a defect detection system, an image processing method, and an image processing program. [Background technology]

[0002] With the miniaturization of semiconductor process nodes, there is an urgent need for even higher sensitivity in the inspection of wafers, masks, and other components. For example, die-to-die inspection and mask-to-mask inspection are known methods for inspecting foreign objects in masks. In such inspections, techniques are being researched and developed to detect defects using machine learning, such as deep learning, with images that have been recognized as defects. [Prior art documents] [Patent Documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2021-124746 [Patent Document 2] Japanese Patent Publication No. 2021-149952 [Overview of the project] [Problems that the invention aims to solve]

[0004] However, even with defect detection techniques using deep learning, false detections can occur due to variations in imaging conditions. Through diligent research, the inventors discovered that in the inspection of defects in wafers and masks, false detections are mainly caused by factors such as brightness and color among the imaging conditions. In other words, they found that differences in brightness and color occur between the images used in machine learning and the images captured in the inspection process, which can lead to false detection of defects.

[0005] The object of this invention is to solve these problems and to provide an image processing device, etc., that generates training data for generating a highly accurate defect detection model using machine learning (deep learning), and as a result improves defect detection accuracy. [Means for solving the problem]

[0006] An image processing apparatus according to one aspect of this embodiment is: An image acquisition unit that acquires an RGB color image of the region containing the defect area of ​​the sample, A color space conversion unit converts the acquired image from the RGB color space to the HSV color space, A range setting unit sets the range of each HSV value for the defect region in the converted image, In the aforementioned image, a first color conversion unit converts pixels within the range of each set HSV value into first color information in which the color contrast value with respect to the background color is equal to or greater than a threshold, It is equipped with.

[0007] A defect detection system according to one aspect of this embodiment is: An imaging optical system for imaging the sample, An image processing device that processes the image captured by the aforementioned imaging optical system, Equipped with, The aforementioned image processing device is An image acquisition unit that acquires an RGB color image of the region containing the defect area of ​​the sample, A color space conversion unit converts the acquired image from the RGB color space to the HSV color space, A range setting unit sets the range of each HSV value for the defect region in the converted image, In the aforementioned image, a first color conversion unit converts pixels within the range of each set HSV value into first color information in which the color contrast value with respect to the background color is equal to or greater than a threshold, It is equipped with.

[0008] An image processing method according to one aspect of this embodiment is: An RGB color image that images a region including a defect region of a sample is acquired. For the acquired image, the RGB color space is converted into the HSV color space. For the defect region in the converted image, a range of each value of HSV is set. In the image, pixels within the set range of each value of HSV are converted into first color information with a color contrast value with respect to the background color being equal to or greater than a threshold value.

[0009] An image processing program according to one aspect of the present embodiment acquires an RGB color image that images a region including a defect region of a sample. For the acquired image, the RGB color space is converted into the HSV color space. For the defect region in the converted image, a range of each value of HSV is set. In the image, the computer is caused to execute an instruction to convert pixels within the set range of each value of HSV into first color information with a contrast value with respect to the background color being equal to or greater than a threshold value.

Effect of the Invention

[0010] According to the present invention, it is possible to provide an image processing apparatus or the like that generates learning data for generating a highly accurate defect detection model by machine learning.

Brief Description of the Drawings

[0011] [Figure 1] It is a block diagram illustrating an image processing apparatus according to Embodiment 1. [Figure 2] It is a diagram for explaining image processing according to Embodiment 1. [Figure 3] It is a diagram for explaining the configuration of a defect detection system according to Embodiment 2. [Figure 4] It is a diagram for explaining color conversion processing of an image according to Embodiment 2. [Figure 5] It is a flowchart for explaining an image processing method according to Embodiment 2. [Figure 6]This is a conceptual diagram illustrating the defect detection method according to Embodiment 2. [Figure 7] This diagram illustrates the overall operation of the defect detection system according to Embodiment 2. [Figure 8] This diagram illustrates the configuration of the defect detection device according to Embodiment 2. [Modes for carrying out the invention]

[0012] The specific configuration of this embodiment will be described below with reference to the drawings. The following description illustrates preferred embodiments of the present invention, and the scope of the present invention is not limited to these embodiments. In the following description, the same reference numerals indicate substantially the same function.

[0013] <Embodiment 1> Figure 1 is a block diagram illustrating an image processing device according to Embodiment 1. Figure 2 is a diagram illustrating the image processing according to Embodiment 1. The image processing device 30 can be used to generate training data for machine learning. The image processing device 30 can be implemented by a computer equipped with a GPU (Graphics Processing Unit) and memory, etc. In a specific example, as shown in Figure 1, the image processing device 30 includes an image acquisition unit 31, a color space conversion unit 32, a range setting unit 33, and a first color conversion unit 34.

[0014] The image acquisition unit 31 acquires an RGB color image of the region of the sample that includes the defective area. For example, in a semiconductor wafer inspection process, it acquires one or more color images of semiconductor wafers that have been recognized as defective. The acquired image may be an image of the entire wafer or an image of only a portion of the wafer.

[0015] The color space conversion unit 32 converts the acquired image from the RGB color space to the HSV color space. While it is possible to determine the hue and brightness within an image using an RGB image, determining the range of colors from light to dark, or from dark to bright, is difficult with an RGB image because the resulting code is discrete. Therefore, the color space conversion unit 32 is used to convert the acquired RGB color image into the HSV image format.

[0016] The range setting unit 33 sets the range of each HSV value for the defective region in the image converted to the HSV color space. For example, in some embodiments, the operator can set the range (upper and lower limits) of each HSV value by operating the control unit of the image processing device 30 (e.g., an input device such as a mouse or keyboard). It is desirable to set the range (upper and lower limits) for all of H, S, and V, rather than just the values ​​for one of them. Alternatively, in other embodiments, a predetermined range (plus or minus threshold) of HSV may be automatically set based on each HSV value (e.g., maximum value, minimum value, average value, etc.) within the defective region of the acquired image.

[0017] The first color conversion unit 34 converts pixels within the range of each set HSV value into first color information such that the color contrast with respect to the background color (for example, any single color if there are multiple background colors) is greater than or equal to a threshold. As used herein, "color contrast" refers to the difference in brightness or sharpness, and represents the difference between the DOI and the HSV values ​​of the background color. Specifically, the color contrast value is: H DOI -H 背景 , S DOI -S 背景 , V DOI -V 背景 This can be defined by:

[0018] If all HSV values ​​of a pixel in an image are within the set upper and lower limits of HSV, it is converted to primary color information. Conversely, if any of the HSV values ​​of a pixel in an image are outside the set upper and lower limits of HSV, it is not converted to primary color information. Therefore, as shown in Figure 2, specific defect areas in an image are converted to specific colors (e.g., primary colors such as red and green). Defect areas may be called DOIs (Defect of Interest).

[0019] By applying color transformation to the DOI in this way, the color contrast between the DOI and the background color is enhanced compared to before the transformation. By using images that have undergone this kind of image processing as training data for machine learning, it is possible to generate a highly accurate defect detection model.

[0020] Other variations The image processing device 30 may further include a second color conversion unit. The second color conversion unit converts pixels outside the range of each set HSV value into second color information such that the color contrast value with the first color information is equal to or greater than a threshold. Background areas other than defect areas can be converted to achromatic (grayscale). In other embodiments, background areas other than defect areas may be converted to a different color (e.g., white, black) than the specific color of the first color conversion unit 34. In some embodiments, the color contrast between the converted DOI and the converted background area can be maximized.

[0021] In this way, training data used for machine learning of defect detection models can be generated. Machine learning algorithms include nearest neighbors, naive Bayes, and support vector machines. Deep learning, which uses neural networks such as CNNs (Convolutional Neural Networks) or RNNs (Recurrent Neural Networks) to generate features and connection weights for learning, can also be used. Deep learning is a type of machine learning that uses multi-layered neural networks. By learning network parameters (weights and biases) from a large number of training images and their corresponding target images (desired outputs), it becomes possible to make highly accurate estimations even for unknown input images. Furthermore, GPUs are suitable for performing efficient calculations by processing data in parallel, making them ideal for multiple training iterations using learning models like deep learning.

[0022] As demonstrated in this modified example, by using images processed in this way as training data for machine learning, it is possible to generate a more accurate defect detection model. Furthermore, it is possible to reduce the variation in brightness between images after color conversion.

[0023] <Embodiment 2> Figure 3 is a diagram illustrating a defect detection system according to Embodiment 2. As shown in Figure 3, the defect detection system 1 comprises an imaging optical system 10, an image processing device 30a, a display device 40, and a stage 50. Here, for the convenience of explaining the defect detection system 1, an XYZ Cartesian coordinate system is introduced. The direction perpendicular to the stage surface 51 of the stage 50 is called the Z-axis direction, and the plane perpendicular to the Z-axis direction is called the XY plane. For example, the +Z-axis direction is called upward, and the -Z-axis direction is called downward. Note that upward and downward are for the convenience of explanation and do not indicate the actual orientation of the defect detection system 1.

[0024] The imaging optical system 10 images the sample 60 placed on the stage surface 51. The imaging optical system 10 includes a light source 11, a beam splitter 12, an objective lens 13, and a detector 14. Note that the imaging optical system 10 shown in Figure 3 is simplified as appropriate. The imaging optical system 10 may also be equipped with optical elements other than those described above, such as lenses, optical scanners, mirrors, filters, beam splitters, etc. For example, the imaging optical system 10 may be a confocal optical system.

[0025] Light source 11 generates illumination light L11. Light source 11 can be a lamp light source, an LED (Light Emitting Diode) light source, a laser light source, etc. Illumination light L11 can be, for example, EUV (Extreme Ultraviolet) light. Note that illumination light L11 is not limited to EUV light, but can also be UV light or visible light. Illumination light L11 from light source 11 is incident on beam splitter 12. Beam splitter 12 is, for example, a half mirror and reflects approximately half of the illumination light L11 in the direction of the sample 60. Illumination light L11 reflected by beam splitter 12 is incident on objective lens 13. The optical axis OX of objective lens 13 is, for example, parallel to the Z axis direction. The objective lens 13 focuses the illumination light L11 onto the sample 60. This allows the sample 60 to be illuminated.

[0026] A sample 60 to be imaged is placed on the stage 50. On the stage surface 51, the sample 60 is held parallel to the XY plane. The thickness direction of the sample 60 is in the Z-axis direction. The stage 50 is a three-dimensional drive stage having a drive mechanism 52. The image processing device 30a controls the drive mechanism 52 to drive the stage 50 in the XYZ axis direction.

[0027] The sample 60 may be, for example, a photomask or semiconductor wafer on which a fine pattern 61 is formed, and is held on the stage surface 51. The pattern 61 of the sample 60 may be, for example, a light-shielding film.

[0028] The reflected light L12 reflected from the sample 60 is focused by the objective lens 13 and incident on the beam splitter 12. The beam splitter 12 transmits approximately half of the reflected light L12. The reflected light L12 that has passed through the beam splitter 12 is incident on the detector 14. This allows the detector 14 to image the sample 60. The objective lens 13 magnifies and projects the image of the sample 60 onto the detector 14. In addition, a lens or the like may be provided to focus the reflected light L12 onto the light-receiving surface of the detector 14.

[0029] In Figure 3, the defect detection system 1 is shown as a bright-field illumination microscope, but the illumination method of the defect detection system 1 is not particularly limited.

[0030] The detector 14 has an image sensor for imaging the sample 60. The detector 14 is, for example, a TDI (Time Delay Integration) sensor. The detector 14 may also be a CCD (Charge Coupled Device) camera or a CMOS (Complementary Metal Oxide Semiconductor) sensor, as long as it detects reflected light L12 from the sample 60 illuminated by illumination light L11. The detector 14 may have multiple pixels arranged in the X-axis direction. The detector 14 may also be a line sensor in which multiple pixels are arranged in a single row.

[0031] The various images captured by the detector 14 may exhibit differences in brightness and color due to various factors such as the quality of the stage 50 and the thickness of the film deposited on the wafer. If images with such differences in brightness and color are used as training data for machine learning such as deep learning, a training model with low defect detection accuracy may be generated. Therefore, even if the specified DOI image has a certain range of variation in color, vividness, and brightness, the observed defect image is converted to a specific fixed color within that range of variation. Details will be described later in relation to Figures 4 and 5.

[0032] Stage 50 is a drive stage that can move the sample 60 in the XYZ axis direction. The image processing device 30a can control the drive mechanism 52. The drive mechanism 52 moves the detection area on the sample 60 relatively. By moving Stage 50 in the XYZ axis direction, the illumination position of the illumination light L11 on the sample 60 can be changed.

[0033] Therefore, any position on the sample 60 can be imaged, and almost the entire surface of the sample 60 can be inspected. Alternatively, the imaging optical system 10 may be driven instead of the stage 50. That is, the relative position of the imaging optical system 10 with respect to the stage 50 may be made movable. Alternatively, the illumination light L11 may be scanned using an optical scanner or the like.

[0034] The image processing device 30a processes the image captured by the imaging optical system 10. Details will be described later in relation to Figures 4 and 5.

[0035] The display device 40 displays a defect if it determines that a defect exists in the defect candidate area. The display device 40 is, for example, the main PC that serves as the user interface with the user.

[0036] Figure 4 is a diagram illustrating the image color conversion process according to Embodiment 2. Images 401 and 402 in Figure 4 are images of the same object having a flower, stem, and leaves. In this example, the flower region is the object to be detected (i.e., corresponding to the DOI). Images 401 and 402 are subtly different in hue and brightness. However, these differences in hue and brightness are assumed to be within the range of variation of the object to be detected. Therefore, we focus on the hue, saturation, and value of the object to be detected (corresponding to the DOI) and set arbitrary HSV ranges (upper and lower limits) for each value to cover the range of variation of these values. In both images 401 and 402, the flower region, which is the object to be detected (i.e., corresponding to the DOI), is converted to a specific single color (e.g., red), and other regions (e.g., background regions other than the defect region) are converted to, for example, achromatic (grayscale) or another color (e.g., white, black). An example of the converted image is shown in Image 410.

[0037] Figure 5 is a flowchart illustrating the image processing method according to Embodiment 2. Step S11: An RGB color image is acquired of the region containing the object to be detected (DOI). Step S12: The acquired image is converted from the RGB color space to the HSV color space. Step S13: The range of each HSV value is set for the target region to be detected in the converted image. Step S14: Pixels in the image that are within the set range of each HSV value are converted to first color information (e.g., red) whose color contrast value with the background color (e.g., achromatic, white, black, etc. after conversion) is equal to or greater than a threshold. Step S15: Pixels in the image that are outside the set range of each HSV value are converted to second color information (e.g., achromatic, white, black, etc.) whose color contrast value with the first color information is equal to or greater than a threshold.

[0038] Note that the color conversion process in step S14 and the color conversion process in step S15 may be performed in any order. Also, although the flowchart in Figure 5 shows the specific order of execution, the order of execution may differ from the form depicted. Furthermore, two or more steps shown consecutively in Figure 5 may be executed simultaneously or partially simultaneously. In addition, in some embodiments, one or more steps shown in Figure 5 may be skipped or omitted.

[0039] Next, an image processing device 30a that processes images captured by the imaging optical system 10 will be described. Figure 6 is a block diagram illustrating the image processing device 30a according to Embodiment 2. As shown in Figure 6, the image processing device 30a includes an image acquisition unit 31a, a color space conversion unit 32a, a range setting unit 33a, a first color conversion unit 34a, a second color conversion unit 35a, an image collection unit 36a, and a learning processing unit 38a. That is, the image processing device 30a is one embodiment of the image processing device 30 according to Embodiment 1, and collects images that have undergone image processing and performs machine learning such as deep learning. The image processing device 30a is, for example, a computer equipped with a GPU (Graphics Processing Unit) and memory. Each component of the image processing device 30a can be realized, for example, by executing a program. Note that the image processing device 30a is not limited to a GPU, but may also be a computer equipped with a CPU (Central Processing Unit), FPGA (Field-Programmable Gate Array), or microcontroller.

[0040] The image acquisition unit 31a acquires an RGB color image of the region containing the defect area of ​​the sample. For example, in a semiconductor wafer inspection process, it acquires one or more color images of semiconductor wafers that have been recognized as having defects.

[0041] The color space conversion unit 32a converts the acquired image from the RGB color space to the HSV color space. Although it is possible to determine the color tone and brightness in the image with an RGB image, for example, when determining an image (i.e., light and shade, brightness and darkness) from a light color tone to a dark color tone or from a dark color to a bright color, in an RGB image, it becomes discrete codes and it is difficult to determine. Therefore, using the color space conversion unit 32, the acquired RGB color image is converted into the HSV image format. The HSV imaging data can be data associated with the RGB imaging data using pixel position information (coordinates) as a key.

[0042] The range setting unit 33a sets the range of each value of HSV for the defect region in the converted image. For example, in some embodiments, by an operator operating the operation unit of the image processing apparatus 30a, the range of each value of arbitrary HSV (the upper and lower limits of each value, that is, H max , H min , S max , S min , V max , V min ) can be set. Alternatively, in other embodiments, based on each value of HSV (for example, the maximum value, the minimum value, the average value, etc.) of the acquired image's defect region, a predetermined range (plus or minus threshold) may be automatically set.

[0043] The first color conversion unit 34a converts pixels within the set range of each value of HSV into first color information with a color contrast value of a threshold or more with respect to the background color. That is, when all the values of HSV of a certain pixel in the image are within the set upper and lower limits of HSV, it is converted into first color information. Conversely, when any one of the values of HSV of a certain pixel in the image is outside the set upper and lower limits of HSV, it is not converted into first color information. A specific defect region in the image is converted to a specific color (for example, a primary color such as red, green). The defect region can be called DOI (Defect of Interest).

[0044] The second color conversion unit 35a converts pixels in the image that are outside the range of the set HSV values ​​into second color information whose color contrast value with the first color information is equal to or greater than a threshold. In other words, the background area other than the defect area is converted to achromatic (grayscale). In other embodiments, the background area other than the defect area may be converted to a different color (e.g., white, black) than the specific color of the first color conversion unit 34. As described above, by using images that have undergone such image processing as training data for machine learning (deep learning), an even more accurate defect detection model can be generated. In addition, by fixing the defect area and the other background area to a specific color, variations in brightness between images after color conversion can be suppressed.

[0045] The image acquisition unit 36a collects and stores a large number of images that have undergone the image processing described above. The image acquisition unit 36a may be a storage unit within the image processing device 30a, or it may be an external storage unit connected to the image processing device 30a.

[0046] The learning processing unit 38a performs machine learning (deep learning) using the images that have undergone such image processing and accumulated as training data (supervised data). The learning processing unit 38a can generate a defect detection model. In some embodiments, the learning processing unit 38a can obtain a learning model by machine learning based on given features. In other embodiments, the learning processing unit 38a can obtain a deep learning model by deep learning that determines the features.

[0047] Figure 7 is a diagram illustrating the overall operation of the defect detection system according to Embodiment 2. In defect detection system 1, images 401 and 402 are identified as DOIs (Defect of Interest) (Step S1). That is, in the inspection process, a color image of the object in which a defect was detected is acquired. Next, as described above using Figure 4, conversion processing from RGB space to HSV space, color conversion processing of the DOI area, and color conversion processing of the background area are performed (Step S3). Images that have undergone such image processing are collected in the storage unit as training data (Step S5). Machine learning such as deep learning is performed on the collected training data to generate a defect detection model (Step S7). The processing up to this point is as described above.

[0048] Next, an inspection process is performed using the trained defect detection model (step S9). Here, an example of the configuration of the defect detection device 300 will be explained with reference to Figure 8. The defect detection device 300 comprises an image acquisition unit 310, a color space conversion unit 320, a first color conversion unit 340, a second color conversion unit 350, and a defect detection unit 370. The defect detection unit 370 can use a learned defect detection model 371 (also called a learning model) stored in a memory unit. Note that the configuration of the defect detection device 300 shown in Figure 8 is merely an example and can be modified and altered.

[0049] The defect detection device 300 is, for example, a computer equipped with a GPU (Graphics Processing Unit) and memory. Each component of the defect detection device 300 can be realized, for example, by executing a program. The image processing device 30 is not limited to a GPU; it may also be a computer equipped with a CPU (Central Processing Unit), FPGA (Field-Programmable Gate Array), or microcontroller.

[0050] The image acquisition unit 310 acquires an RGB color image of the sample captured by the imaging optical system 10. For example, in the inspection process of a semiconductor wafer, one or more color images of the semiconductor wafer (for example, image 501 in Figure 7) are acquired.

[0051] The color space conversion unit 320 converts the acquired image from the RGB color space to the HSV color space.

[0052] The first color conversion unit 340 converts pixels in the image that fall within a preset range of HSV values ​​into first color information such that the color contrast value relative to the background color is equal to or greater than a threshold. As shown in Figure 2, it converts specific defect areas in the image to specific colors (e.g., primary colors such as red or green). Defect areas may be called DOIs (Defect of Interest).

[0053] The second color conversion unit 350 converts pixels in the image that are outside the range of the set HSV values ​​into second color information whose color contrast value with the first color information is equal to or greater than a threshold. In other words, the background area other than the defect area is converted to achromatic (grayscale). In other embodiments, the background area other than the defect area may be converted to a different color (e.g., white, black) than the specific color of the first color conversion unit 34. The converted image is transmitted to the defect detection unit 370.

[0054] The defect detection unit 370 uses the defect detection model 371 to perform defect detection on the color-converted image (for example, image 510 in Figure 7) as described above (step S11 in Figure 7). As described above, by using the defect detection model and the color-converted image, defects can be detected with high accuracy. After that, the original RGB color image corresponding to the color-converted image in which a defect was detected may be reviewed (step S13).

[0055] As explained above, using the defect detection model, defects can be detected with high accuracy in newly acquired inspection images.

[0056] In the above example, the program can be stored and supplied to the computer using various types of non-transitory computer-readable medium. Non-transitory computer-readable medium includes various types of tangible storage medium. Examples of non-transitory computer-readable medium include magnetic recording media (e.g., flexible disks, magnetic tapes, hard disk drives), magneto-optical recording media (e.g., magneto-optical disks), CD-ROMs (Read Only Memory), CD-Rs, CD-R / Ws, DVDs (Digital Versatile Discs), BDs (Blu-ray® Discs), and semiconductor memory (e.g., mask ROMs, PROMs (Programmable ROMs), EPROMs (Erasable PROMs), flash ROMs, RAMs (Random Access Memory)). Alternatively, the program may be supplied to the computer using various types of transient computer-readable medium. Examples of transient computer-readable medium include electrical signals, optical signals, and electromagnetic waves. Temporary computer-readable media can supply programs to a computer via wired communication channels such as electric wires and optical fibers, or via wireless communication channels.

[0057] Although embodiments of the present invention have been described above, the present invention includes appropriate modifications that do not impair its purpose and advantages, and is not limited by the above embodiments. [Explanation of Symbols]

[0058] 1. Defect detection system 10 Imaging optical system 11 Light source 12 Beam Splitter 13 Objective lens 14 detectors 30, 30a Image processing apparatus 31, 31a Image acquisition section 32, 32a Color space conversion section 33, 33a Range setting section 34, 34a First color conversion unit 35, 35a Second color conversion section 36a Image Collection Unit 38a Learning Processing Unit 40 Display device 50 stages 51 Stages 52 Drive mechanism 60 samples 61 patterns 300 Defect detection device 310 Image acquisition unit 320 Color Space Conversion Unit 330 First Color Conversion Unit 350 Second color conversion unit 370 Defect detection unit 371 Defect Detection Models

Claims

1. An image acquisition unit that acquires an RGB color image of the sample, A color space conversion unit converts the acquired image from the RGB color space to the HSV color space, A range setting unit sets the range of each HSV value for the region within the converted image, A first color conversion unit converts pixels within the set range of HSV values ​​in the defective region of the image into first color information such that the color contrast value relative to the background color is greater than or equal to an arbitrarily given non-zero threshold. Equipped with, A learning process is performed to generate a defect detection model by training a deep learning model using the converted image, which is created by processing the image including the defect region with the color space conversion unit and the first color conversion unit to convert the defect region having different brightnesses into a specific fixed color, as training data. A detection process is performed by inputting the converted image, which is processed by the color space conversion unit and the first color conversion unit, onto the RGB color image of the sample, into the trained defect detection model to detect defects in the image. An image processing apparatus characterized by performing the following.

2. The image processing apparatus according to claim 1, further comprising: a second color conversion unit that converts pixels outside the range of each of the HSV values ​​set for the defective region in the image into second color information such that the color contrast value with the first color information is greater than or equal to an arbitrarily given non-zero threshold.

3. The image processing apparatus according to claim 1, characterized in that it outputs an RGB color image of the image in which the defect was detected in the detection process.

4. The aforementioned sample is a wafer or a mask. The image processing apparatus according to claim 1.

5. An imaging optical system for imaging the sample, An image processing device that processes the image captured by the aforementioned imaging optical system, Equipped with, The aforementioned image processing device is An image acquisition unit that acquires an RGB color image of the sample, A color space conversion unit converts the acquired image from the RGB color space to the HSV color space, A range setting unit sets the range of each HSV value for the region within the converted image, A first color conversion unit converts pixels within the set range of HSV values ​​in the defective region of the image into first color information such that the color contrast value relative to the background color is greater than or equal to an arbitrarily given non-zero threshold. Equipped with, A learning process is performed to generate a defect detection model by training a deep learning model using the converted image, which is created by processing the image including the defect region with the color space conversion unit and the first color conversion unit to convert the defect region having different brightnesses into a specific fixed color, as training data. A detection process is performed by inputting the processed image obtained from the RGB color image of the sample, processed by the color space conversion unit and the first color conversion unit, into the trained defect detection model to detect defects in the image. A defect detection system characterized by performing the following.

6. The defect detection system according to claim 5, further comprising a second color conversion unit that converts pixels in the defective region of the image that are outside the range of each of the set HSV values ​​into second color information whose color contrast value with the first color information is greater than or equal to an arbitrarily given non-zero threshold.

7. We acquire an RGB color image of the sample, The acquired image is converted from RGB color space to HSV color space. For the regions within the converted image, set the range of each HSV value, The process includes converting pixels within the set range of HSV values ​​in the defective region of the image into first color information such that the color contrast value relative to the background color is greater than or equal to an arbitrarily given non-zero threshold, A learning process to generate a defect detection model by training a deep learning model using a converted image obtained by converting the image including the defect region to the HSV color space and the first color information, wherein the converted image has the defect region having different brightness levels assigned to a specific fixed color, and the trained model is obtained by training the converted image as training data. A detection process is performed by inputting the converted image, which is obtained by converting the RGB color image of the sample to the color space and the first color information, into the trained defect detection model to detect defects in the image, An image processing method that performs this operation.

8. We acquire an RGB color image of the sample, The acquired image is converted from RGB color space to HSV color space. For the regions within the converted image, set the range of each HSV value, The process includes converting pixels within the set range of HSV values ​​in the defective region of the image into first color information such that the color contrast value relative to the background color is greater than or equal to an arbitrarily given non-zero threshold, A learning process to generate a defect detection model by training a deep learning model using a converted image obtained by converting the image including the defect region to the HSV color space and the first color information, wherein the converted image has the defect region having different brightness levels assigned to a specific fixed color, and the trained model is obtained by training the converted image as training data. A detection process is performed by inputting the converted image, which is obtained by converting the RGB color image of the sample to the color space and the first color information, into the trained defect detection model to detect defects in the image, An image processing program that instructs a computer to execute commands to perform an action.

Images