Apparatus and method for improving specimen images

Abstract

An apparatus and method for improving specimen images are disclosed. The specimen image improvement apparatus includes a specimen image storage unit that stores a specimen image of the specimen surface, and a processor that inputs the specimen image into a pre-trained image improvement algorithm to generate a second specimen image with improved resolution, and inputs the second specimen image into a pre-trained defect analysis algorithm to analyze it at the pixel level to detect defects. Accordingly, by improving the specimen image based on the pre-trained image improvement algorithm, a low-resolution specimen image obtained by rapid scanning (imaging) with a scanning electron microscope (SEM), etc., is restored to a high-resolution level, thereby reducing the imaging time while improving the quality of the specimen image and enhancing the accuracy of defect detection.

Description

Technology Field

[0001] The present invention relates to a specimen image enhancement device and method, and more specifically, to a specimen image enhancement device and method capable of enhancing the resolution of a low-resolution specimen image to a high resolution using a pre-trained image enhancement algorithm and precisely detecting defects using the enhanced specimen image. Background Technology

[0003] A specimen refers to a test sample taken to evaluate the surface characteristics or defects of various materials, such as semiconductors, metals, and glass.

[0004] Surface inspection of these specimens is a critical process for evaluating product quality and reliability, and the early detection of micro-defects is extremely important.

[0005] Conventionally, after capturing images of the specimen using a Scanning Electron Microscope (SEM), a quality inspector identified defects visually.

[0006] However, obtaining high-resolution images with a scanning electron microscope requires long scanning times, and balancing speed and resolution is difficult when photographing large-area samples. Furthermore, using low-resolution specimen images reduces defect detection accuracy and may lead to errors during visual inspection.

[0007] Accordingly, there has been an increased need for technology capable of improving the accuracy of defect analysis by enhancing low-resolution specimen images using artificial intelligence to obtain specimen images comparable to high resolution. Prior art literature

[0009] Korean Registered Patent No. 10-1282984 (Published July 5, 2013) The problem to be solved

[0010] The objective of the present invention is to provide a specimen image enhancement device and method capable of enhancing the resolution of a low-resolution specimen image to a high resolution using a pre-trained image enhancement algorithm and precisely detecting defects using the enhanced specimen image. means of solving the problem

[0012] A specimen image enhancement device according to one embodiment of the present invention for achieving such objectives includes: a specimen image storage unit that stores a specimen image of the surface of a specimen; and a processor that inputs the specimen image into a pre-learned image enhancement algorithm to generate a second specimen image with improved resolution, and inputs the second specimen image into a pre-learned defect analysis algorithm to analyze it pixel by pixel to detect defects.

[0013] The processor is characterized by performing simulation-based preprocessing using pre-built simulation data to correct the noise pattern and scanning pattern of the specimen image for the specimen image before generating the second specimen image.

[0014] The processor is characterized by detecting defects by inputting the second specimen image into a pre-trained defect analysis algorithm and analyzing it pixel by pixel, clustering the detected defects based on defect characteristics, and displaying the type and number of defects for each cluster.

[0015] The above processor is characterized by performing clustering based on at least one of the location, size, orientation, contrast, and shape of the defect.

[0016] The above processor is characterized by visually displaying the clustered defects by different colors or shapes according to type.

[0017] The above processor is characterized by visually displaying the clustered defects by type using different colors or shapes, while also displaying the number of defects.

[0018] The processor is characterized by distinguishing regions in pixel units in the second sample image, but distinguishing only the defective regions of interest.

[0019] The defect includes at least one of a scratch and a dot, and the processor analyzes a second specimen image in pixel units based on a pre-learned defect analysis algorithm, and detects the defect by recognizing the defect present in the second specimen image, classifying the type of the defect, and determining the similarity of the defect.

[0020] The processor is characterized by marking a pixel-unit area containing the detected defect as a bounding box, and marking the type and similarity of the defect in an area adjacent to the bounding box.

[0021] The above processor is characterized by displaying different colors in one area where the type and similarity of the defect are indicated, depending on the type of defect.

[0022] The processor is characterized by performing multi-stage clustering, which involves performing primary clustering of the detected defects based on defect characteristics and secondary clustering of the primary clustered defects based on detailed defect characteristics.

[0023] Meanwhile, a specimen image enhancement method in a specimen image enhancement device according to one embodiment of the present invention comprises: a step of generating a second specimen image with improved resolution by inputting a specimen image stored in a specimen image storage unit into a pre-learned image enhancement algorithm; and a step of detecting defects based on the second specimen image. Effects of the invention

[0025] According to various embodiments of the present invention as described above, by improving specimen images based on a pre-learned image enhancement algorithm, a low-resolution specimen image obtained by rapid scanning (imaging) in a scanning electron microscope (SEM) or the like is restored to a high-resolution level, thereby reducing the imaging time while improving the quality of the specimen image and enhancing the accuracy of defect detection.

[0026] In addition, by clustering detected defects based on defect characteristics to calculate and visually display the types and number of defects, it is possible to summarize and analyze defects at the cluster level, even when the number of defects is vast, thereby reducing the time required for defect detection and improving accuracy.

[0027] In addition, by automatically detecting and displaying specimen images on the specimen surface through artificial intelligence analysis, it is possible to enhance quality control and defect analysis in the product manufacturing process.

[0028] In addition, since complex and minute defects can be accurately recognized at the pixel level based on artificial intelligence analysis, it has the effect of improving the accuracy of the analysis by preventing inspection errors that may occur when humans perform manual visual quality inspections of products.

[0029] In addition, by visualizing and displaying defect information such as the location, type, and similarity of defects within the specimen image, it has the effect of enabling the user (product quality inspector) to quickly and accurately grasp the situation and make judgments (easily identifying process abnormalities if they occur in specific locations or repetitive patterns).

[0030] In addition, when defect information is displayed visually, it is easier for product manufacturing process personnel, engineers, and quality control teams to identify the defect compared to text-based communication, thereby reducing the time required for defect identification and handling. In other words, it increases the efficiency of collaboration among stakeholders in defect handling, which can shorten the time required for defect resolution. Brief explanation of the drawing

[0032] FIG. 1 is a diagram illustrating the configuration of a specimen image enhancement device according to one embodiment of the present invention. FIGS. 2 and 3 are drawings for explaining a specimen image enhancement device according to an embodiment of the present invention. FIG. 4 is a flowchart illustrating a method for improving a specimen image according to an embodiment of the present invention. FIG. 5 is a block diagram showing the specific configuration of a specimen image enhancement device illustrated in FIG. 1 according to an embodiment of the present invention. FIG. 6 is a drawing relating to a software module stored in a storage unit according to an embodiment of the present invention. FIG. 7 is a drawing for explaining a specimen image enhancement device according to another embodiment of the present invention. Specific details for implementing the invention

[0033] The present invention will be described in more detail below with reference to the drawings. Furthermore, in describing the present invention, specific descriptions of related known functions or configurations are omitted if it is determined that such detailed descriptions would unnecessarily obscure the essence of the invention. Additionally, the terms described below are defined considering their functions in the present invention, and these may vary depending on the intentions or relationships of the user or operator. Therefore, their definitions should be based on the content throughout this specification.

[0034] FIG. 1 is a diagram illustrating the configuration of a specimen image enhancement device according to an embodiment of the present invention, and FIG. 2 to 3 are diagrams for explaining a specimen image enhancement device according to an embodiment of the present invention.

[0035] Referring to FIG. 1, a specimen image enhancement device (100) according to one embodiment of the present invention includes a specimen image storage unit (110) and a processor (120).

[0036] The specimen image storage unit (110) can store a specimen image of the specimen surface. At this time, the specimen may refer to a sample extracted or manufactured to evaluate the physical and / or chemical properties, surface condition, etc., of a product such as a metal or semiconductor, and may be composed of various materials such as metal, semiconductor, or glass. Additionally, the specimen surface may be observed through a scanning electron microscope (SEM), and a specimen image representing the shape of the surface may be generated based on the observation and stored in the specimen image storage unit (110).

[0037] Meanwhile, the specimen image storage unit (110) may be a separate storage space that stores only the specimen image of the specimen surface. However, it is not limited to this, and the specimen image storage unit (110) may be the storage unit (130) described later. Accordingly, the specimen image of the specimen surface may be stored in the storage unit (130) described later. Additionally, the specimen image storage unit (110) may be included in the storage unit (130) described later.

[0038] The processor (120) can input the specimen image into a pre-trained image enhancement algorithm to generate a second specimen image with improved resolution. At this time, the pre-trained image enhancement algorithm may be an artificial intelligence image enhancement algorithm. Meanwhile, in describing the present embodiment, the specimen image captured by the electron scanning microscope may be the first specimen image, and the specimen image with improved resolution by the image enhancement algorithm may be the second specimen image.

[0039] At this time, the artificial intelligence image enhancement algorithm may be a known image enhancement algorithm such as deep learning.

[0040] Additionally, according to the embodiment, the artificial intelligence image enhancement algorithm may have a U-Net structure. In this case, the artificial intelligence image enhancement algorithm having a U-Net structure may be a symmetric encoder-decoder network, wherein the encoder part extracts low-dimensional features (e.g., edges, patterns, etc.) of the low-resolution specimen image, and the decoder part can restore the detailed structure of the original resolution through upsampling. Through this process, the processor (120) preserves detailed spatial information of the original specimen image through skip connections, and consequently, can generate a second specimen image that more clearly expresses the microstructure, interface, micro-pattern, etc. of the specimen image captured by a scanning electron microscope (SEM). Meanwhile, the image enhancement algorithm according to the present embodiment may be pre-trained through supervised learning using pairs of 2 frames of low-resolution input specimen images and 32 frames of high-resolution ground truth specimen images. Here, a skip connection refers to a structure in which the output of one layer of an artificial intelligence neural network is skipped and directly connected to the next layer. For example, information loss can be minimized by directly connecting an intermediate feature map on the encoder side to a corresponding position (layer) on the decoder side. In other words, the aforementioned skip connection can contribute to significantly improving the quality of the second sample image, such as resolution and sharpness, by maintaining detailed structural information during the resolution enhancement process of the sample image. At this time, the performance of the image enhancement algorithm can be verified using PSNR, SSIM, and / or LPIPS indices. PSNR (Peak Signal-to-Noise Ratio), SSIM (Structural Similarity Index), and / or LPIPS (Learned Perceptual Image Patch Similarity) indices may be used to quantitatively evaluate the quality of the image.More specifically, the process (120) can objectively measure the quality of the second specimen image using PSNR for noise reduction and / or resolution restoration performance, evaluate the degree of shape retention of the specimen defect structure (scratches, dots, etc.) using SSIM, and quantify the degree of image clarity and / or naturalness as seen by the human eye using LPIPS. In this way, the processor (120) can quantitatively evaluate the quality of the result generated by the image enhancement algorithm by evaluating the quality of the improved second specimen image using PSNR, SSIM, and LPIPS indices.

[0041] According to an embodiment, the processor (120) may receive an original specimen image obtained using an LEO 1560 fE-SEM device. At this time, the original specimen image may be a first specimen image. Here, the first specimen image may be a low-quality specimen image (low-resolution specimen image) with a resolution of 1024x768, and may be specimen data captured under fast scanning conditions. At this time, the processor (120) may divide the first specimen image into random crop units of size 128x128 to 512x512, and normalize each specimen image patch to convert it to fit the input format of an image enhancement algorithm. Accordingly, the converted data may be input to an image enhancement algorithm (artificial intelligence algorithm), that is, an image enhancement network based on the U-Net backbone.

[0042] As described above, as shown in FIG. 2, the original low-resolution specimen image (Raw Image) (Fig. 2 (a)) can be converted into an Enhanced Image (Fig. 2 (b)) with improved resolution, contrast, and noise suppression through a pre-trained image enhancement algorithm.

[0043] That is, as described above, the processor (120) can correct noise patterns, scan streaks, non-uniform brightness, etc. present in the input specimen image using parameters of a pre-learned image enhancement algorithm, thereby generating a second specimen image with improved resolution, clarity, and / or contrast.

[0044] Additionally, the processor (120) may perform preprocessing on the sample image to generate an enhanced image.

[0045] More specifically, the processor (120) may perform simulation-based preprocessing using pre-built simulation data to correct the noise pattern and scanning pattern of the specimen image before generating the second specimen image.

[0046] That is, the processor (120) may perform a preprocessing process using a pre-built simulation data set to correct noise patterns (e.g., line noise, flicker noise, beam drift, etc.) and scanning pattern non-uniformity that can be observed in a scanning electron microscope (SEM) capturing an image of the actual specimen surface, as well as a process of simple noise removal and / or brightness normalization.

[0047] At this time, the preprocessed image can be used as training data to increase the accuracy of defect detection. Accordingly, the processor (120) preprocesses the specimen image as described above before inputting the specimen image into a pre-trained artificial intelligence analysis algorithm (defect analysis algorithm), inputs the preprocessed specimen image into a pre-trained image enhancement algorithm to generate a second specimen image with improved resolution, and inputs the second specimen image into the pre-trained defect analysis algorithm, thereby increasing the accuracy of defect detection in the second specimen image.

[0048] As described above, the processor (120) can generate a second specimen image with improved quality by performing preprocessing on the specimen image to remove noise and / or scanning patterns and inputting the specimen image into an artificial intelligence image enhancement algorithm. Accordingly, effective distinction between actual defects and imaging noise contained in the second specimen image is possible, thereby enabling more accurate detection of defects.

[0049] The processor (120) can detect and display defects in the second sample image by inputting the second sample image into a pre-trained defect analysis algorithm and analyzing it pixel by pixel. At this time, the pre-trained defect analysis algorithm may be an artificial intelligence analysis algorithm. Here, the defect may be a dot, a scratch, etc., and may occur during the product manufacturing process, such as photolithography.

[0050] At this time, the artificial intelligence analysis algorithm may be a known analysis algorithm such as deep learning. Meanwhile, the artificial intelligence analysis algorithm according to the embodiment can perform pre-learning centered on a labeled region formed by labeling the part where the defect exists. As such, since the artificial intelligence analysis algorithm can perform pre-learning centered on the labeled region, the learning effect (accuracy and performance of learning) can be improved.

[0051] Meanwhile, labeling can be the process of assigning meaningful information to each piece of data in an image (e.g., type of defect, location of defect, etc.), and a user can perform labeling on images that contain defects in advance.

[0052] As described above, the artificial intelligence analysis algorithm can perform pre-learning by referring together to the input image and the correct answer label containing defect information.

[0053] Additionally, the processor (120) can distinguish regions in pixel units in the second sample image, but only distinguish regions of interest that have defects. Here, the region of interest may refer to an area containing defects, such as dots or scratches. In this way, the processor (120) undergoes a segmentation process that distinguishes only the defective regions in pixel units, thereby classifying which region each pixel in the second sample image belongs to, and thus can distinguish exactly which pixel range the defect exists in.

[0054] Additionally, the processor (120) can detect defects by analyzing the second specimen image in pixel units based on a pre-learned defect analysis algorithm, recognizing defects present in the second specimen image, classifying the types of defects, and determining the similarity of defects. The processor (120) can analyze whether there are defects by using a sliding window method and / or a CNN-based known algorithm to preferentially identify areas where abnormal features are detected, i.e., pixel units, through feature extraction from the input second specimen image.

[0055] In this case, similarity may be a value that quantitatively measures how similar two images are, and may be an indicator that evaluates how similar a second specimen image containing a specific defect is to a second specimen image containing an existing learned defect. In this case, the closer the similarity is to 1, the higher the similarity may be.

[0056] More specifically, the processor (120) can recognize and classify defects based on defect characteristics in the second specimen image. In this case, defect characteristics may refer to the location where the defect occurred, the shape, size and / or form (type and / or shape), the direction and contrast of the defect, etc. Here, the location where the defect occurred may be the coordinate value of the pixel containing the defect as the second specimen image is analyzed by dividing it into pixel units, and contrast may refer to the difference in brightness between the defect and the background. Meanwhile, if a defect exists, a part of the corresponding pixel area, that is, the part of the area where the defect exists, may have different pixel characteristics as described above when compared to an area where no defect exists.

[0057] According to the embodiment, the processor (120) determines that a defect exists in a specific pixel unit area of ​​the second specimen image if the contrast is high, and, for example, if a defect characteristic of a linear structure with a length (size) of about 10 µm exists, it can classify the type of defect as a 'scratch'.

[0058] Additionally, the processor (120) determines that a defect exists in a specific pixel unit area of ​​the second sample image if the contrast is high, and, for example, if there is a defect characteristic of a small circular structure with a size (length) of about 2 µm, it can classify the type of defect as a 'dot'.

[0059] Meanwhile, recognizing and classifying defects is not limited to what has been described above, and defects can be recognized and classified in various ways based on defect characteristics.

[0060] At this time, the pre-trained defect analysis algorithm can determine similarity by comparing the defects in the second specimen image with the correct label. More specifically, the pre-trained defect analysis algorithm can determine similarity by comparing the defect characteristic information included in the defects of the second specimen image with the defect information included in the correct label.

[0061] For example, if a defect in the second specimen image has a horizontal scratch shape with a size of 10.5 µm, and defect information with a similar value is included in the ground truth label, the defect analysis algorithm can assign a high similarity value close to 1. On the other hand, if the defect in the second specimen image does not match the defect information included in the ground truth label, or if it is a new type of defect and / or has a size outside a preset range, the similarity can be assigned a low value close to 0, and it can be included in the ground truth label as a new defect. Through this, the defect analysis algorithm can perform pre-training.

[0062] Additionally, when a defect is detected as described above, the processor (120) may mark a pixel-unit area containing the detected defect as a bounding box and display the type and similarity of the defect in an area adjacent to the bounding box. That is, the processor (120) can visually display information about the detected defect.

[0063] Referring to Figures 3 (a) and (b), it can be seen that a bounding box is displayed in the area containing the defects detected in the line and hole of the second specimen image, respectively, and that information about the defects, such as the type and similarity of the defects, is displayed.

[0064] Here, a line refers to a long, narrow linear structure formed through a photolithography process, and a hole may refer to a structure in which small holes and / or openings are formed in an insulating film through a photo and / or etching process.

[0065] More specifically, referring to FIG. 3(a), if a scratch defect is detected based on the above description, the processor (120) can mark a pixel-unit area containing the scratch defect as a bounding box (310). Then, the processor (120) can display information about the defect, such as the type and similarity of the defect, in an upper area of ​​the bounding box (320).

[0066] Additionally, the processor (120) may display different colors in the area where information about the defect, such as the type and similarity of the defect, is displayed, depending on the type of defect.

[0067] For example, referring to FIG. 3(b), when a defect is detected based on the above description, the processor (120) may mark a pixel-unit area containing the detected defect as a bounding box and mark the type and similarity of the defect in a part of the upper part of the bounding box, and if the defect is a 'scratch', mark the part of the area marking the type and similarity of the defect in sky blue (330), and if the defect is a 'dot', mark the part of the area marking the type and similarity of the defect in blue (340).

[0068] That is, if there are multiple defects, the processor (120) may cluster the multiple defects based on the defect characteristics. That is, if there are multiple defects within a certain area on the surface of a specimen, the processor (120) may cluster the aforementioned multiple defects based on the defect characteristics. That is, as shown in FIG. 7 (a), if there are multiple 'point' defects, the processor (120) may cluster multiple 'point' defects based on the defect characteristics of the 'point' defects. At this time, the processor (120) may display the number of defects along with the type of clustered defects. As shown in FIG. 7 (b), if there are multiple 'scratch' defects, the processor (120) may cluster multiple 'scratch' defects based on the defect characteristics of the 'scratch' defects. At this time, the processor (120) may display the number of defects along with the type of clustered defects. Additionally, as illustrated in FIG. 7(c), if 'dot' and 'scratch' defects exist in combination, the processor (120) can cluster multiple 'dot' and 'scratch' defects by defect based on the defect characteristics of the 'dot' and 'scratch' defects. At this time, the processor (120) may display the number of defects along with the types of clustered defects. Meanwhile, when clustering by defect, the processor (120) may visualize and display the clustered defects by different colors. That is, the processor (120) may cluster 'dot' defects by displaying them in yellow and cluster 'scratch' defects by displaying them in blue, thereby making it easy to identify that they are different defects. At this time, defect characteristics may refer to the location where the defect occurred, the shape, size and / or shape (type) of the defect, and contrast. That is, the processor (120) can group similar defects together.

[0069] For example, the processor (120) may have defects detected in the second specimen image as shown in Table 1 below.

[0071] Defect ID Types of defects Defect size Defect direction Similarity cluster group A scratch 10.5 µm horizontality 0.99 CG1 B scratch 10.8 um horizontality 0.93 CG1 c scratch 11.3 um horizontality 0.92 CG1 d dot 4 um doesn't exist 0.3 CG2 e scratch 19 um perpendicular 0.5 CG3

[0072] According to this, the processor (120) can cluster defects A to C into the CG1 group based on defect characteristics, as defects A through C have high defect similarity and similar defect characteristics. Additionally, based on defect characteristics, the processor (120) can cluster defect D into the CG2 group and defect E into the CG3 group. Afterward, the processor (120) can visualize and display the formed groups. For example, the CG1 group can be displayed as a blue box, the CG2 group as a red circle, and the CG3 group as a yellow box. In this way, the processor (120) can display defects in different forms depending on the type of defect (e.g., displaying a box if it is a 'scratch' defect and a circle if it is a 'dot' defect), and can display them in different colors depending on the cluster group. In this way, by clustering multiple defects into a single group based on their characteristics and visualizing them differently by defect type and / or cluster group, it becomes easier to trace the causes when specific defect groups occur intensively during the product manufacturing process, and it also facilitates the detection of new defects. Furthermore, when the number of defects is extensive, analysis and summarization are possible at the group level, which has the effect of significantly improving analysis efficiency.

[0073] In other words, the processor (120) can detect defects by inputting the second specimen image into a pre-learned defect analysis algorithm and analyzing it pixel by pixel, and can cluster the detected defects based on defect characteristics and display the type and / or number of defects for each cluster.

[0074] Additionally, the processor (120) can visually display clustered defects by type using different colors and / or shapes, and can also display the number of defects. That is, the processor (120) can visually display clustered defects by type using different colors and / or shapes, and can also display the number of defects.

[0075] For example, when multiple point defects are detected in the second specimen image, the processor (120) can analyze defect characteristic information, such as the location coordinates, size, and contrast of each of these 'point' defects, to form a cluster group of defects distributed in similar areas. Subsequently, the processor (120) can identify that the defects belonging to the cluster group are 'point' defects, calculate that the number is 12, and then display one cluster group as a rectangle on the screen, displaying the color of the rectangle in yellow and simultaneously outputting information such as "points (15)". Additionally, when multiple scratch defects having a linear structure are detected in the second specimen image, the processor (120) can evaluate similarity based on the length and direction of the defects and group similar scratches into another cluster group. At this time, the processor (120) can calculate that the defect type of the group is 'scratch' and the number is 10, and then display information such as "scratch (10)" along with a blue rectangle on the screen. This allows users to intuitively check not only whether defects exist on the specimen surface, but also the number of defects by type and cluster.

[0076] Meanwhile, when the processor (120) clusters defects distributed in similar areas to form a cluster group, it may visually display a shape according to the type of defect belonging to the cluster group. That is, as described above, the processor (120) displays a shape as a rectangle regardless of whether the type of defect is a 'dot' or a 'scratch', but is not limited thereto, and may visually display the clustered defects by using different shapes according to type. That is, the processor (120) may visually display the clustered defects by using different colors and / or shapes according to type.

[0077] For example, the processor (120) may display a shape as a 'circle' when the clustered defect is a 'dot', and a shape as a 'square' when it is a 'scratch'. In this way, by displaying different shapes for each type of clustered defect, the user can more intuitively identify the type of defect. At this time, the processor (120) may visually display the clustered defects by using different colors for each type, as described above.

[0078] Additionally, the processor (120) can perform clustering based on at least one of the location, size, orientation contrast, and shape of the defects.

[0079] For example, the processor (120) can detect multiple scratch defects in the second specimen image and then analyze the location coordinates, length, and directionality of each defect to group scratches with similar location ranges and directions into the same cluster group. Additionally, if a dot-shaped defect is detected, the size and contrast of each dot defect can be evaluated to form another cluster group of dot defects that are similar in size and have a background contrast level above a certain level. In this way, the processor (120) can classify defects into respective cluster groups based on whether the shape of the detected defect is circular or linear, and by displaying the type and number of defects for each group, the user can intuitively check the distribution of defects on the specimen surface.

[0080] Meanwhile, the processor (120) may cluster the defects detected in the second specimen image based on defect characteristics, and may also perform multi-stage clustering.

[0081] For example, the processor (120) can perform primary clustering (classification) of defects detected in the second specimen image based on defect characteristics. That is, the processor (120) can perform primary clustering (classification) of defects detected in the second specimen image into point defects and scratch defects based on defect characteristics. Subsequently, the processor (120) can perform secondary clustering of the primary classified point defect group by further subdividing it according to the size and contrast of each point into a small point defect group and a large point defect group. Meanwhile, the processor (120) can perform secondary clustering of the scratch defect group based on the length and directionality of the scratches to divide them into a horizontal direction scratch group and a vertical direction scratch group. As described above, the present invention forms cluster groups classified by defect type as well as detailed defect characteristics through multi-stage clustering, and the processor (120) calculates the number of defects for each group and displays them on the screen, thereby enabling the user to more precisely identify the occurrence pattern and characteristics of the defects.

[0082] As described above, by clustering detected defects based on defect characteristics to calculate and visually display the types and number of defects, it is possible to summarize and analyze defects at the cluster level even when there is a vast number of defects, thereby reducing the time required for defect detection and improving accuracy.

[0083] In addition, the present invention has the effect of enhancing quality control and defect analysis in the product manufacturing process by automatically detecting and displaying specimen images on the specimen surface through artificial intelligence analysis.

[0084] In addition, since complex and minute defects can be accurately recognized at the pixel level based on artificial intelligence analysis, it has the effect of improving the accuracy of the analysis by preventing inspection errors that may occur when humans perform manual visual quality inspections of products.

[0085] In addition, by visualizing and displaying defect information such as the location, type, and similarity of defects within the specimen image, it has the effect of enabling the user (product quality inspector) to quickly and accurately grasp the situation and make judgments (easily identifying process abnormalities if they occur in specific locations or repetitive patterns).

[0086] In addition, when defect information is displayed visually, it is easier for product manufacturing process personnel, engineers, and quality control teams to identify the defect compared to text-based communication, thereby reducing the time required for defect identification and handling. In other words, it increases the efficiency of collaboration among stakeholders in defect handling, which can shorten the time required for defect resolution.

[0088] FIG. 4 is a flowchart illustrating a method for improving a specimen image according to an embodiment of the present invention.

[0089] Referring to FIG. 4, a specimen image enhancement method in a specimen image enhancement device (100) according to one embodiment of the present invention may include the step (S410) of generating a second specimen image with improved resolution by inputting a specimen image of the specimen surface stored in a specimen image storage unit (110) into a pre-learned image enhancement algorithm, and the step (S410) of detecting defects by inputting the second specimen image into a pre-learned defect analysis algorithm and analyzing it in pixel units.

[0090] In step S410, the specimen image enhancement device (100) can input the specimen image into a pre-trained image enhancement algorithm to generate a second specimen image with improved resolution. At this time, the pre-trained image enhancement algorithm may be an artificial intelligence image enhancement algorithm. Meanwhile, in describing the present embodiment, the specimen image captured by the electron scanning microscope may be the first specimen image, and the specimen image with improved resolution by the image enhancement algorithm may be the second specimen image.

[0091] Meanwhile, the specimen image enhancement device (100) may perform preprocessing on the specimen image to generate an enhanced image. That is, before generating the second specimen image, the specimen image enhancement device (100) may perform simulation-based preprocessing using pre-established simulation data to correct the noise pattern and scanning pattern of the specimen image.

[0092] In step S420, the specimen image enhancement device (100) can detect defects by inputting the second specimen image into a pre-learned defect analysis algorithm and analyzing it pixel by pixel.

[0093] The specimen image enhancement device (100) detects defects by inputting a second specimen image into a pre-learned defect analysis algorithm and analyzing it in pixel units, and clusters the detected defects based on defect characteristics and displays the type and / or number of defects for each cluster.

[0094] The specimen image enhancement device (100) can perform clustering based on at least one of the location, size, orientation contrast and shape of the defect.

[0095] The specimen image enhancement device (100) can visually display clustered defects by different colors and / or shapes according to type.

[0096] The specimen image enhancement device (100) can visually display clustered defects by different colors and / or shapes according to type, and can also display the number of defects.

[0097] The specimen image enhancement device (100) can distinguish regions in pixel units in the second specimen image, but can distinguish only the regions of interest that have defects. At this time, the defects may include at least one of scratches and dots.

[0098] The specimen image enhancement device (100) can detect defects by analyzing the second specimen image in pixel units based on a pre-learned defect analysis algorithm, recognizing defects present in the second specimen image, classifying the types of defects, and determining the similarity of defects.

[0099] The specimen image enhancement device (100) can display a pixel-unit area containing the detected defect as a bounding box, and display the type and similarity of the defect in an area adjacent to the bounding box.

[0100] The specimen image enhancement device (100) may display different colors in one area where information about the defect, such as the type and similarity of the defect, is displayed depending on the type of defect.

[0101] The specimen image enhancement device (100) can perform primary clustering (classification) of defects detected in the second specimen image into point defects and scratch defects based on defect characteristics. Subsequently, the specimen image enhancement device (100) can perform secondary clustering of the primary classified point defect group into small point defect groups and large point defect groups by further subdividing them according to the size and contrast of each point.

[0102] In addition, all operation processes of the processor (120) as described above may be included in the sample image improvement method in the sample image improvement device (100) according to one embodiment of the present invention.

[0103] Meanwhile, as described above, the sample image enhancement method in the sample image enhancement device (100) according to one embodiment of the present invention may be implemented as a computer program (or computer program product) including instructions executable by a computer. The computer program includes programmable machine instructions processed by a processor (120) and may be implemented in a high-level programming language, an object-oriented programming language, an assembly language, or a machine language. Additionally, the computer program may be recorded on a tangible computer-readable recording medium (e.g., memory, hard disk, magnetic / optical medium or SSD (Solid-State Drive), etc.).

[0105] FIG. 5 is a block diagram showing the specific configuration of a specimen image enhancement device illustrated in FIG. 1 according to an embodiment of the present invention.

[0106] Referring to FIG. 5, the specimen image enhancement device (100) includes a specimen image storage unit (110), a processor (120), and a storage unit (130).

[0107] The processor (120) controls the overall operation of the specimen image enhancement device (100).

[0108] Specifically, the processor (120) includes RAM (121), ROM (122), main CPU (123), graphics processing unit (124), first to n interfaces (125-1 to 125-n) and a bus (126).

[0109] RAM (121), ROM (122), main CPU (123), graphics processing unit (124), first to n interfaces (125-1 to 125-n), etc. can be connected to each other via a bus (126).

[0110] The first to n interfaces (125-1 to 125-n) are connected to the various components described above. One of the interfaces may be a network interface connected to an external device through a network.

[0111] The main CPU (123) accesses the storage unit (130) and performs booting using the O / S stored in the storage unit (130). Then, it performs various operations using various programs, content, data, etc. stored in the storage unit (130).

[0112] In particular, the main CPU (123) inputs the specimen image stored in the specimen image storage unit (110) into a pre-learned image enhancement algorithm to generate a second specimen image with improved resolution, and can detect defects based on the second specimen image.

[0113] A set of instructions for booting the system is stored in the ROM (122). When a turn-on command is input and power is supplied, the main CPU (123) copies the O / S stored in the storage unit (130) to the RAM (121) according to the instructions stored in the ROM (122), and executes the O / S to boot the system. When booting is complete, the main CPU (123) copies various application programs stored in the storage unit (130) to the RAM (121), and executes the application programs copied to the RAM (121) to perform various operations.

[0114] The graphics processing unit (124) generates a screen containing various objects such as icons, images, and text using a calculation unit (not shown) and a rendering unit (not shown). The calculation unit (not shown) calculates attribute values ​​such as coordinate values, shape, size, and color for each object to be displayed according to the layout of the screen based on a received control command. The rendering unit (not shown) generates a screen of various layouts containing objects based on the attribute values ​​calculated by the calculation unit (not shown).

[0115] In particular, the graphics processing unit (124) can implement objects generated by the main CPU (123) into a GUI (Graphic User Interface), icon, user interface screen, etc.

[0116] Meanwhile, the operation of the above-described processor (120) can be performed by a program stored in the storage unit (130).

[0117] The storage unit (130) stores various data, such as an O / S (Operating System) software module for operating the specimen image enhancement device (100) and various multimedia content.

[0118] In particular, the storage unit (130) may include a software module for generating a second specimen image with improved resolution by inputting the specimen image stored in the specimen image storage unit (110) into a pre-learned image enhancement algorithm and detecting defects based on the second specimen image.

[0120] FIG. 6 is a drawing relating to a software module stored in a storage unit according to an embodiment of the present invention.

[0121] Referring to FIG. 6, the storage unit (130) may store programs such as a second specimen image generation module (131) and a defect detection module (132).

[0122] Meanwhile, the operation of the processor (120) described above can be performed by a program stored in the storage unit (130). Below, the detailed operation of the processor (120) using the program stored in the storage unit (130) will be explained in detail.

[0123] Specifically, the second specimen image generation module (131) can generate a second specimen image with improved resolution by inputting the specimen image into a pre-trained image enhancement algorithm. At this time, the pre-trained image enhancement algorithm may be an artificial intelligence image enhancement algorithm. Meanwhile, in describing the present embodiment, the specimen image captured by the electron scanning microscope may be the first specimen image, and the specimen image with improved resolution by the image enhancement algorithm may be the second specimen image.

[0124] Meanwhile, a separate preprocessing module (not shown) may perform preprocessing on the specimen image to generate an enhanced image. That is, before the preprocessing module (not shown) generates a second specimen image, simulation-based preprocessing may be performed on the specimen image using pre-built simulation data to correct the noise pattern and scanning pattern of the specimen image.

[0125] The defect detection module (132) can detect defects by inputting the second specimen image into a pre-learned defect analysis algorithm and analyzing it pixel by pixel.

[0126] The defect detection module (132) detects defects by inputting the second specimen image into a pre-learned defect analysis algorithm and analyzing it pixel by pixel, and can cluster the detected defects based on defect characteristics and display the type and / or number of defects for each cluster.

[0127] The defect detection module (132) can perform clustering based on at least one of the location, size, orientation contrast and shape of the defect.

[0128] The defect detection module (132) can visually display clustered defects by different colors and / or shapes according to type.

[0129] The defect detection module (132) can visually display clustered defects by type using different colors and / or shapes, and can also display the number of defects.

[0130] The defect detection module (132) can distinguish regions in pixel units in the second specimen image, but can distinguish only the regions of interest that have defects. At this time, the defect may include at least one of a scratch and a dot.

[0131] The defect detection module (132) can detect defects by analyzing the second specimen image in pixel units based on a pre-learned defect analysis algorithm, recognizing defects present in the second specimen image, classifying the types of defects, and determining the similarity of defects.

[0132] The defect detection module (132) can display a pixel-unit area containing the detected defect as a bounding box, and display the type and similarity of the defect in an area adjacent to the bounding box.

[0133] The defect detection module (132) may display different colors in the area where information about the defect, such as the type and similarity of the defect, is displayed, depending on the type of defect.

[0134] The defect detection module (132) can perform primary clustering (classification) of the defects detected in the second specimen image into point defects and scratch defects based on defect characteristics. Subsequently, the specimen image enhancement device (100) can perform secondary clustering of the primary classified point defect group into small point defect groups and large point defect groups by further subdividing them according to the size and contrast of each point.

[0135] Meanwhile, a non-transitory computer-readable medium storing a program for sequentially performing the method according to the present invention may be provided.

[0136] A non-transient readable medium refers to a medium that stores data semi-permanently and can be read by a device, rather than a medium that stores data for a short period of time, such as a register, cache, or memory. Specifically, the various applications or programs described above may be stored and provided on non-transient readable media such as CDs, DVDs, hard disks, Blu-ray discs, USBs, memory cards, and ROMs.

[0137] In addition, although the bus is not shown in the block diagram above illustrating the specimen image enhancement device (100), communication between each component of the specimen image enhancement device (100) may be performed via the bus. Additionally, each device may further include a processor, such as a CPU or a microprocessor, that performs the various steps described above.

[0138] Furthermore, although preferred embodiments of the present invention have been illustrated and described above, the present invention is not limited to the specific embodiments described above. It is understood that various modifications can be made by those skilled in the art without departing from the essence of the invention as claimed in the claims, and such modifications should not be understood individually from the technical spirit or perspective of the present invention. Explanation of the symbols

[0140] 100: Specimen image enhancement device 110 : Psalm Image Storage 120 : Processor 130 : Storage unit

Claims

Claim 1 A specimen image enhancement device comprising: a specimen image storage unit for storing a specimen image of the surface of a specimen; and a processor for processing the specimen image; wherein the processor performs simulation-based preprocessing on the specimen image using pre-established simulation data to correct noise patterns and scanning patterns occurring during the process of being captured by a scanning electron microscope, inputs the specimen image in which the noise patterns and scanning patterns are corrected through the simulation-based preprocessing into a pre-learned image enhancement algorithm to generate a second specimen image with improved resolution, and inputs the second specimen image into a pre-learned defect analysis algorithm to detect defects present in the second specimen image at the pixel level. Claim 2 delete Claim 3 A specimen image enhancement device according to claim 1, wherein the processor detects defects by inputting the second specimen image into a pre-learned defect analysis algorithm and analyzing it pixel by pixel, clusters the detected defects based on defect characteristics, and displays the type and number of defects for each cluster. Claim 4 A specimen image enhancement device according to claim 3, wherein the processor performs clustering based on at least one of the location, size, direction, contrast, and shape of a defect. Claim 5 A specimen image enhancement device according to claim 3, wherein the processor visually displays the clustered defects by different colors or shapes according to type. Claim 6 A specimen image enhancement device according to claim 5, wherein the processor visually displays the clustered defects by type using different colors or shapes, and also displays the number of defects. Claim 7 A specimen image enhancement device according to claim 3, wherein the processor distinguishes regions in pixel units in the second specimen image, but distinguishes only the defective regions of interest. Claim 8 A specimen image enhancement device according to claim 7, wherein the defect comprises at least one of a scratch and a dot, and the processor analyzes a second specimen image in pixel units based on the pre-learned defect analysis algorithm, and detects the defect by recognizing the defect present in the second specimen image, classifying the type of the defect, and determining the similarity of the defect. Claim 9 A specimen image enhancement device according to claim 8, wherein the processor marks a pixel-unit area containing the detected defect as a bounding box, and marks the type and similarity of the defect in an area adjacent to the bounding box. Claim 10 A specimen image enhancement device according to claim 9, wherein the processor displays the color of a region in which the type of defect and similarity are indicated differently depending on the type of defect. Claim 11 A specimen image enhancement device according to claim 3, wherein the processor performs multi-stage clustering in which it performs primary clustering of the detected defects based on defect characteristics and secondary clustering of the primary clustered defects based on detailed defect characteristics. Claim 12 A method for improving a specimen image in a specimen image improvement device, comprising: a step of performing simulation-based preprocessing on the specimen image using pre-established simulation data to correct noise patterns and scanning patterns occurring during the process of being captured by a scanning electron microscope; a step of generating a second specimen image with improved resolution by inputting the specimen image, in which the noise patterns and scanning patterns are corrected through the simulation-based preprocessing, into a pre-trained image improvement algorithm; and a step of detecting defects present in the second specimen image in pixel units by inputting the second specimen image into a pre-trained defect analysis algorithm.

Images