Ai-based display inspection method

Abstract

An artificial intelligence-based display inspection method is disclosed. The artificial intelligence-based display inspection method of the present invention comprises: a pattern display step of displaying an inspection pattern on a display panel; a first shooting step of capturing the display panel on which the inspection pattern is displayed to obtain a first inspection image; a defect candidate detection step of calculating a defect candidate region and a reliability of the defect candidate region from the first inspection image using a learned first artificial intelligence model; a determination step of determining that if the reliability is greater than or equal to a preset judgment threshold value, the defect candidate region is confirmed as a defect, and if the reliability is less than the judgment threshold value and greater than or equal to a preset re-inspection threshold value, a re-inspection step is determined; and a re-inspection step of generating a re-inspection pattern in which the grayscale of a region corresponding to the defect candidate region is changed from the inspection pattern based on location information of the defect candidate region, capturing the display panel on which the re-inspection pattern is displayed to obtain a second inspection image, and using a learned second artificial intelligence model to determine whether the defect candidate region is defective and the type of defect based on the change in brightness of the defect candidate region between the first inspection image and the second inspection image.

Description

Technology Field

[0001] The present invention relates to a display inspection method, and more specifically, to an artificial intelligence-based display inspection method that uses an artificial intelligence model to detect screen defects in the manufacturing process of a display panel applied to a TV, etc. Background Technology

[0002] In the manufacturing process of display panels used in TVs and the like, illumination inspection is performed to detect screen defects prior to shipment. This inspection is conducted by photographing the screen with a camera while displaying specific inspection patterns—such as white, black, solid colors, and intermediate gradations—on the panel, and then detecting defects like bright spots, dark spots, line defects, and stains from the captured images. Since the display panel is branched into repair, cleaning, disposal, or shipment processes depending on the type of defect detected, not only the accuracy of defect detection but also the accuracy of identifying defect types directly impacts manufacturing yield and shipment quality.

[0003] Traditionally, visual inspection, in which an inspector directly observes an illuminated screen, has been primarily used. However, visual inspection makes it difficult to ensure consistency in inspection quality as judgment results vary depending on the inspector's skill level and fatigue. Furthermore, as display panels become larger and higher-resolution, the visual detection of minute defects has reached its limits, and there are issues regarding its application to mass production lines in terms of securing inspection personnel and inspection time.

[0004] Rule-based automatic optical inspection (AOI) has been introduced to compare the luminance value of a captured image with a threshold value or to calculate the difference in luminance with surrounding areas. However, rule-based automatic inspection has low detection performance for stain defects, which have irregular shapes, sizes, and luminance distributions. It also has the inconvenience of having to reset inspection parameters, such as threshold values, whenever the inspection target model or inspection pattern changes, and frequently suffers from over-detection, where normal areas with luminance distributions similar to defects are judged as defects.

[0005] Recently, technologies for detecting defects from captured images using artificial intelligence models such as Convolutional Neural Networks (CNNs) are being introduced. Inspection using AI models autonomously learns defect characteristics from training data in which defect images are labeled; therefore, it offers the advantages of superior detection performance for irregular defects compared to rule-based inspection and reduced burden of manually setting inspection parameters.

[0006] However, even when using artificial intelligence models, the following problems still exist.

[0007] It is difficult to distinguish whether abnormal areas appearing in captured images are pixel defects within the display panel itself or are caused by foreign matter attached to the panel surface or the camera optical system. Since pixel defects and foreign matter often appear with similar shapes and brightness distributions in a single static inspection image, artificial intelligence models that rely solely on a single image input face limitations in distinguishing between the two. If a visual defect caused by foreign matter is misidentified as a pixel defect, good panels are unnecessarily sent to the repair process or discarded, leading to a decrease in yield; conversely, if a pixel defect is misidentified as foreign matter, defective panels are shipped, resulting in quality issues. Furthermore, while subsequent measures (panel cleaning or maintenance of inspection equipment) vary depending on whether the foreign matter is attached to the panel surface or the camera optical system, conventional inspection methods make it difficult to pinpoint the exact location of attachment.

[0008] The handling of borderline cases, where the judgment reliability of the AI ​​model is low, is problematic. AI models output a probabilistic reliability regarding the presence of defects; if defect candidates with intermediate reliability are uniformly judged as defects, yield reduction occurs due to over-detection, and if they are uniformly judged as normal, quality degradation occurs due to non-detection. Although a method of transferring these borderline cases to a visual re-inspection by an inspector is currently used, the overall inspection time increases due to waiting and re-inspection execution, and the judgment of the re-inspection also depends on the inspector's proficiency, thus undermining the purpose of introducing automated inspection. In this case, simply re-photographing the borderline cases using the same inspection pattern yields images identical to the initial inspection, and therefore fails to provide new grounds for judgment.

[0009] There are issues stemming from the conditions of acquiring inspection images. When capturing large display panels with a single camera, conventional imaging optical systems exhibit perspective distortion and magnification deviations between the center and periphery of the screen. Consequently, positional errors become larger towards the edges of the screen during the process of converting defect locations in the inspection image into pixel coordinates of the display panel. Furthermore, if illumination from the inspection environment is specularly reflected from the panel surface and incident on the camera, reflected images unrelated to the panel's self-luminescence are superimposed on the inspection image, causing false detections. Additionally, vibrations transmitted from conveyor systems surrounding the inspection equipment induce misalignment of image positions between shooting times. Moreover, if foreign substances such as dust adhere to the front of the camera lens in a manufacturing line environment, these substances continuously appear in the inspection image, leading to false detections of panel defects.

[0010] Therefore, there is a need for inspection technology that can determine the presence of defects without human intervention in boundary cases where the judgment of the artificial intelligence model is withheld, distinguish between pixel defects and defects caused by foreign substances, and eliminate errors and false detection factors attributable to the acquisition conditions of inspection images.

[0011] The aforementioned technical configuration is provided as background technology to aid in understanding the present invention and does not constitute prior art widely known in the technical field to which the present invention belongs. Prior art literature

[0012] Korean Registered Patent Publication No. 10-2188568 (December 2, 2020) The problem to be solved

[0013] The problem that the present invention aims to solve is to provide an AI-based display inspection method capable of determining the presence of defects without human intervention by automatically performing a re-inspection with an adaptive change in the inspection pattern for defect candidates with low judgment reliability of the AI ​​model.

[0014] Another problem that the present invention aims to solve is to provide an artificial intelligence-based display inspection method capable of distinguishing between pixel defects and appearance defects caused by foreign substances in a display panel based on the luminance response according to the grayscale change of a defect candidate region.

[0015] Another problem that the present invention aims to solve is to provide an artificial intelligence-based display inspection method in which pixel-level alignment between a first inspection image and a second inspection image is optically and mechanically guaranteed, foreign matter attached to the imaging optical system is excluded prior to re-inspection, and the judgment result of the re-inspection can be identified as a defect on the display panel side.

[0016] The problems of the present invention are not limited to those mentioned above, and other problems not mentioned will be clearly understood by those skilled in the art from the description below. means of solving the problem

[0017] According to one aspect of the present invention, an artificial intelligence-based display inspection method may be provided, comprising: a pattern display step of displaying an inspection pattern on a display panel; a first shooting step of capturing the display panel on which the inspection pattern is displayed to obtain a first inspection image; a defect candidate detection step of calculating a defect candidate region and a reliability of the defect candidate region from the first inspection image using a learned first artificial intelligence model; a determination step of determining that if the reliability is greater than or equal to a preset judgment threshold value, the defect candidate region is confirmed as a defect, and if the reliability is less than the judgment threshold value and greater than or equal to a preset re-inspection threshold value, the performance of a re-inspection step is determined; and a re-inspection step of generating a re-inspection pattern in which the grayscale of a region corresponding to the defect candidate region is changed from the inspection pattern based on location information of the defect candidate region, capturing the display panel on which the re-inspection pattern is displayed to obtain a second inspection image, and using a learned second artificial intelligence model to determine whether the defect candidate region is defective and the type of defect based on the change in brightness of the defect candidate region between the first inspection image and the second inspection image.

[0018] The above re-inspection step generates the re-inspection pattern by inverting the grayscale of the area corresponding to the defect candidate area from the grayscale of the inspection pattern and maintaining the grayscale of the remaining area identical to the inspection pattern, and using the second artificial intelligence model, if the brightness of the defect candidate area changes according to the inversion of the grayscale, the defect candidate area is determined to be a pixel defect of the display panel, and if the brightness of the defect candidate area is maintained despite the inversion of the grayscale, the defect candidate area is determined to be an appearance defect caused by foreign matter.

[0019] The first inspection image and the second inspection image are acquired by the same shooting unit, and

[0020] The above-described imaging unit comprises a housing, an image sensor module disposed inside the housing, a telecentric lens unit disposed on the light path between the image sensor module and the display panel to cause the main light ray to be incident parallel to the optical axis at the center and periphery of the screen of the display panel, a polarizing filter unit disposed in front of the telecentric lens unit to block the specular reflection component of external light reflected from the surface of the display panel, a protective window coupled to the front end of the housing to seal the light path from the outside, a lens cleaning unit that sprays air onto the outer surface of the protective window, and a vibration isolation unit that elastically supports the housing with respect to a support bracket, wherein in the defect candidate detection step, the location information of the defect candidate area is calculated as pixel coordinates of the display panel from the first inspection image in which perspective distortion is excluded by the telecentric lens unit, so that the area where the gradation of the re-inspection pattern has changed is aligned with the defect candidate area, and the lens cleaning unit sprays air onto the outer surface of the protective window after the first imaging step and before acquiring the second inspection image to remove foreign matter attached to the protective window, and the re-inspection The step can identify the defect candidate area determined to be an external defect caused by foreign matter as a defect caused by foreign matter on the display panel side by using the second inspection image, which maintains alignment with the first inspection image at the pixel level by the vibration isolation unit. Effects of the invention

[0021] According to an embodiment of the present invention, for a defect candidate with low reliability, a re-inspection pattern is automatically generated by changing only the gradation of the defect candidate area to perform a re-inspection, so that the defect status of a boundary case can be determined without the intervention of an inspector, thereby shortening the inspection time and reducing over-detection and non-detection.

[0022] Since pixel defects and external defects caused by foreign substances in a display panel can be distinguished based on the luminance response of defect candidate regions to changes in grayscale, subsequent processes (repair, cleaning, etc.) can be accurately branched according to the type of defect.

[0023] Since the pixel coordinates of the defect candidate area are calculated without perspective distortion in the center and periphery of the screen by the telecentric lens unit, the grayscale change area of ​​the re-inspection pattern is accurately aligned without deviating from the defect candidate area, thereby improving the reliability of the re-inspection judgment.

[0024] Since foreign matter attached to the protective window is removed by air injection from the lens cleaning unit between the first shooting and the acquisition of the second inspection image, foreign matter on the shooting optical system side is excluded before re-inspection, so that defect candidates judged as foreign matter during re-inspection are identified as foreign matter on the display panel side, thereby preventing unnecessary input into repair processes or disposal of good products due to errors in foreign matter judgment.

[0025] Since the housing is elastically supported against the support bracket by the vibration isolation unit and the specular reflection component of external light is blocked by the polarization filter unit, pixel-level alignment between the first inspection image and the second inspection image is maintained, and the change in brightness between the two images is caused solely by the change in grayscale of the re-inspection pattern, thereby improving the accuracy of defect judgment based on brightness change.

[0026] The effects according to the present invention are not limited to those exemplified above, and various other effects are included in this specification. Brief explanation of the drawing

[0027] FIG. 1 is a configuration diagram of an inspection system in which an artificial intelligence-based display inspection method according to an embodiment of the present invention is performed. FIG. 2 is a flowchart of an artificial intelligence-based display inspection method according to an embodiment of the present invention. FIG. 3 is a diagram illustrating an inspection pattern and a re-inspection pattern in accordance with an embodiment of the present invention. FIG. 4 is a cross-sectional view illustrating a shooting unit according to one embodiment of the present invention. Specific details for implementing the invention

[0028] In order to fully understand the present invention, the operational advantages of the present invention, and the objectives achieved by the implementation of the present invention, reference must be made to the accompanying drawings illustrating preferred embodiments of the present invention and the contents described therein.

[0029] The present invention will be described in detail below by explaining preferred embodiments of the invention with reference to the attached drawings. Identical reference numerals in each drawing indicate identical components.

[0030] Hereinafter, an artificial intelligence-based display inspection method according to an embodiment of the present invention will be described with reference to FIGS. 1 to 4.

[0031] Referring to FIG. 1, the inspection system (1) includes a pattern supply unit (100), a shooting unit (200), a detection unit (300), a judgment unit (400), and a re-inspection unit (500). The display panel (10) to be inspected may be a liquid crystal display panel (LCD), an organic light-emitting diode display panel (OLED), a micro LED panel, etc., and the inspection method according to the present embodiment may be applied not only to large panels for TVs but also to panels for monitors, vehicles, etc. The display panel (10) may be loaded onto an inspection stage and inspected in a cell state where the module process is completed or in a finished product state where the set assembly is completed.

[0032] The pattern supply unit (100) applies an image signal of an inspection pattern (P1) to a display panel (10) that is to be inspected. The pattern supply unit (100) may be composed of a pattern generator connected to a driving board of the display panel (10), and in the inspection of a finished product, an image signal may be applied through an image input terminal of the set. The pattern supply unit (100) may include a pattern storage unit (110) in which a plurality of inspection patterns (P1) are stored. The inspection patterns (P1) may include a white pattern, a black pattern, a red / green / blue single-color pattern, a gray pattern having a plurality of gray levels, a checkerboard pattern, etc., and a plurality of inspection patterns (P1) may be displayed sequentially according to a preset order. The type of detectable defect varies depending on the type of inspection pattern (P1). For example, since bright spot defects are relatively easily visible in the black pattern, dark spot defects in the white pattern, and mura defects in the intermediate grayscale pattern, various types of defects can be detected without omission by sequentially displaying multiple inspection patterns (P1).

[0033] The shooting unit (200) captures the screen of a display panel (10) on which an inspection pattern (P1) is displayed to obtain a first inspection image (IMG1). The shooting unit (200) may be composed of an area scan camera or a line scan camera and is positioned so that the entire screen of the display panel (10) is included in the shooting field of view. It is preferable that the resolution of the shooting unit (200) be set such that a single pixel of the display panel (10) corresponds to a multiple pixel of the first inspection image (IMG1) so that individual pixel defects of the display panel (10) can be identified. In another embodiment, the shooting unit (200) may include a plurality of cameras to divide and capture the screen of the display panel (10), and align the divided images to generate a first inspection image (IMG1). The exposure time and gain of the shooting unit (200) can be set differently depending on the gradation of the displayed inspection pattern (P1), and the exposure time can be set as an integer multiple of the frame cycle of the display panel (10) so that flicker caused by interference between the driving frequency of the display panel (10) and the shooting cycle does not appear in the first inspection image (IMG1).

[0034] The detection unit (300) includes a preprocessing unit (320) and a learned first artificial intelligence model (310).

[0035] The preprocessing unit (320) can perform at least one of lens distortion correction, moiré reduction filtering by pixel alignment between the shooting unit (200) and the display panel (10), and shading correction to correct luminance non-uniformity between the center and the periphery of the screen for the first inspection image (IMG1). The preprocessing unit (320) can also calculate a conversion relationship between the coordinates on the first inspection image (IMG1) and the pixel coordinates of the display panel (10). The coordinate conversion relationship can be obtained by displaying an alignment mark pattern on the four corners of the display panel (10) before the inspection begins, photographing it, and calibrating it. This coordinate conversion relationship is used to convert the image coordinates of the defect candidate area (DA) into pixel coordinates of the display panel (10) when generating the re-inspection pattern (P2) described later.

[0036] The first artificial intelligence model (310) is an object detection model trained with training data in which an inspection image containing defects and defect locations are labeled, and receives the first inspection image (IMG1) as input and outputs location information of a defect candidate region (DA) and confidence indicating the probability that the defect candidate region (DA) is a defect. The first artificial intelligence model (310) may be composed of a convolutional neural network-based object detection network (e.g., YOLO, Faster R-CNN series) or a segmentation network that divides defect regions into pixel units. The training data of the first artificial intelligence model (310) may consist of data in which defect locations and defect types are labeled on defect images collected from an actual production line, and for defect types where training data is insufficient due to a low defect occurrence frequency, data augmentation may be performed by synthesizing the defect image onto a normal image. Along with the location information and confidence of the defect candidate region (DA), the first artificial intelligence model (310) may also output classification information for defect types such as bright spots, dark spots, line defects, and stains.

[0037] The location information of the defect candidate region (DA) can be calculated as a value obtained by converting the coordinates on the first inspection image (IMG1) into pixel coordinates of the display panel (10) according to the coordinate transformation relationship calculated by the preprocessing unit (320). Multiple defect candidate regions (DA) may be detected from a single first inspection image (IMG1), and in this case, the reliability is calculated individually for each defect candidate region (DA).

[0038] The judgment unit (400) compares the reliability with a pre-set judgment standard value (TH1) and a re-examination standard value (TH2). The re-examination standard value (TH2) is set lower than the judgment standard value (TH1). If the reliability is greater than or equal to the judgment standard value (TH1), the judgment unit (400) confirms the defect candidate area (DA) as a defect. If the reliability is less than the re-examination standard value (TH2), the judgment unit (400) determines the defect candidate area (DA) as normal. If the reliability is less than the judgment standard value (TH1) and greater than or equal to the re-examination standard value (TH2), the judgment unit (400) reserves the confirmation of whether there is a defect and instructs the re-examination unit (500) to re-examine.

[0039] The judgment criterion value (TH1) and the re-inspection criterion value (TH2) may be set differently depending on the type of defect. For example, for stain defects that cause significant yield loss upon false detection, the judgment criterion value (TH1) may be set relatively high, and for bright spot defects that have a significant impact on quality upon non-detection, the re-inspection criterion value (TH2) may be set relatively low. Additionally, the judgment criterion value (TH1) and the re-inspection criterion value (TH2) may be statistically determined from the reliability distribution of the first artificial intelligence model (310) for the verification dataset to satisfy the target over-detection rate and the target non-detection rate. By separately setting a reliability interval in which judgment is withheld in this way, the over-detection and non-detection that occur when boundary cases are uniformly judged as defects or normal are reduced.

[0040] The re-examination unit (500) includes a re-examination pattern generation unit (510) and a second artificial intelligence model (520).

[0041] The re-inspection pattern generation unit (510) generates a re-inspection pattern (P2) based on location information of a defect candidate region (DA). The re-inspection pattern (P2) is a pattern in which the grayscale of the region corresponding to the defect candidate region (DA) is changed from the inspection pattern (P1), and the grayscale of the remaining region is maintained the same as the inspection pattern (P1). In one embodiment, the re-inspection pattern generation unit (510) can generate the re-inspection pattern (P2) by inverting the grayscale of the region corresponding to the defect candidate region (DA) from the grayscale of the inspection pattern (P1). For example, if the inspection pattern (P1) is a white pattern, the re-inspection pattern (P2) may be a pattern in which only the defect candidate region (DA) is inverted to a black grayscale (see FIG. 3).

[0042] The re-inspection pattern generation unit (510) can generate a re-inspection pattern (P2) by setting a margin area (MA) that is extended by a predetermined margin from the defect candidate area (DA) and changing the grayscale of the margin area (MA). The margin area (MA) is intended to absorb positional errors and coordinate transformation errors of the defect candidate area (DA) output by the first artificial intelligence model (310), and can be set by extending from the outer edge of the defect candidate area (DA) by a predetermined number of pixels. Accordingly, even if there is an error in the position calculation of the defect candidate area (DA), the defect area is prevented from deviating from the grayscale change area.

[0043] In one embodiment, when a plurality of defect candidate regions (DA) spaced apart from each other are detected in a first inspection image (IMG1) and the reliability of each defect candidate region (DA) corresponds to a re-inspection target section, the re-inspection pattern generation unit (510) can generate a single re-inspection pattern (P2) in which the grayscale of a plurality of regions corresponding to the plurality of defect candidate regions (DA) is simultaneously changed. Accordingly, compared to the case where a re-inspection pattern (P2) is generated individually for each defect candidate region (DA) and sequentially captured, the number of captures and inspection time required for re-inspection are reduced.

[0044] When the pattern supply unit (100) displays the re-inspection pattern (P2) on the display panel (10), the shooting unit (200) re-photographs the display panel (10) on which the re-inspection pattern (P2) is displayed to obtain a second inspection image (IMG2). At this time, the display panel (10) is not moved on the inspection stage, and the shooting position, exposure conditions, and optical conditions of the shooting unit (200) are maintained the same as when the first inspection image (IMG1) was obtained. Therefore, the first inspection image (IMG1) and the second inspection image (IMG2) can be aligned at the pixel level, and since the difference in brightness between the two images is caused solely by a change in the display gradation, the judgment based on brightness change described later can be performed without disturbance.

[0045] The second artificial intelligence model (520) determines whether there is a defect and the type of defect based on the change in brightness of the defect candidate region (DA) in the first inspection image (IMG1) and the second inspection image (IMG2). In one embodiment, the second artificial intelligence model (520) may be composed of a classification model that receives the patch image of the first inspection image (IMG1) and the patch image of the second inspection image (IMG2) regarding the defect candidate region (DA) as input. The second artificial intelligence model (520) may be composed of a shared weight structure (Sham network structure) that extracts features from each of the two patch images, and a difference image representing the pixel-by-pixel difference between the two patch images may be input together as an additional input channel. Unlike the first artificial intelligence model (310), the second artificial intelligence model (520) processes only the patch image around the defect candidate region (DA) rather than the entire screen, so the amount of computation for re-inspection is significantly reduced compared to the case where the entire screen is re-analyzed.

[0046] The determination principle of the second artificial intelligence model (520) is as follows. Since pixel defects (bright spots, dark spots, etc.) of the display panel (10) are defects in which a specific pixel responds abnormally to an applied image signal, the visibility pattern changes according to the grayscale inversion. For example, a dark spot visible in a white pattern becomes indistinguishable from the surroundings when the defect candidate area (DA) is inverted to a black grayscale, and a bright spot that emits light constantly becomes more clearly visible in the area inverted to a black grayscale. On the other hand, foreign matter attached to the panel surface or the imaging optical system appears in the same location on the image with the same shape and similar relative brightness regardless of the display grayscale. Therefore, the second artificial intelligence model (520) can determine the defect candidate area (DA) as a pixel defect of the display panel (10) if the brightness and visibility pattern of the defect candidate area (DA) change according to the grayscale inversion, and can determine it as an appearance defect caused by foreign matter if the brightness and visibility pattern are maintained despite the grayscale inversion.

[0047] In another embodiment, the re-inspection pattern generation unit (510) generates a plurality of re-inspection patterns (P2) by sequentially changing the grayscale of an area corresponding to a defect candidate region (DA) to a plurality of grayscale levels, and the imaging unit (200) can acquire a plurality of second inspection images (IMG2) for each re-inspection pattern (P2). In this case, the second artificial intelligence model (520) can determine whether there is a defect and the type of defect based on the luminance response curve of the defect candidate region (DA) according to the change in grayscale level. While the luminance of a normal pixel changes according to the gamma characteristic with respect to the applied grayscale, the luminance of a pixel defect exhibits a response that deviates from the gamma characteristic, and the luminance of a foreign substance is substantially unresponsive to the change in grayscale, so the type of defect can be distinguished more precisely from the shape of the luminance response curve. This embodiment is particularly effective for determining low-luminosity stain defects that are difficult to determine with only a single grayscale inversion.

[0048] Depending on the type of defect determined, the display panel (10) may be branched into different subsequent processes. For example, a display panel (10) determined to have a pixel defect may be transferred to a repair process, a display panel (10) determined to have an appearance defect may be transferred to a re-inspection line after undergoing a cleaning process, and a display panel (10) determined to be normal may be transferred to an inspection or shipment process for a subsequent inspection pattern. Since the subsequent processes are automatically branched according to the type of defect in this way, it is prevented that a good panel is discarded or unnecessarily fed into the repair process due to an appearance defect caused by foreign matter being misjudged as a pixel defect.

[0049] Meanwhile, the judgment result of the re-examination unit (500) can be stored in a database along with the first inspection image (IMG1), the second inspection image (IMG2), and the patch image of the defect candidate region (DA) that served as the basis for the judgment, and the stored data can be used as training data for retraining the first artificial intelligence model (310). Since the defect candidate subject to re-examination corresponds to a boundary case that was difficult for the first artificial intelligence model (310) to judge, the judgment performance of the first artificial intelligence model (310) is continuously improved by assigning the judgment result confirmed by the re-examination as a label and retraining, and as a result, the frequency of occurrence of defect candidates requiring re-examination itself is gradually reduced.

[0050] Referring to FIG. 2, the artificial intelligence-based display inspection method according to the present embodiment is described as follows. The artificial intelligence-based display inspection method includes a pattern display step (S100), a first shooting step (S200), a defect candidate detection step (S300), a judgment step (S400), and a re-inspection step (S500).

[0051] In the pattern display step (S100), the pattern supply unit (100) displays the inspection pattern (P1) on the display panel (10). In the first shooting step (S200), the shooting unit (200) captures the display panel (10) on which the inspection pattern (P1) is displayed to obtain the first inspection image (IMG1). In the defect candidate detection step (S300), the preprocessing unit (320) preprocesses the first inspection image (IMG1), and then the first artificial intelligence model (310) calculates the defect candidate region (DA) and reliability from the first inspection image (IMG1). In the judgment step (S400), the judgment unit (400) compares the reliability with the judgment criterion value (TH1) and the re-inspection criterion value (TH2) to determine whether to confirm the defect, determine it to be normal, or perform a re-inspection.

[0052] The re-inspection step (S500) includes a re-inspection pattern generation step (S510) for generating a re-inspection pattern (P2), a second shooting step (S520) for capturing a display panel (10) displaying the re-inspection pattern (P2) to acquire a second inspection image (IMG2), and a re-evaluation step (S530) in which a second artificial intelligence model (520) determines whether there is a defect and the type of defect based on the brightness change of a defect candidate area (DA). If the determination is not finalized even in the re-evaluation step (S530), the re-inspection step (S500) may be repeated a preset number of times with different grayscale levels, and only defect candidate areas (DA) for which the determination is not finalized even after repeated execution may be transferred to a visual re-inspection by an inspector. Accordingly, the number of subjects for re-inspection by the inspector is minimized, thereby shortening the overall inspection time.

[0053] FIG. 4 is a cross-sectional view illustrating a shooting unit according to one embodiment of the present invention.

[0054] Referring to FIG. 4, the shooting unit (200) captures an inspection pattern (P1) and a re-inspection pattern (P2) displayed on a display panel (10) to obtain a first inspection image (IMG1) and a second inspection image (IMG2). The shooting unit (200) may be configured so that the first inspection image (IMG1) and the second inspection image (IMG2) are obtained under identical or substantially identical shooting conditions.

[0055] To this end, the shooting unit (200) may include a housing (210), an image sensor module (220), a telecentric lens unit (230), a polarizing filter unit (240), a focus adjustment unit (250), an optical axis alignment unit (260), a vibration isolation unit (270), a lens cleaning unit (280), a protective window (290), and a control connector (295).

[0056] The housing (210) forms the outer shape of the shooting unit (200) and can accommodate an image sensor module (220), a telecentric lens unit (230), a polarizing filter unit (240), a focus adjustment unit (250), and an optical axis alignment unit (260) inside. The housing (210) may be formed with a sealed or semi-sealed structure to prevent external dust, moisture, or foreign matter from entering the interior of the shooting unit (200).

[0057] The housing (210) may be formed in a tubular shape extending along the optical axis direction. An image sensor module (220) may be disposed on one side of the housing (210), and a telecentric lens unit (230), a polarizing filter unit (240), a lens cleaning unit (280), and a protective window (290) may be disposed sequentially on the other side of the housing (210).

[0058] An optical path connecting an image sensor module (220) and a telecentric lens unit (230) may be formed inside the housing (210). The optical path may be configured so that the imaging surface of the image sensor module (220) and the optical axis of the telecentric lens unit (230) coincide with each other or are located within a preset allowable range.

[0059] The image sensor module (220) converts light transmitted from the display panel (10) into an electrical image signal. The image sensor module (220) may include a CCD image sensor or a CMOS image sensor.

[0060] The image sensor module (220) may be configured in an area scan method that captures the entire screen of the display panel (10) as a single image. In another embodiment, the image sensor module (220) may be configured in a line scan method that continuously captures the display panel (10) according to the relative movement of the display panel (10) or the capturing unit (200).

[0061] The image sensor module (220) can be controlled to acquire the first inspection image (IMG1) and the second inspection image (IMG2) with the same exposure time, the same sensor gain, and the same frame conditions. This prevents the mixing of brightness changes caused by the difference between the inspection pattern (P1) and the re-inspection pattern (P2) and brightness changes caused by changes in shooting conditions.

[0062] The image sensor module (220) can be installed so as to be movable in the optical axis direction inside the housing (210). The optical axis position of the image sensor module (220) can be adjusted by the focus adjustment unit (250).

[0063] The telecentric lens unit (230) transmits light incident from the display panel (10) to the image sensor module (220). The telecentric lens unit (230) may be composed of an object-side telecentric lens, an upper-side telecentric lens, or both-side telecentric lenses.

[0064] The telecentric lens unit (230) ensures that changes in the magnification of the captured image are suppressed even if the distance between the display panel (10) and the capturing unit (200) is partially changed. This minimizes the difference in the location and size of the defect candidate area (DA) between the first inspection image (IMG1) and the second inspection image (IMG2).

[0065] In particular, even if the position of the display panel (10) changes slightly or bending occurs on the surface of the display panel (10) during the return process of the display panel (10), the telecentric lens unit (230) can reduce shape distortion and magnification error of the defect candidate region (DA). Therefore, the accuracy of aligning the first inspection image (IMG1) and the second inspection image (IMG2) at the pixel level can be improved.

[0066] The telecentric lens portion (230) can be supported by the optical axis alignment portion (260) inside the housing (210). The telecentric lens portion (230) can be rotated or tilted within a preset angle range centered on the optical axis alignment portion (260).

[0067] The polarizing filter unit (240) is positioned in front of or behind the telecentric lens unit (230) and selectively transmits light of a specific polarization direction among the light incident from the display panel (10) to the imaging unit (200).

[0068] The polarizing filter section (240) can reduce specular reflections occurring on the surface of the display panel (10), the protective film, or the cover glass. Accordingly, it is possible to prevent hot spots, glare, or reflective stains occurring in the captured image from being mistaken for defect candidate areas (DA).

[0069] The polarizing filter unit (240) can be installed so as to be rotatable around the optical axis of the telecentric lens unit (230). The rotation angle of the polarizing filter unit (240) can be adjusted according to the polarization characteristics of the display panel (10), the type of inspection pattern (P1), or ambient lighting conditions.

[0070] In one embodiment, the polarization filter unit (240) may include a polarization filter and a filter rotation ring. A plurality of positioning grooves or angle indicators may be formed on the filter rotation ring, and the housing (210) may be provided with a position fixing part that is selectively coupled to the positioning grooves. Accordingly, the polarization filter unit (240) can maintain a state adjusted to a preset polarization angle.

[0071] The focus adjustment unit (250) adjusts the focus of the shooting unit (200) by moving one or more of the image sensor module (220) and the telecentric lens unit (230) in the direction of the optical axis.

[0072] The focus adjustment unit (250) may include a moving guide installed inside the housing (210), a moving block movably coupled to the moving guide, and a driving unit that moves the moving block in the direction of the optical axis. The image sensor module (220) is coupled to the moving block and can move together with the moving block in the direction of the optical axis.

[0073] The drive unit may include a screw shaft, ball screw, rack and pinion, or linear actuator that converts rotational motion into linear motion. The drive unit may be operated manually by an operator or may operate automatically under the control of a control unit.

[0074] In one embodiment, the focus adjustment unit (250) can automatically adjust the focus using the clarity of the boundary line of the display panel (10) or a preset reference pattern before acquiring the first inspection image (IMG1). After the focus is adjusted, the movement position of the focus adjustment unit (250) can be fixed, and the first inspection image (IMG1) and the second inspection image (IMG2) can be acquired while maintaining the same position of the focus adjustment unit (250).

[0075] The focus adjustment unit (250) may further include a position sensor that detects the movement position of the image sensor module (220). The position sensor may be composed of an encoder, a linear scale, or a proximity sensor. The position information detected by the position sensor may be stored together with shooting condition information.

[0076] During re-inspection, it can be checked whether the image sensor module (220) is positioned at the same location as the focus position when acquiring the first inspection image (IMG1). If the position of the image sensor module (220) is outside the allowable range, the shooting unit (200) can stop the re-inspection shooting or readjust the focus.

[0077] The optical axis alignment unit (260) aligns the optical axis of the telecentric lens unit (230) with the imaging center of the image sensor module (220). The optical axis alignment unit (260) may include a lens support that supports the telecentric lens unit (230), a rotation axis that rotatably supports the lens support, and a control unit that adjusts the rotational position of the lens support.

[0078] The optical axis alignment unit (260) can move the telecentric lens unit (230) up and down, left and right, or in an inclined direction. Accordingly, if the optical axis of the telecentric lens unit (230) deviates from the imaging center of the image sensor module (220) due to assembly error, installation error, or external impact, it can be corrected.

[0079] The optical axis alignment unit (260) may include a locking unit that fixes the position of the telecentric lens unit (230) after the alignment of the telecentric lens unit (230) is completed. The locking unit may include a fixing bolt, a clamp, a wedge member, or an elastic pressure member.

[0080] It is possible to prevent the optical axis position of the telecentric lens unit (230) from changing while acquiring the first inspection image (IMG1) and the second inspection image (IMG2).

[0081] A vibration isolation member (270) is positioned between the housing (210) and the support bracket supporting the imaging unit (200). The vibration isolation member (270) reduces the transmission of vibrations generated from the conveyor, production equipment, or peripheral devices of the display panel (10) to the housing (210) and the image sensor module (220).

[0082] The vibration isolation member (270) may include a vibration damping pad formed of an elastic material, a coil spring, an air spring, or a viscoelastic damper. A plurality of vibration isolation members (270) may be spaced apart from each other in the lower part of the housing (210).

[0083] The vibration isolation unit (270) can absorb vibrations in a direction intersecting the shooting direction while supporting the weight of the shooting unit (200). Shaking and positional deviation of the first inspection image (IMG1) and the second inspection image (IMG2) can be reduced.

[0084] The lens cleaning unit (280) is positioned in front of the telecentric lens unit (230) and sprays cleaning fluid onto the surface of the protective window (290) or the front area of ​​the telecentric lens unit (230).

[0085] The lens cleaning unit (280) may include an annular body formed along the outer circumference of the protective window (290) and a plurality of spray holes formed in the annular body. The plurality of spray holes may be formed at an angle toward the center of the protective window (290) or arranged so that air flows along the surface of the protective window (290).

[0086] The lens cleaning unit (280) may be configured in an air purge manner that sprays compressed air or inert gas onto the surface of the protective window (290). The air sprayed from the lens cleaning unit (280) can remove dust, particles, or moisture attached to the surface of the protective window (290).

[0087] The lens cleaning unit (280) can form an air film in front of the protective window (290) to prevent foreign matter generated during the production process from adhering to the protective window (290). Accordingly, the likelihood of foreign matter attached to the protective window (290) being mistaken for a defect candidate area (DA) in the first inspection image (IMG1) and the second inspection image (IMG2) can be reduced.

[0088] A plurality of spray holes of the lens cleaning unit (280) may be spaced apart at regular intervals along the circumferential direction of the protective window (290). The plurality of spray holes may have the same spray angle from each other or different spray angles so as to form a uniform airflow over the entire surface of the protective window (290).

[0089] In one embodiment, the lens cleaning unit (280) may operate before capturing the first inspection image (IMG1) and before capturing the second inspection image (IMG2), respectively. In another embodiment, the lens cleaning unit (280) may operate continuously from after capturing the first inspection image (IMG1) until before capturing the second inspection image (IMG2).

[0090] By operating the lens cleaning unit (280), it is possible to prevent new foreign matter from adhering to the protective window (290) between the first inspection image (IMG1) and the second inspection image (IMG2). Therefore, it is possible to more accurately distinguish whether the change in brightness between the first inspection image (IMG1) and the second inspection image (IMG2) is due to a change in gradation of the display panel (10) or due to foreign matter attached to the optical system of the shooting unit (200).

[0091] The protective window (290) shields the open portion of the housing (210) in front of the telecentric lens portion (230). The protective window (290) is formed of a light-transmitting material and prevents the telecentric lens portion (230) from being directly exposed to external dust, moisture, impact, or contaminants.

[0092] The protective window (290) may be formed of tempered glass, quartz glass, or a transparent synthetic resin. One or more of an anti-reflective coating, a water-repellent coating, an anti-fouling coating, or an anti-static coating may be formed on the surface of the protective window (290).

[0093] The protective window (290) can be detachably coupled to the housing (210). The protective window (290) can be replaced without separating the telecentric lens portion (230) in the event of contamination or damage.

[0094] In one embodiment, the protective window (290) is positioned inside the lens cleaning unit (280), and the lens cleaning unit (280) may be formed to surround the outer circumference of the protective window (290). Accordingly, air sprayed from the lens cleaning unit (280) can be concentrated in the central area of ​​the protective window (290).

[0095] A control connector (295) is installed on the outside of the housing (210) and electrically connects the image sensor module (220), the focus adjustment unit (250), the lens cleaning unit (280), and the external control unit.

[0096] The control connector (295) can transmit the first inspection image (IMG1) and the second inspection image (IMG2) generated by the image sensor module (220) to the detection unit (300) or the re-inspection unit (500). Additionally, the control connector (295) can receive a shooting start signal, an exposure condition setting signal, a focus adjustment signal, and a lens cleaning signal from an external control unit.

[0097] In this embodiment, the shooting unit (200) may store the shooting conditions of the first inspection image (IMG1) as shooting condition information. The shooting condition information may include one or more of the exposure time of the image sensor module (220), sensor gain, frame rate, the position of the focus adjustment unit (250), the rotation angle of the polarization filter unit (240), and the operating state of the lens cleaning unit (280).

[0098] When the re-inspection pattern (P2) is displayed on the display panel (10), the shooting unit (200) can acquire a second inspection image (IMG2) with shooting conditions that are identical to the shooting conditions of the first inspection image (IMG1) or within a preset allowable range, based on the stored shooting condition information.

[0099] The shooting unit (200) can check the position of the focus adjustment unit (250), the rotation angle of the polarization filter unit (240), and the shooting setting value of the image sensor module (220) before shooting the second inspection image (IMG2). If the checked value is different from the shooting condition of the first inspection image (IMG1), the value can be restored to the shooting condition of the first inspection image (IMG1) and then the second inspection image (IMG2) can be shot.

[0100] The capturing unit (200) can calculate the positional deviation of an image using a reference mark, panel outline, or reference pixel of a display panel (10) included in the first inspection image (IMG1) and the second inspection image (IMG2). The calculated positional deviation can be corrected during the image preprocessing process.

[0101] The defect candidate regions (DA) of the first inspection image (IMG1) and the second inspection image (IMG2) can be placed at corresponding locations, and the second artificial intelligence model (520) can more accurately compare the brightness changes of the defect candidate regions (DA).

[0102] The operation of the imaging unit (200) according to the present embodiment is described as follows.

[0103] First, when the display panel (10) is placed at the inspection position, the optical axis alignment unit (260) aligns the optical axis of the telecentric lens unit (230) with the inspection area of ​​the display panel (10) and the imaging center of the image sensor module (220).

[0104] The focus adjustment unit (250) adjusts the focus on the display panel (10) by moving the image sensor module (220) or the telecentric lens unit (230) in the direction of the optical axis. The polarization filter unit (240) is adjusted to a polarization angle that minimizes reflected light generated from the surface of the display panel (10).

[0105] The lens cleaning unit (280) sprays air onto the surface of the protective window (290) to remove foreign matter attached to the protective window (290). In this state, the image sensor module (220) photographs the display panel (10) on which the inspection pattern (P1) is displayed to obtain a first inspection image (IMG1).

[0106] When the first inspection image (IMG1) is acquired, the shooting unit (200) stores the shooting condition information of the first inspection image (IMG1). When the first artificial intelligence model (310) detects a defect candidate region (DA) in the first inspection image (IMG1) and the judgment unit (400) decides to perform a re-inspection, a re-inspection pattern (P2) is displayed on the display panel (10).

[0107] The shooting unit (200) maintains or restores the exposure time, sensor gain, focus position, and polarization angle of the image sensor module (220) according to the stored shooting condition information. The lens cleaning unit (280) can clean the protective window (290) again before shooting the second inspection image (IMG2).

[0108] Afterwards, the image sensor module (220) captures the display panel (10) on which the re-inspection pattern (P2) is displayed to obtain a second inspection image (IMG2).

[0109] Since the first inspection image (IMG1) and the second inspection image (IMG2) are acquired under the same optical axis, same focus, same polarization conditions, and same shooting settings, the difference in brightness of the defect candidate region (DA) occurring between the two images can represent the response of the display panel (10) according to the grayscale change of the re-inspection pattern (P2).

[0110] Accordingly, the second artificial intelligence model (520) can more accurately determine whether a defect candidate region (DA) is a pixel defect of the display panel (10) or an appearance defect caused by foreign matter attached to the surface of the display panel (10) or the optical system of the imaging unit (200) based on the change in brightness of the first inspection image (IMG1) and the second inspection image (IMG2).

[0111] In particular, the telecentric lens unit (230), focus adjustment unit (250), optical axis alignment unit (260), and vibration isolation unit (270) maintain the same geometric shooting conditions for the first inspection image (IMG1) and the second inspection image (IMG2), the polarization filter unit (240) suppresses brightness fluctuations caused by reflected light, and the lens cleaning unit (280) and protective window (290) can suppress image changes caused by contamination of the shooting optical system.

[0112] It is possible to prevent differences in brightness caused by changes in shooting conditions or contamination of the optical system from being mistaken for changes in brightness caused by actual defects, and to improve the accuracy of determining whether a defect candidate region (DA) is defective and the type of defect.

[0113] As such, it is obvious to those skilled in the art that the present invention is not limited to the described embodiments and can be modified and varied in various ways without departing from the spirit and scope of the invention. Accordingly, such modified or varied embodiments should be deemed to fall within the scope of the claims of the present invention. Explanation of the symbols

[0114] 100: Pattern supply unit 200: Filming Department 210: Housing 220: Image sensor module 230: Telecentric lens unit 240: Polarizing filter section 250: Focus adjustment section 260: Optical axis alignment section 270: Vibration isolation section 280: Lens cleaning unit 290: Protected Window 295: Control Connector 300: Detector 310: The first artificial intelligence model 400: Judgment Division 500: Re-examination Department 510: Re-examination pattern generation unit 520: The Second Artificial Intelligence Model P1: Test pattern P2: Retest pattern IMG1: First examination video IMG2: Second examination video DA: Candidate defect area 10: Display panel

Claims

Claim 1 A method for inspecting an artificial intelligence-based display, comprising: a pattern display step for displaying an inspection pattern on a display panel; a first shooting step for obtaining a first inspection image by shooting the display panel on which the inspection pattern is displayed; a defect candidate detection step for calculating a defect candidate region and a reliability for the defect candidate region from the first inspection image using a learned first artificial intelligence model; a determination step for determining that if the reliability is greater than or equal to a preset judgment threshold value, the defect candidate region is confirmed as a defect, and if the reliability is less than the judgment threshold value and greater than or equal to a preset re-inspection threshold value, a re-inspection step is determined; and a re-inspection step for generating a re-inspection pattern in which the grayscale of a region corresponding to the defect candidate region is changed from the inspection pattern based on location information of the defect candidate region, obtaining a second inspection image by shooting the display panel on which the re-inspection pattern is displayed, and determining whether the defect candidate region is a defect and the type of defect based on the change in brightness of the defect candidate region between the first inspection image and the second inspection image using a learned second artificial intelligence model. Claim 2 An artificial intelligence-based display inspection method according to claim 1, wherein the re-inspection step generates the re-inspection pattern by inverting the grayscale of the area corresponding to the defect candidate area from the grayscale of the inspection pattern and maintaining the grayscale of the remaining area identical to the inspection pattern, and using the second artificial intelligence model, if the brightness of the defect candidate area changes according to the inversion of the grayscale, the defect candidate area is determined to be a pixel defect of the display panel, and if the brightness of the defect candidate area is maintained despite the inversion of the grayscale, the defect candidate area is determined to be an appearance defect caused by foreign matter. Claim 3 In claim 1, the first inspection image and the second inspection image are acquired by the same shooting unit, and the shooting unit comprises a housing, an image sensor module disposed inside the housing, a telecentric lens unit disposed on the light path between the image sensor module and the display panel to cause the main light ray to be incident parallel to the optical axis at the center and periphery of the screen of the display panel, a polarizing filter unit disposed in front of the telecentric lens unit to block the specular reflection component of external light reflected from the surface of the display panel, a protective window coupled to the front end of the housing to seal the light path from the outside, a lens cleaning unit that sprays air onto the outer surface of the protective window, and a vibration isolation unit that elastically supports the housing with respect to a support bracket, wherein in the defect candidate detection step, the location information of the defect candidate area is calculated as pixel coordinates of the display panel from the first inspection image in which perspective distortion is excluded by the telecentric lens unit, so that the area where the gradation of the re-inspection pattern has changed is aligned with the defect candidate area, and the lens cleaning unit is after the first shooting step and before the acquisition of the second inspection image An artificial intelligence-based display inspection method characterized by removing foreign matter attached to the protective window by spraying air onto the outer surface of the protective window, and the re-inspection step using the second inspection image, which maintains pixel-level alignment with the first inspection image by the vibration isolation unit, to identify the defect candidate area determined to be an external defect caused by foreign matter as a defect caused by foreign matter on the display panel side.

Images