A defect detection method based on digital image processing and automatic optical inspection

Abstract

The application discloses a kind of defect detection methods based on digital image processing and automatic optical detection, comprising the following steps: step S1: correction calculation is carried out to one of image in fixed region of interest, and the corrected transformation relationship is obtained;Step S2: dust image is carried out dust calculation, and dust distribution result is obtained;Step S3: according to the defect shown by each image, corresponding image processing operator and preset parameter are used to detect, and defect detection result is obtained;Step S4: the defect result and pre-set label shielding area are compared with dust distribution area, and the final detection result is obtained, the application proposes a set of screen detection method and process, as far as possible to increase the screen defect type that can be detected and exclude dust, label interference.

Description

Technical fields:

[0001] This invention relates to the fields of image processing, machine vision, and computer vision. Background technology:

[0002] Most existing screen defect detection inventions are only applicable to specific devices, and the types of defects that can be detected are quite limited. Different defects on a product screen may require different lighting conditions to appear on the image, and dust interference is also likely to be present.

[0003] For example, application number CN201911057633.5, entitled "A Defect Detection Method Based on Image Gray-Scale Features," discloses the following steps: 1) Performing the following steps on the measured feature region image to obtain: ① Converting the image to grayscale to obtain a grayscale image; ② Establishing a selection area in the grayscale image, generating feature values ​​for each pixel within the current selection area; concatenating the feature values ​​within the selection area by row / column to form the feature vector of the current pre-selected pixel; ③ Marking new pre-selected pixels, repeating step ② until the feature vector of the last pre-selected pixel is obtained; ④ Sort all feature vectors, discarding some feature vectors; weighting the retained feature vectors; concatenating all weighted feature vectors to form a global feature vector.

[0004] 2) Calculate cosine similarity; 3) Mark the feature image type corresponding to the larger similarity value as the shape of the object being tested; it can meet the low requirements for imaging quality and is suitable for quality inspection of products in industrial sites; however, the inspection process is too cumbersome. Summary of the Invention:

[0005] To address the shortcomings of existing technologies, the purpose of this invention is to provide a defect detection method based on digital image processing and automated optical inspection. Currently, mass-produced screen products (including mobile phone screens, automotive screens, etc.) may have various defects during the production process. These different types of defects only become visible under different lighting conditions. Furthermore, the products themselves may have labels, and dust inevitably accumulates on the screens during production, interfering with defect detection. This invention proposes a screen inspection method and process to maximize the types of detectable screen defects and eliminate interference from dust and labels.

[0006] To achieve the above objectives, the present invention provides the following technical solution:

[0007] A defect detection method based on digital image processing and automated optical inspection includes the following steps:

[0008] Step S1: Perform correction calculations on one of the images within a fixed region of interest to obtain the correction transformation relationship;

[0009] Step S2: Perform dust calculations on the dust image to obtain the dust distribution results;

[0010] Step S3: Based on the defects displayed in each image, use the corresponding image processing operators and preset parameters to perform detection and obtain the defect detection results;

[0011] Step S4: Compare the defect results with the pre-set label shielding area and the dust distribution area to obtain the final detection result.

[0012] As a further aspect of the present invention, step S1 further includes: correcting the product positions in all other images in the same way using an affine transformation according to the transformation relationship.

[0013] As a further aspect of the present invention, the correction method is to segment the image by using a grayscale threshold to obtain a binary image, thereby obtaining the connected components of the relatively bright part of the image.

[0014] As a further aspect of the present invention, the area of ​​the obtained connected components is filtered to obtain the connected components of the product, and a minimum bounding rectangle operation is performed to obtain the minimum bounding rectangle of the connected components of the product.

[0015] As a further aspect of the present invention, the translation transformation relationship of moving the product to the target position is obtained based on the distances from the four vertices of the circumscribed rectangle to the upper left corner of the image; based on the obtained coordinate transformation relationship, an affine transformation is performed to move the product position to the target position.

[0016] As a further embodiment of the present invention, step S4 further includes: defects whose center point coordinates are in the label shielding area and the dust area are regarded as interference items and removed from the detection results to obtain the final detection result.

[0017] As a further aspect of the present invention, the image is preprocessed by performing binarization segmentation using a grayscale threshold; the segmentation result is then subjected to morphological processing to obtain the final segmentation result.

[0018] This invention can detect a wide variety of defects. Because a single process uses different lights, some defects that are not easily visible on a regular image or a particular image may appear on another image, allowing as many defects as possible to be detected by a single process, thus improving the efficiency of inspection and production.

[0019] To more clearly illustrate the structural features and effects of the present invention, the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. Attached image description:

[0020] Figure 1 This is a flowchart diagram of the present invention.

[0021] Figure 2 This is an example diagram illustrating the correction effect in this invention;

[0022] Figure 3 This is the binary image after threshold segmentation in this invention;

[0023] Figure 4 This is the correction of the original image in this invention;

[0024] Figure 5 This is the corrected image of the original image in this invention.

[0025] Figure 6 This is the original diagram used for dust calculation in this invention.

[0026] Figure 7 It is a corrected binary image of the original image used for dust calculation in this invention, and the image used to calculate the dust distribution.

[0027] Figures 8-15 In this invention, taking the blocky internal stains caused by ink as an example, the ink marks on the display screen have different display effects when illuminated by different lights. Detailed implementation method:

[0028] The present invention will now be further described in conjunction with the accompanying drawings and relevant knowledge, and will be described clearly and completely. Obviously, the described applications are only some embodiments of the present invention, and not all embodiments.

[0029] This invention presents a defect detection method based on digital image processing and automated optical inspection, specifically for screen products that can be illuminated by light. Currently mass-produced screen products (including mobile phone screens, automotive screens, etc.) may have various defects during the production process. These different types of defects only become visible under different lighting conditions. Furthermore, the products may have labels, and dust inevitably accumulates on the screens during production, interfering with defect detection. This invention provides a screen inspection method and process that maximizes the types of detectable screen defects and eliminates interference from dust and labels. Figure 1 As shown, each product undergoes a set of inspection processes: switching different colored lights to illuminate the product, generating images such as white, red, black, and yellow images, as well as a cleaned image to show dust, which are then fed into the algorithm to perform image processing.

[0030] See Figures 1-7 As shown, a defect detection method based on digital image processing and automated optical inspection includes the following steps:

[0031] Includes the following steps:

[0032] Step S1: Perform correction calculations on one of the images within a fixed region of interest to obtain the correction transformation relationship;

[0033] Step S2: Perform dust calculations on the dust image to obtain the dust distribution results;

[0034] Step S3: Based on the defects displayed in each image, use the corresponding image processing operators and preset parameters to perform detection and obtain the defect detection results;

[0035] Step S4: Compare the defect results with the pre-set label shielding area and the dust distribution area to obtain the final detection result.

[0036] In step S1, according to the transformation relationship, the product positions in all other images are corrected in the same way using an affine transformation. Specifically, a correction calculation is performed on one image, and the product's position in the image is uniformly moved to a fixed Region of Interest (ROI) to obtain the corrected transformation relationship. Then, according to the transformation relationship, the product positions in all other images are corrected in the same way using an affine transformation.

[0037] Further optimization, referring to Figure 2 , Figure 3 As shown, correction typically uses an image that is significantly different from the background, such as a white image. The general process is as follows:

[0038] Image segmentation using a grayscale threshold yields a binary image (note: containing only black and white; areas above the threshold are considered foreground, also the product portion, and displayed in white; areas below the threshold are considered background and displayed in black), thus obtaining the connected components of the relatively brighter parts of the image (note: connected white areas in the image). Area filtering is then applied to these connected components to obtain the connected components of the product.

[0039] Perform a minimum bounding rectangle operation on the connected components obtained above to obtain the minimum bounding rectangle of the product's connected components. Based on the distances from the four vertices of the bounding rectangle to the top-left corner of the image, obtain the translation transformation relationship for moving the product to the target position (generally the top-left corner of the image). Based on the coordinate transformation relationship obtained in the previous step, perform an affine transformation to move the product to the target position (generally the top-left corner of the image).

[0040] In addition, the calculation of dust distribution uses specially captured images, and the main steps are as follows:

[0041] The image is preprocessed (median filtering or Gaussian filtering, etc.), and then binarized using a grayscale threshold. Morphological processing (erosion, dilation, opening, closing operations, etc.) is then applied to the segmentation results to obtain the final segmentation outcome.

[0042] In this invention, step S3 specifically includes: based on the defects displayed in each image, using corresponding image processing operators and preset parameters to perform detection, and obtaining defect detection results (including the center point coordinates of the defect on the product).

[0043] In this invention, step S4 specifically includes: comparing the defect results with the pre-set label shielding area and the dust area obtained in step 2. Defects whose center point coordinates are in the label shielding area and the dust area are regarded as interference items and removed from the detection results to obtain the final detection results.

[0044] In this invention, reference is made to Figure 6 and Figure 7 As shown, the dust distribution is calculated using a dust removal diagram. The original dust removal diagram is shown below. Figure 6 , Figure 7 This is the corrected dust distribution result image. This image is also a binary image, with the white part (dust particles and the part outside the region of interest ROI) considered as the dust area.

[0045] The following provides a specific embodiment of the present invention.

[0046] Example 1

[0047] Reference Figures 1-7 As shown, the present invention provides a defect detection method based on digital image processing and automatic optical inspection, which specifically includes: setting the illuminated image according to the product to be inspected; selecting the corresponding operator and setting parameters;

[0048] The specific testing steps are as follows:

[0049] Select an original image (e.g.) Figure 4 Perform correction calculations (usually using a white image), move the product to the top left corner of the image, and obtain the corrected image (e.g., ...). Figure 5 (Results)

[0050] Other diagrams adjust the product position according to the calibration relationship.

[0051] The operator detection for each image yields the defect detection results (the dust removal image is used to calculate the dust distribution).

[0052] If no defects are detected in any of the images, the product is considered good and is reported as OK; otherwise, it is reported as NG and a rectangle is marked on the corresponding image based on the coordinates of the defect's center point.

[0053] Advantages of this invention: Current mass-produced screen products (including mobile phone screens, automotive screens, etc.) may have various defects during the production process. These different types of defects only become visible under different lighting conditions. Furthermore, the products themselves may have labels, and dust inevitably accumulates on the screens during production, interfering with defect detection. This invention maximizes the number of detectable screen defect types and eliminates interference from dust and labels. For illuminated screen products (including but not limited to mobile phone screens and automotive screens), this invention can detect a wider range of defect types. Because a single process uses different lighting conditions, defects that are difficult to see on a regular image or a particular image may appear on another image, allowing as many defects as possible to be detected by a single process, thus improving detection and production efficiency.

[0054] Reference Figures 8-15 The images shown are white, black, red, blue, green, light gray, dark gray, and top-clear gray, illustrating different lighting conditions. Taking ink-induced blocky internal stains as an example, the ink marks on the display screen appear differently under different lighting conditions. Different colored images represent the display effects under different colored lights. The top-clear gray image is used to calculate dust distribution, while the other images are used to display defects. It can be seen that this type of defect is most clearly displayed in the blueprint; therefore, adding operators to the blueprint is the best approach for detecting this type of defect.

[0055] The technical principles of the present invention have been described above with reference to specific embodiments, which are merely preferred embodiments of the present invention. The scope of protection of the present invention is not limited to the above embodiments; all technical solutions falling within the scope of the present invention's concept are within its protection scope. Those skilled in the art can conceive of other specific embodiments of the present invention without creative effort, and these embodiments will all fall within the protection scope of the present invention.

Claims

1. A defect detection method based on digital image processing and automated optical inspection, characterized in that, Includes the following steps: Step S1: Perform correction calculations on one of the images within a fixed region of interest to obtain the correction transformation relationship; The correction calculation includes: segmenting the image using a grayscale threshold to obtain a binary image, thereby extracting the connected components of the relatively high-brightness parts of the image; performing area filtering on the connected components, retaining the connected components corresponding to the product, and performing a minimum bounding rectangle operation to obtain the minimum bounding rectangle of the product's connected components; calculating the translation transformation relationship of the product moving to the target position based on the distances from the four vertices of the minimum bounding rectangle to the upper left corner of the image; correcting the product positions in all other images in the same way according to the translation transformation relationship, so that the product positions in all images are uniformly located in a fixed region of interest; and preprocessing the image, including performing binarization segmentation using a grayscale threshold, and performing morphological processing on the segmentation results to obtain the final segmentation result. Step S2: Dust calculation is performed using specially captured dust removal images. Specifically, this includes: preprocessing the dust removal images with median filtering or Gaussian filtering, and obtaining the dust distribution results through grayscale threshold binarization segmentation and morphological processing. Step S3: For screen products that can be lit up, the product is lit up by switching different colored lights to generate multiple sets of images; according to the defect type shown in each set of images, the corresponding image processing operator and preset parameters are used for detection to obtain defect detection results containing the coordinates of the defect center point. Step S4: Compare the defect detection results with the pre-set label shielding area and the dust distribution area obtained in step S2; if the center point coordinates of the defect are located in the label shielding area or the dust area, the defect is regarded as an interference item and removed from the detection results, and finally the true defect detection results of the screen product are obtained; through the above steps, the detectable screen defect types are increased, the interference of dust and labels is eliminated, and a set of processes is used to detect multiple types of defects.

Images