Adhesion Analysis Method and System for Cross-Cut Test

Abstract

An adhesion analysis method and system for cross-cut test. In the present disclosure, an object recognition model is used to find the peeling area in the image to be tested, and a semantic segmentation model is used to identify the scratch area in the image to be tested to obtain a processed image. Then, a cropped area corresponding to the peeling area is taken out from the processed image, the peeling area and the cropped area are combined to obtain a combined image, and the peeling ratio is calculated based on the combined image.

Description

Technical Field

[0001] The present invention relates to an analysis method and system, and particularly to an adhesion analysis method and system for cross-cut test. Background Art

[0002] Cross-cut test is a way to evaluate the adhesion of coating. It is widely used in the industry because of its easy operation and observation. The test method is to use a cross-cut knife to scrape scratches on the object to be tested. Then, after closely attaching a tape to the cross-cut position, the tape is torn off with an instantaneous force, and the adhesion of the coating is judged by the degree of peeling of the coating on the object to be tested.

[0003] Traditional adhesion analysis is that testers use a microscope to inspect the grid-like area on the sample surface to evaluate the adhesion of the coated surface. For example, two photos are taken before and after the tape is pasted through a camera, and then the difference between the two photos is obtained by subtracting them to find the difference between the two photos, which is the peeling area. However, if the shooting light sources before and after the tape is pasted are different or the magnification of the microscope is different, misjudgment will occur when subtracting the two photos subsequently. In addition, the circuit pattern of the sample will also cause interference in the judgment. In addition, the peeling condition can also be judged by manual visual inspection. However, the subjective judgment of the cross-cut peeling condition by humans lacks an objective unified standard, and if the peeling area is too small or not obvious, it will also cause difficulty in judgment for users. And when looking for the most serious peeling area, the glass slide needs to be moved back and forth many times, increasing the time cost. In addition, the photographing device needs to borrow a single-lens camera additionally, and the borrowing and returning will increase unnecessary time costs. And the test results and the photographed images are stored in the personal computer of the tester, which also leads to messy data and difficulty in sorting. Summary of the Invention

[0004] The present invention provides an adhesion analysis method and system for cross-cut test, which can greatly save time costs and objectively perform efficient judgment.

[0005] The adhesion analysis method for cross-cut test of the present invention is executed by an electronic device. The adhesion analysis method includes: using an object recognition model to find the peeling area in the image to be tested; using a semantic segmentation model to identify the scratch area in the image to be tested to obtain a processed image; taking out a cropped area corresponding to the peeling area from the processed image; combining the peeling area and the cropped area to obtain a combined image; and calculating the peeling ratio based on the combined image.

[0006] In an embodiment of the present invention, the above adhesion analysis method further includes: obtaining a plurality of frames from the image signal; using a Laplacian operator to delete the out-of-focus frames from the frames; and deleting the frames with duplicate pixels among the determined in-focus frames, and taking each remaining frame as the image to be tested.

[0007] In an embodiment of the present invention, the above adhesion analysis method further includes: performing a defuzzification process on the image to be measured, and inputting the image to be measured after the defuzzification process into an object recognition model and a semantic segmentation model.

[0008] In an embodiment of the present invention, the step of obtaining a combined image by combining the peeling area and the cutting area includes: performing edge detection on the peeling area by using an edge detection model to obtain an edge image; and combining the edge image and the cutting area to obtain a combined image.

[0009] In an embodiment of the present invention, the step of calculating the peeling ratio based on the combined image includes: performing a black and white inversion process on the combined image to obtain an inverted image; performing an edge enhancement process on the inverted image to obtain an enhanced image; using an edge detection algorithm to find the contour of the peeling block in the enhanced image; calculating the peeling area based on the contour of the peeling block; and calculating the peeling ratio based on the peeling area and the total area of the image to be measured.

[0010] The adhesion analysis system of the present invention includes: a microscope having an image acquisition device for capturing an image signal of a test object placed on the microscope through the image acquisition device; and an electronic device for receiving the image signal from the image acquisition device. The electronic device is configured to obtain a plurality of images to be measured from the image signal and perform the following operations respectively on each image to be measured: using an object recognition model to find the peeling area in the image to be measured; using a semantic segmentation model to identify the scratch area in the image to be measured to obtain a processed image; taking out a cutting area corresponding to the peeling area from the processed image; combining the peeling area and the cutting area to obtain a combined image; and calculating the peeling ratio based on the combined image.

[0011] Based on the above, the present disclosure provides an adhesion analysis method and system for cross cut test, which replaces manual work with automation and establishes an objective judgment standard. Accordingly, the time cost is greatly saved, and the efficient judgment is objectively carried out. Description of the Drawings

[0012] Figure 1 is a block diagram of an adhesion analysis system according to an embodiment of the present invention.

[0013] Figure 2 is a schematic diagram of a user interface according to an embodiment of the present invention.

[0014] Figure 3 is an architecture diagram of adhesion analysis according to an embodiment of the present invention.

[0015] Figure 4 is a schematic diagram of an image to be measured according to an embodiment of the present invention.

[0016] Figure 5It is a flowchart of an adhesion analysis method for cross-cut test according to an embodiment of the present invention.

[0017] Figure 6 It is a schematic diagram of the results obtained by a semantic segmentation model using different algorithms according to an embodiment of the present invention.

[0018] Figure 7 It is a schematic diagram of the image processing flow before obtaining the combined image according to an embodiment of the present invention.

[0019] Figure 8 It is a schematic diagram of calculating the peeling area according to an embodiment of the present invention.

[0020] Figure 9 It is a schematic diagram of analysis according to an embodiment of the present invention.

[0021] Figure 10 It is a schematic diagram of analysis according to another embodiment of the present invention.

[0022] Figure 11 It is a schematic diagram of accuracy comparison according to an embodiment of the present invention.

[0023] Description of reference numerals:

[0024] 110: Electronic device

[0025] 111: Processor

[0026] 113: Storage element

[0027] 120: Microscope

[0028] 121: Image acquisition device

[0029] 200: User interface

[0030] 201~209: Working frame

[0031] 400, 600, B, T: Images to be measured

[0032] 410: Peeling block

[0033] 420: Scratch area

[0034] 610, 620, 920, 1020, T2: Processed images

[0035] 901: Object frame

[0036] 910, 1010, T1: Recognized images

[0037] 930, 1030: Judgment result images

[0038] A: Image signal

[0039] B1 to Bn: Candidate images

[0040] b1: Peeling area

[0041] b2: Cropping area

[0042] b3: Edge image

[0043] b4: Composite image

[0044] b5: Inverted image

[0045] b6: Enhanced image

[0046] b7: Contour image

[0047] b8: Final image

[0048] M1: Object recognition model

[0049] M2: Semantic segmentation model

[0050] S305 to S320: Steps of adhesion analysis

[0051] S505 to S525: Steps of adhesion analysis Detailed implementation manners

[0052] Figure 1 is a block diagram of an adhesion analysis system according to an embodiment of the present invention. Please refer to Figure 1 , the adhesion analysis system includes an electronic device 110 and a microscope 120. The electronic device 110 includes a processor 111 and a storage element 113. An image acquisition device 121 is installed in the microscope 120.

[0053] The processor 111 is, for example, a central processing unit (CPU), a physics processing unit (PPU), a programmable microprocessor, an embedded control chip, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), or other similar devices.

[0054] The storage element 113 is, for example, any form of fixed or removable random access memory (RAM), read-only memory (ROM), flash memory, hard disk, or other similar devices, or a combination of these devices. The storage element 113 includes one or more code segments, which, after being installed, are executed by the processor 111. In one embodiment, the storage element 113 also stores a user interface for displaying the user interface on a display (not shown) for the user to use.

[0055] The image acquisition device 121 is, for example, a camera, a camera, etc. that uses a charge coupled device (CCD) lens or a complementary metal oxide semiconductor transistor (CMOS) lens. The image acquisition device 121 is used to capture a test object (e.g., a glass subjected to a hundred-grid test) disposed on the stage of the microscope 120.

[0056] Figure 2 It is a schematic diagram of a user interface according to an embodiment of the present invention. Please refer to Figure 2 , the user interface 200 includes a plurality of work frames 201-209. The work frame 201 is used for browsing, selecting, and storing images. For example, a plurality of images are displayed in the left column of the work frame 201 for the user to browse, and the user can select among these images by clicking the left mouse button. The file name of the selected test image can be displayed in the right column of the work frame 201, and the user can store the selected image by double-clicking the left mouse button. The work frame 203 is used for the user to input data of the test object (e.g., a sampling glass) for archiving. The work frame 205 can view the video screen of the image acquisition device 121 in real time, and also serves as presenting the selected test image and reflecting the analysis result in the test image. The work frame 207 provides de-molding and visualization tools and displays the analyzed image information and determination result. The work frame 209 includes five operation buttons for controlling video recording, stopping, snapshot, storing, and browsing the database.

[0057] Figure 3 It is an architecture diagram of adhesion analysis according to an embodiment of the present invention. Please refer to Figure 3 , first, the image signal A is input to the electronic device 110, and the processor 111 obtains a plurality of frames from the image signal A. For example, with the image acquisition device 121 stationary, the test object is moved to ensure that the image acquisition device 121 can capture every part of the test object, thereby obtaining the image signal A.

[0058] Next, in step S305, the processor 111 deletes out-of-focus frames. For example, the variance of each frame is calculated using the Laplacian operator, and the variance is compared with a set threshold (e.g., 12), and frames with a variance less than 12 are deleted. The set threshold can be determined based on the test results of all the pictures in the project.

[0059] After that, in step S310, the processor 111 deletes frames with duplicate pixels among the determined non-out-of-focus frames, thereby obtaining a plurality of candidate images B1 to Bn. There are no duplicate regions in each of the candidate images B1 to Bn. Since in the process of obtaining a plurality of frames from the image signal A, there may be duplicate regions in two adjacent frames in the time series, therefore, in step S310, it is possible to determine whether duplicate pixels are included in two frames through pixel comparison operations. For example, two adjacent frames f1 and f2 can be taken out according to the time series. If these two frames f1 and f2 include the same pixels, the frame f2 with a later time series is deleted, and then the next frame f3 is taken down according to the time series to determine whether the frame f3 has the same pixels as the frame f1. If the frame f3 and the frame f1 do not include the same pixels, the frame f1 (candidate picture) is retained, and the frame f3 is continuously compared with the frame f4, and so on to check each frame.

[0060] After obtaining the candidate images B1 to Bn, each candidate image is taken as the test image B one by one, and the test image B is input into the object recognition model M1 and the semantic segmentation model M2 respectively to find the peeling area and the scratch area in the test image B. In addition, after taking out the test image B, the test image B can be further subjected to a deblurring process, and the deblurred test image B is input into the object recognition model M1 and the semantic segmentation model M2. The deblurring process can adopt algorithms such as DeblurGAN-v2 or MPRNet.

[0061] Figure 4 is a schematic diagram of a test image according to an embodiment of the present invention. Please refer to Figure 4 , through the object recognition model M1, the peeling block 410 can be found in the test image B, and through the semantic segmentation model M2, the scratched area 420 where the blade has passed can be separated in the test image B to optimize the area calculation of the peeling area.

[0062] Return Figure 3, after finding the peeling area and scratch area in the image B to be measured, adhesion analysis is further performed in step S315 to calculate the peeling ratio of the peeling area. And, in step S320, the analysis result is obtained. For example, the grade of the image to be measured is judged according to the peeling ratio. According to the ASTM (American Society for Testing and Materials) standard, the grades can be divided into 1B - 5B. Grade 5B means no peeling at all, grade 4B means the peeling ratio is less than or equal to 5%, grade 3B means the peeling ratio is greater than 5% and not greater than 15%, grade 2B means the peeling ratio is greater than 15% and not greater than 35%, grade 1B means the peeling ratio is greater than 35% and not greater than 65%, and grade 0B means the peeling ratio is greater than 65%.

[0063] The following is another embodiment to illustrate the detailed process of adhesion analysis. Figure 5 is a flowchart of an adhesion analysis method for cross-cut test according to an embodiment of the present invention. Please refer to Figure 3 and Figure 5 , in step S505, the processor 111 uses the object recognition model M1 to find the peeling area in the image to be measured. The object recognition model M1 can adopt one of the following algorithms: R-CNN (Region Based Convolutional Neural Networks), YOLO (You Only Look Once), SSD (Single Shot MultiBox Detector), EfficientDet, CenterNet, Swin transformer.

[0064] After multiple predicted bounding boxes that may be the peeling area are detected in the image to be measured by the object recognition model M1, the IOU (Intersection over Union) value is calculated for each predicted bounding box. The IOU value is the ratio between the intersection and union of the predicted bounding box and the reference bounding box. If the IOU value is greater than a threshold (for example, 0.5), then this predicted bounding box is determined to be the peeling area; otherwise, this predicted bounding box is ignored.

[0065] On the other hand, in step S510, the processor 111 uses the semantic segmentation model M2 to identify the scratch area in the image to be measured, thereby obtaining a processed image. The semantic segmentation model M2 can adopt one of the following algorithms: FCN (Fully Convolutional Networks), U-Net, DeepLab, ReNet, ReSeg, GAN (Generative Adversarial Networks), HRNet (High-Resolution Net), OCRNet (object-context representation Net).

[0066] Figure 6 FIG. is a schematic diagram of the results obtained by using semantic segmentation models with different algorithms according to an embodiment of the present invention. In this embodiment, the results of the semantic segmentation model M2 using a general CNN (such as U-Net) and the semantic segmentation model M2 using GAN (such as pix2pix) are compared.

[0067] The processed image 610 is obtained by using the semantic segmentation model M2 of U-Net to analyze the image 600 to be measured. From the processed image 610, it can be seen that the effect of the semantic segmentation model M2 using U-Net in the crosshatch test is not ideal. It is speculated that the reason may be that there are no obvious features in the scratch area except at the edges, and U-Net may recognize it as a single-color area, resulting in serious prediction inaccuracy.

[0068] The processed image 620 is obtained by using the semantic segmentation model M2 of GAN to analyze the image 600 to be measured. From the processed image 620, it can be seen that the effect of the semantic segmentation model M2 using GAN is better than that of the semantic segmentation model M2 using U-Net. It is speculated that the reason is that GAN has a discriminator, which can judge whether the generated processed image 620 is similar in appearance to the image 600 to be measured.

[0069] Return Figure 5 , in step S515, the processor 111 extracts the cropped area corresponding to the peeling area from the processed image. In step S520, the processor 111 combines the peeling area and the cropped area to obtain a combined image. And, in step S525, the processor 111 calculates the peeling ratio based on the combined image.

[0070] The following further illustrates steps S515 to S525 with examples. Figure 7 FIG. is a schematic diagram of the image processing flow before obtaining the combined image according to an embodiment of the present invention. Figure 8 FIG. is a schematic diagram of calculating the peeling area according to an embodiment of the present invention.

[0071] Please refer to 7. The image T to be measured is obtained as the recognized image T1 through the object recognition model M1. In the recognized image T1, the peeling area is marked using object frames. Then, cropping is performed at the object frames in the recognized image T1 to obtain the peeling area b1 enclosed by the object frames. Moreover, the image T to be measured is processed through the semantic segmentation model M2 to obtain the processed image T2. Next, based on the coordinate range enclosed by the object frames in the recognized image T1, the cropping area b2 corresponding to the peeling area b1 is extracted from the processed image T2.

[0072] To make the subsequent results more accurate, the edge detection model M3 can be further used to perform edge detection on the peeling area b1 to obtain the edge image b3. The edge detection model M3, for example, adopts the Canny edge detection algorithm. Then, the combined image b4 is obtained by combining the edge image b3 and the cropping area b2 (as Figure 8 shown), and the peeling area is calculated in the combined image b4, and then the peeling ratio is obtained. To accurately calculate the peeling area, the following processing can also be performed on the combined image b4.

[0073] Refer to Figure 8 , after combining the edge image b3 and the cropping area b2 to obtain the combined image b4, the black-and-white inversion processing is performed on the combined image b4 to obtain the inverted image b5. Then, the edge enhancement processing is performed on the inverted image b5 to obtain the enhanced image b6. Then, the peeling block contour is found in the enhanced image b6 using the edge detection algorithm to obtain the contour image b7. After that, the final image b8 is obtained based on the peeling block contour in the contour image b7, and the pixel number of the black area in the final image b8 is calculated as the peeling area. Then, based on the peeling area and the total area of the image to be measured, the peeling ratio (peeling area / total area) is calculated. For example, assuming that the total area of the image to be measured is 800×600 and the pixel number of the peeling area is 6704, the peeling ratio is 6704 / (800×600). Based on this, the processor 111 can determine the grade according to the peeling ratio.

[0074] Two more embodiments are given below to illustrate the analysis results obtained by applying the above adhesion analysis system. Figure 9 It is an analysis schematic diagram according to an embodiment of the present invention. Figure 10 It is an analysis schematic diagram according to another embodiment of the present invention.

[0075] In Figure 9The illustrated embodiment includes the recognized image 910 (including the object frame 901 that outlines the peeling area) obtained via the object recognition model M1 and the processed image 920 obtained via the semantic segmentation model M2. Based on the recognized image 910 and the processed image 920, the final determination result image 930 is obtained, with a peeling ratio of 1.026% and determined to be grade 4B.

[0076] In Figure 10 The illustrated embodiment includes the recognized image 1010 (no peeling area detected) obtained via the object recognition model M1 and the processed image 1020 obtained via the semantic segmentation model M2. Based on the recognized image 1010 and the processed image 1020, the final determination result image 1030 is obtained. The determination result image 1030 does not include any peeling and is determined to be grade 5B.

[0077] Figure 11 It is a schematic diagram of the accuracy comparison according to an embodiment of the present invention. In Figure 11 it, the horizontal axis represents the predicted grade, the vertical axis represents the actual grade, and the numbers in the figure are the judgment numbers of the predicted grade and the actual grade. In this embodiment, for the predicted grade of 1B, there are 37 test images with a predicted grade of 1B, and the judgment number of the actual grade of 1B is 34, and the numbers of the actual grades of 2B - 5B are all 0. Figure 11 In it, the total number of the predicted grades that match the actual grades is 198 (34 + 30 + 35 + 32 + 67), and the total number of all predictions is 216. Thus, it can be seen that the accuracy of the intelligent analysis disclosed in the embodiment has reached 91.7% (198 / 216).

[0078] In summary, the present disclosure provides an adhesion analysis method and system for the crosshatch test, which replaces manual labor with automation and establishes an objective judgment standard. Accordingly, the time cost is greatly saved, and the determination is objectively and efficiently carried out. Moreover, data sorting and visualization tools are provided, and an intuitive and smooth operation experience is provided to improve the user experience.

Claims

1. An adhesion analysis method for cross-cut test, which is executed by an electronic device. The adhesion analysis method includes: Using an object recognition model to find a peeling area in a to-be-tested image; Using a semantic segmentation model to identify a scratch area in the to-be-tested image to obtain a processed image; Taking out a cropped area corresponding to the peeling area from the processed image; Combining the peeling area and the cropped area to obtain a combined image; And Calculating a peeling ratio based on the combined image, where the step of calculating the peeling ratio based on the combined image includes: Performing a black-and-white inversion process on the combined image to obtain an inverted image; Performing an edge enhancement process on the inverted image to obtain an enhanced image; Using an edge detection algorithm to find a peeling block contour in the enhanced image; Calculating a peeling area based on the peeling block contour; and Calculating the peeling ratio based on the peeling area and the total area of the to-be-tested image.

2. The adhesion analysis method according to claim 1, further including: Obtaining a plurality of frames from an image signal; Using a Laplacian operator to delete out-of-focus frames from the plurality of frames; And Deleting frames with duplicate pixels among the determined non-out-of-focus frames, and taking each remaining frame as the to-be-tested image.

3. The adhesion analysis method according to claim 1, further including: Performing a deblurring process on the to-be-tested image so as to input the to-be-tested image after the deblurring process into the object recognition model and the semantic segmentation model.

4. The adhesion analysis method according to claim 1, wherein the step of combining the peeling area and the cropped area to obtain the combined image includes: Performing edge detection on the peeling area using an edge detection model to obtain an edge image; And Combining the edge image and the cropped area to obtain the combined image.

5. An adhesion analysis system, including: A microscope having an image acquisition device, and obtaining an image signal by photographing a to-be-tested object placed on the microscope through the image acquisition device; And An electronic device, receiving the image signal from the image acquisition device. The electronic device is configured to obtain a plurality of to-be-tested images from the image signal, and respectively perform the following operations on each of the to-be-tested images: Using an object recognition model to find a peeling area in each of the to-be-tested images; Using a semantic segmentation model to identify a scratch area in each of the to-be-tested images to obtain a processed image; Taking out a cropped area corresponding to the peeling area from the processed image; Combining the peeling area and the cropped area to obtain a combined image; and Calculating a peeling ratio based on the combined image, where the electronic device is configured to: Perform a black-and-white inversion process on the combined image to obtain an inverted image; Perform an edge enhancement process on the inverted image to obtain an enhanced image; Use an edge detection algorithm to find a peeling block contour in the enhanced image; Calculate a peeling area based on the peeling block contour; and Calculate the peeling ratio based on the peeling area and the total area of the to-be-tested image.

6. The adhesion analysis system according to claim 5, wherein the electronic device is configured to: obtain a plurality of frames from the image signal; use a Laplacian operator to remove out-of-focus frames from the plurality of frames; and remove frames with pixel overlap among the determined in-focus frames, and use each remaining frame as each of the images to be measured.

7. The adhesion analysis system according to claim 5, wherein the electronic device is configured to: perform a blurring process on each of the images to be measured, so as to input the blurred image to be measured into the object recognition model and the semantic segmentation model.

8. The adhesion analysis system according to claim 5, wherein the electronic device is configured to: perform edge detection on the peeling area by using an edge detection model to obtain an edge image; and combine the edge image and the cropped area to obtain the combined image.

Images