Glass fragment segmentation method based on edge enhancement and gradient-guided secondary segmentation

Abstract

The present invention proposes a glass fragment segmentation method based on edge enhancement and gradient-guided secondary segmentation, which is used for fragment image segmentation in tempered glass testing, and the method comprises the following steps: step one, after sticking tape to a single-sided area of ​​the tempered glass, it is shattered with a test knocker to form a shattered glass aggregate fixed by the tape; step two, placing the shattered glass aggregate on a luminous transparent platform, and then obtaining a scanned image of the shattered glass aggregate with a line scan camera located above the platform; step three, identifying the edges of the glass fragments in the scanned image with an edge enhancement method and a gradient-guided method, and obtaining the fragment segmentation state of the glass after being shattered; the present invention captures the image of the glass area with a line scan camera, which can solve the problem that in practical applications, the edge discontinuity caused by image noise and uneven illumination, and the jitter of the image edge caused by other factors make it difficult to segment the glass using a unified threshold.

Description

Technical Field

[0001] The invention relates to the technical field of image generation and image retrieval, in particular to a glass fragment segmentation method based on edge enhancement and gradient-guided secondary segmentation. Background Art

[0002] Since May 1, 2003, my country has implemented a mandatory inspection system for tempered glass, requiring inspection and testing of glass in buildings, cars and other scenes. Tempered glass segmentation technology is a necessary technology for performance testing of tempered glass. Through communication with manufacturers, when producing tempered glass on the production line, at least the first piece of glass, the middle piece of glass, and the last piece of glass produced are broken, and the densest and sparsest areas of the glass are counted. The factory has a limited budget for ordinary quality inspection equipment. The use of deep learning algorithms to segment the entire image requires a large amount of GPU memory, which will increase the hardware cost by tens of thousands. Therefore, this article's glass segmentation algorithm based on computer vision technology aims to accurately segment glass fragments through common computer computing devices such as CPUs.

[0003] Among them, the fragment segmentation method, such as the image segmentation method that only uses edge detection, has the following disadvantages: first, the photos taken by digital cameras have different viewing angles, resulting in different sizes of segmented glass. Secondly, the pixel-by-pixel glass detection and segmentation method is only suitable for small areas. For large-area skylights and building glass, the amount of calculation is too large.

[0004] If only the first-order operator is used to study what kind of detection operator is used for glass edge segmentation, the possible edge breakage in glass imaging is not analyzed, which limits the actual use.

[0005] In short, the research on glass fragment segmentation algorithm is still in the stage of academic research, and there is no in-depth study on the segmentation in the glass production process. In order to solve the above differences in imaging perspective, the patent of this invention uses a line scan camera to parallel scan the surface of the glass to solve the imaging difference. The Laplace operator is used for edge enhancement, and the gradient guidance is combined for secondary segmentation to solve the problem of edge breakage. Summary of the invention

[0006] The present invention proposes a glass fragment segmentation method based on edge enhancement and gradient-guided secondary segmentation. By capturing images of the glass area with a line scan camera, the problem of edge discontinuities caused by image noise and uneven lighting, as well as jittering of the image edge due to other factors in practical applications, which makes it difficult to segment using a unified threshold.

[0007] The present invention adopts the following technical solutions.

[0008] A glass fragment segmentation method based on edge enhancement and gradient-guided secondary segmentation is used for fragment image segmentation in tempered glass testing. The method comprises the following steps:

[0009] Step 1: After sticking the tape on a single surface of the tempered glass, the glass is broken with a test knocker to form a collection of broken glass fixed by the tape;

[0010] Step 2: placing the broken glass aggregate on a luminous transparent platform, and then obtaining a scan image of the broken glass aggregate with a line scan camera located above the platform;

[0011] Step 3: Use edge enhancement method and gradient guidance method to identify the edges of glass fragments in the scanned image to obtain the fragment segmentation state after the glass is broken.

[0012] The light source of the transparent platform adopts a long strip line scanning light source, and the light source is arranged at the bottom of the transparent platform; when the light of the transparent platform is transmitted from the broken glass aggregate, its light is fully transmitted in the central area of ​​the glass fragments of the broken glass aggregate, so that the central area of ​​the glass fragments appears white in the scanned image; the light of the transparent platform is refracted and reflected in the edge area of ​​the glass fragments to form scattered light, so that the edge area of ​​the glass fragments appears gray and black in the scanned image.

[0013] The edge enhancement method in step 3 comprises the following steps:

[0014] Step A1, determine the glass area of ​​the broken glass aggregate, separate the glass area from the background, and find the glass area by finding the maximum contour. For the input image, use the threshold thresh_a=60 to binarize the image, so that the darker glass edge area becomes black, and the glass center area becomes white. Use the contour search function to find the contour with the largest area in the original image of the scanned image, and use it as the glass area;

[0015] Step A2: Perform preliminary segmentation on the glass area, first perform Laplace edge enhancement on the glass area I, and obtain the Laplace gradient map of the glass area. The formula is:

[0016] I laplance =Laplance(I) Formula 1;

[0017] Then, the corresponding ladder image is subtracted from the glass area of ​​the captured original image to perform edge enhancement. The formula is:

[0018] I enhance =II laplance Formula 2:

[0019] After enhancement I enhanceThe threshold value is thresh_b = 120 for thresholding, which can obtain the preliminary glass segmentation result I mask_raw : The formula is:

[0020] I mask_raw =threhold(I laplance ) Formula 3.

[0021] The gradient-guided method is used to analyze the preliminary glass segmentation result I in step A2. mask_raw Further segmenting the glass blocks that are difficult to segment, that is, secondary segmenting the glass blocks whose area is less than 1% of the glass area, includes the following steps:

[0022] Step B1: First, for the connected glass area, obtain its mask area, and use the mask to obtain the original image of the difficult-to-segment area and define it as I pieces ;

[0023] Step B2: Take a larger threshold to perform stronger segmentation on the edges of the glass area that is difficult to segment. The trade-off between the tape line and the glass gap is to obtain the threshold of thresh_c=180, and the formula is

[0024] I pieces_threh =threshold(I pieces ) Formula 4;

[0025] Step B3: For rough edges, there may still be edge segments that are segmented. Further erosion operation is performed on them. The kernel size is selected as the erosion kernel of the kernel. The formula is:

[0026] I pieces_erode =erode(I pieces_threh , kernel) Formula 5;

[0027] Step B4, refine the eroded edge, for all the endpoints of the edge area, find that each point has only one foreground point in the n*n neighborhood, calculate the gradient of the point every 5 degrees, see which direction has a larger gradient, create a 2n*n rectangle, set the x-axis to n, the y-axis height to 2n, if the y-direction gradient is larger, take the neighborhood as n*2n, if there is a foreground point of a broken edge, connect the endpoint of the broken edge and the point between the foreground point, and change the middle of the connection line to the value of the foreground point; form an edge connection result that can connect adjacent foreground points while ensuring the correctness of edge information, expressed as the formula:

[0028] I pieces_line =connect_line(thinning(I pieces_erode )) Formula 6;

[0029] Step B5: The edge I segmented in the previous step pieces_line The edge I of the previous segmentation pieces_threh Perform the fusion operation and paste it back to I mask_raw The segmentation results of all glass fragments areas can be obtained.

[0030] In the gradient-guided method, all the small glass regions are first sorted, and then 1% of the glass region is selected for secondary segmentation.

[0031] After step B5, the segmentation result in step B5 needs to be suppressed for small areas. The small areas are small fragment areas that do not need to be paid attention to due to small impurities in the glass image after multiple segmentation and thresholding processes. The specific method is to suppress the small areas with upper contours in the glass blocks in the image or the area of ​​the glass is less than a minimum defined threshold min_care_area, and then suppress these glass areas to become background areas, so as to obtain the final glass segmentation result I mask .

[0032] In the method, a CPU is used to execute an algorithm to locate and segment the glass breakage area.

[0033] The advantages of the present invention are mainly reflected in two aspects:

[0034] First, by introducing line scan camera imaging and computer vision technology, the segmentation process of tempered glass was defined for the first time in a way that is consistent with industrial applications.

[0035] Second, the innovative introduction of a glass edge reconnection method based on edge gradient guidance solves the problem of unclear glass edges leading to incorrect definition of the largest and longest blocks, and assists in more accurately locating the densest and sparsest areas.

[0036] The present invention optimizes the algorithm and uses a CPU to locate and segment the glass area, thereby greatly reducing the equipment purchase cost. At the same time, the positioning and segmentation of the glass area is accurate and reliable.

[0037] The present invention captures images of the glass area by a line scan camera, which can solve the problem of edge discontinuity caused by image noise and uneven lighting in tempered glass testing, as well as jitter of the image edge caused by other factors, which makes it difficult to segment the image using a unified threshold. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] The present invention is further described in detail below with reference to the accompanying drawings and specific embodiments:

[0039] Attached Figure 1 is a schematic diagram of a scanning result image of broken glass according to the present invention;

[0040] Attached Figure 2 It is a schematic diagram of the process of the method of the present invention;

[0041] Attached Figure 3 is a schematic diagram of the secondary segmentation method of the present invention;

[0042] Attached Figure 4 Schematic diagram of the edge enhancement method of the present invention. DETAILED DESCRIPTION

[0043] As shown in the figure, a glass fragment segmentation method based on edge enhancement and gradient-guided secondary segmentation is used for fragment image segmentation in tempered glass testing. The method includes the following steps:

[0044] Step 1: After sticking the tape on a single surface of the tempered glass, the glass is broken with a test knocker to form a collection of broken glass fixed by the tape;

[0045] Step 2: placing the broken glass aggregate on a luminous transparent platform, and then obtaining a scan image of the broken glass aggregate with a line scan camera located above the platform;

[0046] Step 3: Use edge enhancement method and gradient guidance method to identify the edges of glass fragments in the scanned image to obtain the fragment segmentation state after the glass is broken.

[0047] The light source of the transparent platform adopts a long strip line scanning light source, and the light source is arranged at the bottom of the transparent platform; when the light of the transparent platform is transmitted from the broken glass aggregate, its light is fully transmitted in the central area of ​​the glass fragments of the broken glass aggregate, so that the central area of ​​the glass fragments appears white in the scanned image; the light of the transparent platform is refracted and reflected in the edge area of ​​the glass fragments to form scattered light, so that the edge area of ​​the glass fragments appears gray and black in the scanned image.

[0048] The edge enhancement method in step 3 comprises the following steps:

[0049] Step A1, determine the glass area of ​​the broken glass aggregate, separate the glass area from the background, and find the glass area by finding the maximum contour. For the input image, use the threshold thresh_a=60 to binarize the image, so that the darker glass edge area becomes black, and the glass center area becomes white. Use the contour search function to find the contour with the largest area in the original image of the scanned image, and use it as the glass area;

[0050] Step A2: Perform preliminary segmentation on the glass area, first perform Laplace edge enhancement on the glass area I, and obtain the Laplace gradient map of the glass area. The formula is:

[0051] I laplance=Laplance(I) Formula 1;

[0052] Then, the corresponding ladder image is subtracted from the glass area of ​​the captured original image to perform edge enhancement. The formula is:

[0053] I enhance =II laplance Formula 2:

[0054] After enhancement I enhance The threshold value is thresh_b = 120 for thresholding, which can obtain the preliminary glass segmentation result I mask_raw : The formula is:

[0055] I mask_raw =threhold(I laplance ) Formula 3.

[0056] The gradient-guided method is used to analyze the preliminary glass segmentation result I in step A2. mask_raw Further segmenting the glass blocks that are difficult to segment, that is, secondary segmenting the glass blocks whose area is less than 1% of the glass area, includes the following steps:

[0057] Step B1: First, for the connected glass area, obtain its mask area, and use the mask to obtain the original image of the difficult-to-segment area and define it as I pieces ;

[0058] Step B2: Take a larger threshold to perform stronger segmentation on the edges of the glass area that is difficult to segment. The trade-off between the tape line and the glass gap is to obtain the threshold of thresh_c=180, and the formula is

[0059] I pieces_threh =threshold(I pieces ) Formula 4;

[0060] Step B3: For rough edges, there may still be edge segments that are segmented. Further erosion operation is performed on them. The kernel size is selected as the erosion kernel of the kernel. The formula is:

[0061] I pieces_erode =erode(I pieces_threh , kernel) Formula 5;

[0062] Step B4, refine the eroded edge, for all the endpoints of the edge area, find that each point has only one foreground point in the n*n neighborhood, calculate the gradient of the point every 5 degrees, see which direction has a larger gradient, create a 2n*n rectangle, set the x-axis to n, the y-axis height to 2n, if the y-direction gradient is larger, take the neighborhood as n*2n, if there is a foreground point of a broken edge, connect the endpoint of the broken edge and the point between the foreground point, and change the middle of the connection line to the value of the foreground point; form an edge connection result that can connect adjacent foreground points while ensuring the correctness of edge information, expressed as the formula:

[0063] I pieces_line =connect_line(thinning(I pieces_erode )) Formula 6;

[0064] Step B5: The edge I segmented in the previous step pieces_line The edge I of the previous segmentation pieces_threh Perform the fusion operation and paste it back to I mask_raw The segmentation results of all glass fragments areas can be obtained.

[0065] In the gradient-guided method, all the small glass regions are first sorted, and then 1% of the glass region is selected for secondary segmentation.

[0066] After step B5, the segmentation result in step B5 needs to be suppressed for small areas. The small areas are small fragment areas that do not need to be paid attention to due to small impurities in the glass image after multiple segmentation and thresholding processes. The specific method is to suppress the small areas with upper contours in the glass blocks in the image or the area of ​​the glass is less than a minimum defined threshold min_care_area, and then suppress these glass areas to become background areas, so as to obtain the final glass segmentation result I mask .

[0067] In the method, a CPU is used to execute an algorithm to locate and segment the glass breakage area.

Claims

1. A glass fragment segmentation method based on edge enhancement and gradient-guided secondary segmentation is used for fragment image segmentation in tempered glass testing, which is characterized by: The method comprises the following steps: Step 1: After sticking the tape on a single surface of the tempered glass, the glass is broken with a test knocker to form a collection of broken glass fixed by the tape; Step 2: placing the broken glass aggregate on a luminous transparent platform, and then obtaining a scan image of the broken glass aggregate with a line scan camera located above the platform; Step 3: Using edge enhancement method and gradient guidance method to identify the edges of glass fragments in the scanned image, and obtain the fragment segmentation state after the glass is broken; The light source of the transparent platform adopts a long strip line scanning light source, and the light source is arranged at the bottom of the transparent platform; when the light of the transparent platform is transmitted from the broken glass aggregate, the light is fully transmitted in the central area of ​​the glass fragments of the broken glass aggregate, so that the central area of ​​the glass fragments appears white in the scanned image; the light of the transparent platform is refracted and reflected in the edge area of ​​the glass fragments to form scattered light, so that the edge area of ​​the glass fragments appears gray-black in the scanned image; The edge enhancement method in step 3 comprises the following steps: Step A1, determine the glass area of ​​the broken glass aggregate, separate the glass area from the background, find the glass area by finding the maximum contour, use the threshold value thresh_a=60 for the input image to binarize the image, make the darker glass edge area black, and make the glass center area white, use the contour search function to find the contour with the largest area in the original image of the scanned image, and use it as the glass area; Step A2: Perform preliminary segmentation on the glass area, first perform Laplace edge enhancement on the glass area I, and obtain the Laplace gradient map of the glass area. The formula is: I laplance =Laplance(I) Formula 1; Then, the corresponding gradient image is subtracted from the glass area of ​​the captured original image to perform edge enhancement. The formula is: I enhance =II laplance Formula 2; After the enhancement of I enhance The threshold value is thresh_b = 120, and the preliminary glass segmentation result I is obtained. mask_raw : The formula is: I mask_raw =threshold(I laplance ) Formula 3; The gradient-guided method is used to analyze the preliminary glass segmentation result I in step A2. mask_raw Further segmenting the glass blocks that are difficult to segment, that is, secondary segmenting the glass blocks whose area is less than 1% of the glass area, includes the following steps: Step B1: First, for the connected glass area, obtain its mask area, and use the mask to obtain the original image of the difficult-to-segment area and define it as I pieces ; Step B2: Take a larger threshold to perform stronger segmentation on the edges of the glass area that is difficult to segment. The trade-off between the tape line and the glass gap is to obtain the threshold of thresh_c=180, and the formula is I pieces_threh =threshold(I pieces ) Formula 4; Step B3: For rough edges, there may still be edge segments that are segmented. Further erosion operation is performed on them. The kernel size is selected as the erosion kernel of the kernel. The formula is: I pieces_erode = erode(I pieces_threh , kernel); Equation 5 Step B4, refine the eroded edge, for all the endpoints of the edge area, find that each point has only one foreground point in the n*n neighborhood, calculate the gradient of the endpoint every 5 degrees, see which direction has a larger gradient, create a 2n*n rectangle, set the x-axis to n, the y-axis height to 2n, if the y-direction gradient is larger, take the neighborhood as n*2n, if there is a foreground point of a broken edge, connect the endpoint of the broken edge and the point between the foreground point, and change the middle of the connection line to the value of the foreground point; form an edge connection result that can connect adjacent foreground points while ensuring the correctness of edge information, which can be expressed as the formula: I pieces_line =connect_line(thinning(I pieces_erode )) Formula 6; Step B5: The edge I segmented in the previous step pieces_line The edge I of the previous segmentation pieces_threh Perform the fusion operation and paste it back to I mask_raw The segmentation results of all glass fragments areas can be obtained.

2. The glass fragment segmentation method based on edge enhancement and gradient-guided secondary segmentation according to claim 1, characterized in that: In the gradient-guided method, all the small glass regions are first sorted, and then the glass regions with an area less than 1% are selected for secondary segmentation.

3. The glass fragment segmentation method based on edge enhancement and gradient-guided secondary segmentation according to claim 1, characterized in that: After step B5, the segmentation result in step B5 needs to be suppressed for small areas. The small areas are small fragment areas that do not need to be paid attention to due to small impurities in the glass image after multiple segmentation and thresholding processes. The specific method is to suppress the small areas with upper contours in the glass blocks in the image or the area of ​​the glass is less than a minimum defined threshold min_care_area, and then suppress these glass areas to become background areas, so as to obtain the final glass segmentation result I mask .

4. The glass fragment segmentation method based on edge enhancement and gradient-guided secondary segmentation according to claim 1, characterized in that: In the method, a CPU is used to execute an algorithm to locate and segment the glass breakage area.

Images