Quality inspection algorithm for automobile glass awning

Abstract

The invention belongs to the technical field of automobile glass sky screen quality detection, and particularly relates to an automobile glass sky screen quality inspection algorithm, which comprises the following steps of: 1, dividing a glass sky screen into an ROI (Region of Interest) area and a residual area, drawing the ROI area on a large image and projecting the ROI area into a small image; step 2, obtaining the position of each ROI mapped to the small image; 3, calculating points of each ROI in the range of each small image, and obtaining a mapping matrix of the standard small image and the test small image; 4, calculating quality inspection results of the test small graph and the standard small graph; 5, the quality inspection result of the test small image is fused into the large image, the interference caused by poor imaging of the spliced image is overcome, and meanwhile high-precision sky screen point quality inspection is kept.

Description

Technical Field

[0001] The present invention belongs to the technical field of quality inspection of automotive glass sunroofs, and particularly relates to a quality inspection algorithm for automotive glass sunroofs. Background Art

[0002] In the field of the automotive industry, automotive glass sunroofs have dense patterns, and their quality inspection includes the degree of roundness of sunroof points, the brightness and chromaticity of round dots, and the integrity of other patterns. Manual judgment is often affected by subjective factors and eye fatigue, resulting in misjudgment. In addition, enterprises will also face problems such as high labor costs and low efficiency.

[0003] At present, applying computer vision technology in the field of automotive glass industry has become a mainstream trend, and machine vision technologies and imaging solutions emerge in an endless stream. Since the field of view of most camera models cannot cover the entire glass, multiple cameras or a moving camera are often required to take pictures and stitch images. At the same time, different manufacturers often have different detection indicators for different regions of the entire glass. Secondly, such equipment often needs to simultaneously meet the detection requirements of brightness and chromaticity, which requires the use of a chrominance camera, further restricting the selection of cameras. In addition, automotive sunroof glass often has a certain curvature, but the quality inspection accuracy is still relatively high. By retaining a certain overlapping area for taking pictures, the stitching effect can be optimized, or the influence of the curvature on the detection accuracy can be reduced, and the part with better depth-of-field imaging is selected to retain the detection results. Although this solution is feasible, there are still many relatively large algorithm detection difficulties.

[0004] Disadvantage 1: It is difficult to stitch images. Even if each image has an overlapping area reserved with adjacent pictures, due to camera distortion and the glass surface, ordinary cutting and stitching cannot stitch the entire glass image. For example, Figure 1 cannot meet the situation where all points coincide. The stitching algorithm using feature point matching and correction has low robustness and it is difficult to ensure that the stitching effect meets the subsequent detection accuracy every time. In addition, due to the mechanical accuracy of the feeding being difficult to be very high, there is also a problem of not being able to confirm which side of the overlapping part to retain even if the features of adjacent area pictures are matched. The smallest dot radius may be only 5 pixels, and the degree of cutting of the edge dots at the same photographing station for each feeding is deviated. For example, Figure 2 the semi-circular dots at the edge of the stitched image cannot be stitched.

[0005] Disadvantage 2: The detection accuracy is not high. First of all, the glass has a curved surface. If taking pictures with multiple cameras is costly, the depth of field of a single moving camera for taking pictures will be limited, and the patterns imaged at different positions will have a certain degree of defocus and distortion. For example, Figure 3 , the imaging on the left side at the stitching edge is better, while the right side is defocused and distorted to be close to a triangle. Therefore, an additional detection result fusion mechanism is required. Summary of the Invention

[0006] To solve the problems of difficult spliced images and limited detection accuracy in the existing quality inspection technology for glass canopies, the present invention provides a quality inspection algorithm for automotive glass canopies, which overcomes the interference of poor imaging of spliced images and maintains high-precision quality inspection of canopy points.

[0007] The technical solution of the present invention is as follows: A quality inspection algorithm for automotive glass canopies, comprising the following steps: Step 1: Divide the glass canopy into an ROI region and a remaining region, draw the ROI region on the large image and project it onto the small image; Step 2: Obtain the position of each ROI region mapped onto the small image; Step 3: Calculate the centroid of the points within the range of each ROI on each small image, obtain the point coordinates, and thus obtain the mapping matrix between the standard small image and the test small image; Step 4: Calculate the comparison quality inspection result between the test small image and the standard small image; Step 5: Integrate the quality inspection result of the test small image into the large image.

[0008] Further, Step 1 is specifically: Draw the ROI region on the large image. When creating the automotive canopy glass variety, collect a group of small images with a standard sheet, select the large image with the best effect from multiple spliced large images using feature point matching, and record the homography matrix required for each small image to be projected onto the large image.

[0009] Further, Step 2 is specifically: Obtain the ROI regions with different quality inspection standards, obtain the remaining region from the difference set between the region of the entire glass image and the ROI regions, then dilate each ROI region, and calculate the position of each ROI on the standard sheet small image group according to the transformation matrix obtained in Step 1.

[0010] Further, the obtaining of the mapping matrix between the standard small image and the test small image in Step 3 is specifically: Calculate the mapping matrix by using the ransac algorithm for the centers of the canopy points in each standard small image and the test small image.

[0011] Further, Step 4 is specifically: Overlap the circular dots of the standard small image and the test small image through the mapping matrix, obtain the contour of each circular dot by means of mean dynamic threshold segmentation to calculate the area of each circular dot, and then calculate the IOU (Intersection over Union) of each circular dot. Calculate the minimum distance from all canopy points to the edge of the image or the ROI edge contour, and set a threshold for the minimum distance so that the points below this value are not included in the detection result.

[0012] Further, step 5 is specifically as follows: Repeat step 3 and step 4 to confirm that each ROI and the remaining area are detected on each small image, fuse the detection results, obtain the mapping coordinates from the canopy points on each small image to the large image through two inverse matrices, select the maximum IOU (Intersection over Union) as the result for the quality inspection of the canopy point contour, and fuse the small image corresponding to this IOU ratio onto the large image.

[0013] Compared with the prior art, the present invention has the following beneficial effects: (1) In terms of cost, this algorithm of the present invention achieves the adaptation ability of maintaining high-precision detection only by moving the camera to take pictures at multiple workstations, reducing the hardware cost. At the same time, the method of fusing the small image detection results onto the large image provides more fault tolerance mechanisms for mechanical precision and image distortion errors, and the imaging can preferentially ensure that the canopy points at the edge are only calculated once.

[0014] (2) The algorithm of the present invention breaks the inherent idea of first piecing together the images and then performing inspections. Whether it is camera calibration or feature point stitching, a large amount of computing resources are required, especially when dealing with large images with high resolution and high precision, which is particularly time-consuming. The algorithm of the present invention adopts a two-step approach. First, calculate the matrix required for stitching the large image once, and then obtain the approximate ROI mapping area through this matrix, and then perform fine matching. Compared with directly performing quality inspection on the large image, the detection difficulty of the algorithm is greatly reduced, and the high precision required during image stitching is avoided.

[0015] (3) The algorithm of the present invention avoids the error of the canopy points being split due to the division of the image edge and the ROI area. At the same time, by dilating the ROI area at the beginning, a certain overlapping area is ensured in the ROI area, that is, it is ensured that all points are detected.

[0016] (4) The algorithm of the present invention has good versatility. Whether it is image data or brightness and chromaticity data, relatively robust results can be obtained using this method. Description of the Drawings

[0017] Figure 1 Schematic diagram of the overlapping area of the camera taking pictures unable to overlap the canopy points due to distortion and the glass curved surface; Figure 2 Schematic diagram of the difficulty in piecing together the semi-circular points of the stitched image; Figure 3 Schematic diagram of the abnormal imaging of the canopy round dots at the stitching edge due to the glass curved surface and the depth of field; Figure 4 Schematic diagram of the ROI area mapped to the small image; Figure 5 Schematic diagram of the calculation of the intersection and union of the canopy points; Figure 6 Schematic diagram of the inability to match the edge points of the ROI; Figure 7 Schematic diagram of the fusion of the small image results. Specific embodiments

[0018] The present invention will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0019] See Figures 1-3 , a quality inspection algorithm for an automotive glass sunroof, comprising the following steps: Step 1: Divide the glass sunroof into an ROI region and a remaining region, draw the ROI region on the large image and project it onto the small image; Step 2: Obtain the position of each ROI region mapped onto the small image; Step 3: Calculate the points within the range of each ROI on each small image, and obtain the mapping matrix between the standard small image and the test small image; Step 4: Calculate the quality inspection results of the test small image and the standard small image; Step 5: Integrate the quality inspection results of the test small image into the large image.

[0020] The present invention will be further described below with reference to a specific embodiment: Step 1: Since the splicing effect on the large image is not good and there is also a problem that the overlapping region cannot determine which part to retain, then break the traditional idea and directly perform comparison and quality inspection on the small image, and then integrate the quality inspection results of each small image and display them on the large image. Since the quality inspection standards for different ROIs of the entire glass are different, it is divided into several ROIs and a remaining region, and the union of these regions is exactly the entire glass. For the convenience of the user's logic, the ROI region will be drawn on the large image on the user interface. In order to project the ROI of the large image onto the small image, it is necessary to first perform a splicing from the small image to the large image. This does not need to be spliced every time the material is fed, but only when creating the automotive sunroof glass variety, use the standard piece to collect a group of small images, and use feature point matching algorithms such as Harris to select the large image with the best effect from multiple spliced large images, and record the homography matrix required for each small image to be projected onto the large image. Among them, the cutting and retaining method for the overlapping part here can be manually set. The main purpose here is that the ROI drawn on the large image can be roughly projected onto the ROI of the small image, rather than ensuring that the spliced large image can be directly used as the object for direct inspection. The ROI projected onto the small image will be further matched later.

[0021] Step 2: Obtain the ROIs with different quality inspection standards from the user interface, obtain the remaining region from the difference set between the region of the entire glass image and these ROIs, and then perform a certain dilation on each ROI region. This is because later it is necessary to exclude the points near the ROI and the image edge to avoid misjudgment. Then, according to the transformation matrix obtained in step one, calculate the position of each ROI on the small image group of the standard piece. This calculation method is homography matrix calculation, which is a conventional technical means in the art. As Figure 4 shown, multiple ROIs are mapped onto 6 small images.

[0022] Step 3: Calculate the centroid of the points within the range of each ROI on each small image. The centroid can be obtained through the first moment in graphics, which is a conventional technical means in this field and will not be explained here. Note that when comparing, it is the standard small image and the test small image that are compared. Due to mechanical deviation and distortion, the ROI may just have the sky curtain points cut on the standard image and the test image, or the number of sky curtain points within this ROI area in the two images may be different. At this time, first calculate the mapping matrix of the center of each sky curtain point through the RANSAC algorithm. The RANSAC algorithm can eliminate the influence of the inconsistent quantity or defective points mentioned above. At this time, the accurate mapping matrix of the two images is obtained, and at the same time, the points outside the ROI are avoided from being included in the matrix calculation, because the comprehensive calculation of too many points will instead reduce the matching accuracy.

[0023] Although the homography matrix is 3x3, its degree of freedom is 8, and at least 4 pairs of matching points are required to calculate the matrix. In the case of multiple points, the RANSAC algorithm is needed to perform the fitting calculation to exclude the outlier points.

[0024] Step 4: Overlap the dots of the standard small image and the test small image through the mapping matrix, obtain the contour of each dot through mean dynamic threshold segmentation, and then calculate the area of each dot, and further calculate the IOU (Intersection over Union) of each dot, as Figure 5 The white area in the figure is the intersection part, and the white and gray areas together are the union part. At this time, at the edge of the ROI and the edge of the image, it will be found that some points are judged as defects due to mechanical accuracy and being cropped, or the IOU values calculated for them are very low. In order to avoid this part of misjudgment, it is necessary to calculate the minimum distance from all sky curtain points to the edge contour of the image or the ROI, and set a threshold for the minimum distance so that the points below this value are not included in the detection results of this part. As Figure 6 , the solid line is the edge of the ROI. The shaded points represent within the range of this ROI. The brighter shade represents that both points are matched, and the darker shade represents that one side is not matched. The closer to the edge of the ROI, the more likely it is to occur, so this part of the false detection needs to be ignored. At the same time, there is no need to worry that these excluded points will be ignored in this detection. All small images have overlapping areas, and in Step 2, the remaining areas are also dilated for each ROI, and each ROI has an overlapping area.

[0025] Step 5: Repeat Step 3 and Step 4. Each ROI and the remaining areas are detected on each small image, that is, all sky curtain points are detected according to different ROI standards. At this time, it will be found that due to the overlapping areas of the small images, the same point may be calculated 1 to 4 times. Therefore, it is necessary to fuse the detection results, referring to Figure 7, a point on the glass will be covered by multiple small images when taking pictures. Since overlapping pictures are taken, a point may have results from 1 - 4 small images. By obtaining the coordinate change information through the previously calculated matrix, it can be determined which small image results a point has. These results ultimately need to be fused.

[0026] Since the two transformation matrices have been calculated, the mapping coordinates of each point to the large image can be obtained through the two inverse matrices. Due to the glass surface curvature and distortion, it is difficult to ensure the same imaging effect for the same position. Therefore, the maximum IOU in multiple calculations is used as the result. For example, Figure 7 in the right figure under the same standard, the abnormal points marked by triangles due to the distortion of points are determined to be ok points in the left figure. So these points are ok points after the result fusion. In this way, the configuration of the camera focal length and depth of field can be mainly based on the sky curtain points at the image edge because there are more error tolerance mechanisms in the middle area, thereby improving the overall detection accuracy.

[0027] Similarly, it also includes brightness quality inspection and chromaticity quality inspection. The brightness quality inspection and chromaticity quality inspection use the detection mean value. As long as it is judged whether the corresponding value is within the set range, those not within the range will not be fused.

[0028] The algorithms for brightness - chromaticity data and image data are the same. The detection results are calculated on the small images and then fused onto the large image. The only differences are that the data types are floating - point values and grayscale values, and the mean value is used for brightness and chromaticity when fusing the results.

[0029] The sky curtain contour quality inspection method is to compare the IOU (Intersection over Union) of the standard points and the test points. The fusion method for the results of one point with multiple images is to take the maximum value. This is because the camera has a depth - of - field limit and the glass has a curvature. The best result represents the imaging angle closest to the vertical, so the maximum value is taken. The fusion method for brightness and chromaticity takes the average value because these two values are calibrated by the camera and will not be affected by the height and angle, so taking the average value is more stable.

[0030] In summary, the present invention avoids the algorithmic difficulty of re - inspection after splicing and avoids splicing errors. It reduces the complicated steps of traditional camera calibration and distortion correction. By dilating to create an ROI overlapping area, combined with a two - step matrix calculation mapping, one for a rough range and one for precise mapping, and using the ransac algorithm to reduce the interference of outliers and poorly segmented points, it ensures that all points can be accurately quality - inspected with high precision.

[0031] The present invention proposes a mechanism for fusing small-image results into large-image quality inspection results. For a single camera in different workstations during imaging, due to partial defocus and distortion caused by the glass curved surface, better imaged small images are used to replace the defocused parts. When a point is inspected multiple times, the best quality inspection result is adopted. This method provides a more fault-tolerant mechanism for the imaging overlapping area and can preferentially ensure that there are no sky curtain points in the image overlapping area at the edge of the focused image.

[0032] The above are only embodiments of the present invention and do not limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made by using the content of the specification and drawings of the present invention, or directly or indirectly applied in other related technical fields, shall be equally included in the patent protection scope of the present invention.

Claims

1. A quality inspection algorithm for an automotive glass sunroof, characterized in that, It includes the following steps: Step 1: Divide the glass canopy into ROI regions and the remaining regions, draw the ROI regions on the large image and project them onto the small image; Step 2: Obtain the positions of each ROI region mapped onto the small image; Step 3: Calculate the centroid of the points within the range of each ROI on each small image, obtain the point coordinates, and thus obtain the mapping matrix between the standard small image and the test small image; Step 4: Calculate the comparison and quality inspection results between the test small image and the standard small image; Step 5: Integrate the quality inspection results of the test small image into the large image.

2. The quality inspection algorithm for an automotive glass sunroof according to claim 1, characterized in that Specifically, Step 1 is as follows: Draw the ROI regions on the large image. When creating the glass varieties of the car canopy, collect the small image groups with the standard sheet, use feature point matching to select the large image with the best effect from the multiple stitched large images, and record the homography matrix required for each small image to be projected onto the large image.

3. The quality inspection algorithm for an automotive glass sunroof according to claim 2, wherein Specifically, Step 2 is as follows: Obtain the ROI regions with different quality inspection standards, obtain the remaining regions from the difference set between the entire image glass region and the ROI regions, then dilate each ROI region, and calculate the positions of each ROI on the standard sheet small image group according to the transformation matrix obtained in Step 1.

4. The quality inspection algorithm for an automotive glass sunroof according to claim 1, wherein, The obtaining of the mapping matrix between the standard small image and the test small image in Step 3 is specifically as follows: Calculate the mapping matrix by using the ransac algorithm for the centers of the canopy points in each standard small image and the test small image.

5. The quality inspection algorithm for an automotive glass sunroof according to claim 1, characterized in that Specifically, Step 4 is as follows: Overlap the circular dots of the standard small image and the test small image through the mapping matrix, obtain the contour of each circular dot through mean dynamic threshold segmentation to calculate the area of each circular dot, and then calculate the IOU (Intersection over Union) of each circular dot. Calculate the minimum distance from all canopy points to the edge of the image or the ROI edge contour, and set a threshold for the minimum distance so that the points below this value are not included in the detection results.

6. The quality inspection algorithm for an automotive glass sunroof according to claim 1, characterized in that, Specifically, Step 5 is as follows: Repeat Step 3 and Step 4 to confirm that each ROI and the remaining regions have been detected on each small image, fuse the detection results, obtain the mapping coordinates of the canopy points on each small image to the large image through two inverse matrices, select the maximum IOU as the result for the canopy point contour quality inspection, and fuse the small image corresponding to this IOU onto the large image.

Images