A texture filling method and terminal based on a glass heating line coverage area repair

Abstract

The application discloses a texture filling method and terminal based on a glass heating line coverage area repair, comprising the following steps: S1, a glass area image is acquired, and a heating line of the glass area image is identified to obtain a heating line selected area image; S2, the heating line selected area image is repaired and filled by using an algorithm SSD algorithm. It can be seen that the heating line area is identified to obtain the heating line selected area, the heating line selected area is taken as a mask of an input to-be-repaired area, the SSD algorithm is used to realize repair and filling of the area covered by the heating line, so that errors in statistics and measurement of the fragments during detection are prevented.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, in particular to a texture filling method based on glass heating line coverage area repair and a terminal. BACKGROUND

[0002] The existing technology has strict detection requirements for the fragment state of tempered glass, and needs to count the number of fragments of tempered glass after impact to ensure the safety of the glass. At present, the existing technology generally uses manual detection or industrial detection to complete.

[0003] Among them, the industrial detection first needs to paste the tempered glass with a transparent film on one side. Then a special knocker is used to break the tempered glass, and with the cooperation of light source and camera, the line scan camera captures the image of the glass area as shown in Figure 1 .

[0004] However, due to the existence of heating lines and other obstructions in the automobile glass, the previous detection algorithm will have a large deviation in the statistics and measurement of fragments. SUMMARY

[0005] The technical problem to be solved by the present application is to provide a texture filling method based on glass heating line coverage area repair and a terminal, which can avoid errors in the statistics and measurement of fragments during detection.

[0006] In order to solve the above technical problems, the technical scheme adopted by the present application is:

[0007] A texture filling method based on glass heating line coverage area repair, comprising the steps of:

[0008] S1, acquiring a glass area image and identifying the heating line of the glass area image to obtain a heating line selected area image;

[0009] S2, repairing and filling the heating line selected area image using the SSD algorithm.

[0010] In order to solve the above technical problems, another technical scheme adopted by the present application is:

[0011] A texture filling terminal based on glass heating line coverage area repair, comprising a memory, a processor and a computer program stored in the memory and executable on the processor, and the processor executes the computer program to realize the above method.

[0012] The beneficial effect of the present application is that a texture filling method and terminal based on glass heating line coverage area repair, the heating line area is identified to obtain a heating line selection area, the heating line selection area is used as a mask of an input repair area, and an SSD algorithm is used to repair and fill the area covered by the heating line, so as to prevent errors in the statistics and measurement of the fragments during detection. BRIEF DESCRIPTION OF DRAWINGS

[0013] Figure 1 The glass fragment scanning image of the embodiment of the present application;

[0014] Figure 2 The flowchart of the texture filling method based on the glass heating line coverage area repair of the embodiment of the present application.

[0015] Figure 3 The image of the area near the heating line found by the heating line identification algorithm of the embodiment of the present application;

[0016] Figure 4 The schematic diagram of the image of the area near the heating line cut into a plurality of small images of the area near the heating line of the embodiment of the present application;

[0017] Figure 5 The schematic diagram of the curve fitting of the heating line identification algorithm of the embodiment of the present application;

[0018] Figure 6 The schematic diagram of the iteration process of the original SSD algorithm from the outer ring to the inner ring repair of the embodiment of the present application;

[0019] Figure 7 The schematic diagram of the glass fragment partial image before repair of the embodiment of the present application;

[0020] Figure 8 The schematic diagram of the glass fragment partial image after repair of the embodiment of the present application;

[0021] Figure 9 The structure schematic diagram of the texture filling terminal based on the glass heating line coverage area repair of the embodiment of the present application.

[0022] REFERENCE NUMERALS:

[0023] 1. A texture filling terminal based on glass heating line coverage area repair; 2. A processor; 3. A memory. DETAILED DESCRIPTION

[0024] To explain the technical content, the purpose and the effect of the present application in detail, the following will be explained in combination with the embodiments and the drawings.

[0025] Please refer to Figures 1 to 8A texture filling method based on a glass heating line coverage area repair, comprising the steps of:

[0026] S1, acquire a glass area image, and identify a heating line of the glass area image to obtain a heating line selected area image;

[0027] S2, repair and fill the heating line selected area image using an SSD algorithm.

[0028] As can be known from the above description, the beneficial effects of the present application are that a texture filling method based on a glass heating line coverage area repair and a terminal, by identifying a heating line area to obtain a heating line selected area, taking the heating line selected area as a mask of an input area to be repaired, and using an SSD algorithm to repair and fill the area covered by the heating line, errors in the statistics and measurement of the fragments during detection are prevented.

[0029] Further, the step S1 comprises the steps of:

[0030] S11, perform Hough transformation on the glass area image to obtain an arc image relative to the vertical and horizontal directions, dilate the arc image after filtering out impurities to obtain a heating line vicinity area image;

[0031] S12, cut the heating line vicinity area image into a plurality of heating line vicinity area small images;

[0032] S13, for each heating line vicinity area small image, use a set of spaced points to perform polynomial linear fitting to obtain a heating line upper edge and a heating line lower edge relative to the horizontal direction;

[0033] S14, integrate each heating line vicinity area small image into a heating line vicinity area image, and perform polynomial linear fitting after rotating to obtain a heating line upper edge and a heating line lower edge relative to the vertical direction;

[0034] S15, combine the heating line upper edge and the heating line lower edge relative to the horizontal and vertical directions to obtain a heating line selected area image.

[0035] As can be known from the above description, in order to ensure better repair effect of the heating line repair algorithm, a more accurate heating line identification algorithm is needed, which can process a certain arc and be controlled within a pixel error, and the heating line mask found by simply using Hough transformation has large errors and is easily disturbed by the texture of the glass fragments, which leads to the introduction of impurities or the selection of a larger area, so the heating line is improved to be better identified.

[0036] Further, the repair and filling of the heating line selected area image using the algorithm SSD algorithm is specifically repairing and filling each pixel of the heating line selected area image using the following formula:

[0037]

[0038] In the formula, d ssd is the value of the pixel to be filled, G is a Gaussian kernel function; winPixel is the pixel in the sliding window centered on the pixel to be filled; globalWinPixel is the pixel in the global corresponding sliding window centered on the matching pixel, and Pixel(x, y) is the pixel value with the horizontal and vertical coordinates of x and y.

[0039] As described above, the heating line selection area is repaired and filled by the SSD algorithm.

[0040] Further, the range of the pixel winPixel in the sliding window centered on the pixel to be filled is limited to a set size.

[0041] As described above, the size of the sliding window is controlled, thereby reducing the demand for computing power.

[0042] Further, the pixel globalWinPixel in the global corresponding sliding window centered on the matching pixel is limited to a partial area centered on the matching pixel.

[0043] As described above, the area of pixel matching is limited, reducing unnecessary calculations.

[0044] Further, the step S2 specifically divides the heating line selection area image into a plurality of heating line selection area small images using a skeleton extraction similar algorithm, and uses the SSD algorithm to repair and fill each heating line selection area small image.

[0045] As described above, the dynamic multi-thread saturates the computing power of the CPU, and optimizes the logical sequence of repair.

[0046] Further, the step S2 further includes selecting a cross point area in the heating line selection area, and when the heating line selection area image is divided into a plurality of heating line selection area small images using a skeleton extraction similar algorithm, the heating line selection area small image in which the cross point area is located is larger than the heating line selection area small image in which the cross point area is not located.

[0047] As described above, using a larger window size at the cross point makes the fusion effect better.

[0048] Further, the step S2 specifically judges the pixel sum centered on each pixel of the heating line selection area image, and judges whether the corresponding pixel is possibly at the edge of the crack according to the pixel sum, and if so, uses the SSD algorithm to repair and fill, otherwise uses the assignment method to fill.

[0049] From the above description, a large part of the pixels to be repaired is saved.

[0050] Further, the pixel and whether the pixel is likely to be on the crack edge is determined, specifically whether the pixel and is less than a set threshold value, if yes, it is determined that the corresponding pixel is likely to be on the crack edge, otherwise it is determined that the corresponding pixel is not likely to be on the crack edge.

[0051] From the above description, the determination of whether the pixel is likely to be on the crack edge is realized.

[0052] A texture filling terminal based on glass heating line coverage area repair, comprising a memory, a processor and a computer program stored on the memory and executable on the processor, and the processor realizes the method described above when executing the computer program.

[0053] The present application is used for the identification, repair and filling of the heating line of the glass scanning image in the glass test.

[0054] Please refer to Figures 1 to 8 , the embodiment one of the present application is:

[0055] To this end, a texture filling method based on glass heating line coverage area repair, comprising the following steps:

[0056] S1, acquiring a glass area image, and identifying the heating line of the glass area image to obtain a heating line selected area image.

[0057] Specifically, step S1 comprises the following steps:

[0058] S11, performing Hough transform on the glass area image to obtain an arc line image relative to vertical and horizontal, and after filtering out impurities, dilating the arc line image to obtain a heating line vicinity area image.

[0059] Through the opening and closing operation of image processing, the arc line relative to vertical and horizontal is found by Hough transform, and a certain impurity is cleaned through threshold value, and dilated to obtain the heating line vicinity area image, to obtain the image as shown in Figure 3 .

[0060] S12, cutting the heating line vicinity area image into a plurality of heating line vicinity area small images.

[0061] Using a skeleton extraction similar algorithm such as Figure 4 Cutting the heating line vicinity area image large image into heating line vicinity area small images to facilitate subsequent multi-threaded calculation.

[0062] S13, performing polynomial linear fitting on each heating line vicinity area small image using a set of scattered points to obtain the upper edge and lower edge of the horizontal heating line.

[0063] Specifically, for each heating line near area small image, a set of spaced points is used for polynomial linear fitting, and outliers in the points are calculated and excluded, and after multiple repetitions, the heating line upper edge and the heating line lower edge are obtained as shown in Figure 5 .

[0064] S14, each heating line near area small image is integrated into a heating line near area image, and after rotation, polynomial linear fitting is performed to obtain the heating line upper edge and the heating line lower edge in the vertical direction.

[0065] S15, the heating line upper edge and the heating line lower edge in the horizontal and vertical directions are merged to obtain a heating line selection area image.

[0066] S2, the heating line selection area image is repaired and filled by using the algorithm SSD.

[0067] Specifically, the embodiment researches and reproduces the texture filling algorithm SSD (sum of squared difference), which specifies the size of the sliding window, performs template matching on the sliding window with the center of the pixel to be filled and other regions in the image, and finds the minimum value of the product of the pixel squared difference and the Gaussian kernel function in the sliding window. The minimum value represents the closest template, and the value of the pixel can be filled into the target pixel. At the same time, the order of filling the pixels needs to be from outside to inside, from the outer circle to the inner circle, and the effect is as shown in Figure 6 .

[0068] Among them, the SSD formula is as follows:

[0069]

[0070] In the formula, d ssd is the Euclidean distance of matching, G is the Gaussian kernel function, winPixel is the pixel in the sliding window with the center of the pixel to be filled, globalWinPixel is the pixel in the global corresponding sliding window with the center of the matching pixel, and Pixel(x, y) is the pixel value with horizontal and vertical coordinates x and y.

[0071] However, it can be seen that the original SSD algorithm has many loops and repeated calculations in the syntax and logic structure, and has no ability to adapt to specific scenarios. The glass fragment image is huge, and the scanning image of the glass fragment is generally above 4k resolution. The original SSD algorithm consumes a lot of time.

[0072] Therefore, the embodiment improves the SSD algorithm, realizes several directions of optimizing the SSD algorithm, greatly improves the execution efficiency, and can also provide different repair effects for different accuracy requirements. Specifically, it includes:

[0073] Sliding window design:

[0074] Generally speaking, the larger the size of the matching sliding window, the larger the field of view, but the more computing power is required. Moreover, according to the distribution of the Gaussian kernel function, the edge weight of the sliding window is lower, and the amount of calculation of this part can be discarded to a certain extent through testing. Therefore, the pixels winPixel in the sliding window centered on the pixel to be filled are limited to a certain size to balance the calculation and effect.

[0075] Pixel matching area design:

[0076] After analysis, a large part of the optimal solution of the template matching area of the pixel point is near the pixel. Therefore, the pixels globalWinPixel in the global corresponding sliding window centered on the matching pixel are limited to a partial area centered on the matching pixel. For example, in the simple single straight line crack line repair scene, the best area for sliding matching is generally the upstream and downstream of the broken line, and a 5x5 size matching window can obtain good repair effect and reduce a large part of the calculation consumption.

[0077] Dynamic multi-thread design:

[0078] The intersection point area in the heating line selection area is selected, and a skeleton extraction algorithm is used to divide the heating line selection area image into a plurality of heating line selection area small images. When dividing, the size of the heating line selection area small image in which the intersection point area is located is greater than that of the heating line selection area small image in which the intersection point area is not located. The plurality of heating line selection area small images are calculated by multi-threading.

[0079] The heating line selection area image is divided into a plurality of heating line selection area small images by using a skeleton extraction algorithm. The small images are divided while the to-be-repaired line segment is as centered as possible. The thread pool is matched to perform parallel calculation as much as possible. For the intersection point of the heating line, a step-by-step line repair is used, and a larger window size is used at the intersection point to make the fusion effect better.

[0080] Surrounding information domain design and scaled pixel mapping design:

[0081] The SSD algorithm is a pixel-by-pixel repair, which can greatly reduce the calculation time by reducing the number of pixels to be calculated. The linear interpolation mapping can effectively reduce the calculation consumption, but there will be a certain blur in the repair effect, which needs to be matched with thresholding to sharpen the edges. In addition, the to-be-filled pixels are classified, and the sum of the pixels centered on each pixel of the heating line selection area image is calculated. According to the pixel sum, it is judged whether the corresponding pixel is on the crack edge. If so, the SSD algorithm is used for repair and filling, otherwise the assignment method is used for filling.

[0082] The determining whether the pixel is possibly on the crack edge is specifically determining whether the pixel and the judging pixel are less than a set threshold, if yes, it is determined that the corresponding pixel is possibly on the crack edge, otherwise, it is determined that the corresponding pixel is not possibly on the crack edge.

[0083] The glass area picture before repair is as shown in Figure 7 The repaired glass area picture is as shown in Figure 8

[0084] Please refer to Figure 9 Embodiment two of the present application is:

[0085] A texture filling terminal 1 based on glass heating line coverage area repair, comprising a memory 3, a processor 2 and a computer program stored on the memory 3 and executable on the processor 2, the processor 2 executes the computer program to realize the steps of the above embodiment one.

[0086] In summary, the present application provides a texture filling method and terminal based on glass heating line coverage area repair, which is realized based on CPU, and has lower cost and lower error than the deep learning algorithm requiring a graphics card. The execution efficiency of the improved algorithm has been greatly improved, and the precision and speed are dynamically adjustable. The execution efficiency of the SSD algorithm before and after improvement is compared in the experiment. The experimental picture size is 300x150 resolution, and the experimental CPU is Intel-12700F.

[0087] Original SSD algorithm Performance optimization Improved algorithm Repair time 32 seconds 26 seconds 0.2 seconds

[0088] Note: Performance optimization refers to optimization of variable storage and call without changing the original algorithm logic. The improved algorithm has a speed improvement of nearly 150 times compared to the original algorithm under the condition of basically the same repair effect.

[0089] In order to identify the improvement effect, 60 glass selected areas are sampled for comparison experiment:

[0090]

[0091] Note: The error calculation formula is MSE (Mean Squared Error), and the existence of the heating line will cause the segmentation of glass fragments. The fitting function modification uses the fragment density and the number of heating lines to fit the final statistical number. The repair algorithm uses the parameters of 17 seconds for 40 million pixels in execution efficiency.

[0092] From the two experimental results, the improved SSD algorithm has greatly improved in efficiency and performance.

[0093] ​The above merely illustrates the embodiments of the present application, and is not intended to limit the patent scope of the present application. Any equivalent transformation or direct or indirect application in the related technical field based on the content of the present application specification and drawings is also included in the patent protection scope of the present application.

Claims

1. A texture filling method based on repairing the area covered by a glass heating wire, characterized in that, Including the following steps: S1. Obtain an image of the glass region and identify the heating lines in the glass region image to obtain an image of the selected heating line region. S2. Use the SSD algorithm to repair and fill the selected area of ​​the image with heating lines; Step S1 includes the following steps: S11. Perform Hough transform on the glass region image to obtain relatively vertical and horizontal arc images. After filtering out impurities, dilate the arc image to obtain the image of the region near the heating line. S12. The image of the area near the heating line is segmented into multiple smaller images of the area near the heating line; S13. For each small image near the heating line, perform polynomial linear fitting using scatter points with a set interval to obtain the upper edge of the heating line and the lower edge of the heating line in the relative horizontal direction. S14. Combine the small images near each heating line into a single image near the heating line, rotate it, and then perform polynomial linear fitting to obtain the upper and lower edges of the heating line in the relatively vertical direction. S15. Merge the upper and lower edges of the heating line in the relatively horizontal and vertical directions to obtain the selected area image of the heating line.

2. The texture filling method based on glass heating wire coverage area repair according to claim 1, characterized in that, The SSD algorithm is used to repair and fill the image of the selected area of ​​the heating line. Specifically, each pixel of the selected area of ​​the heating line image is repaired and filled using the following formula: ； In the formula, d ssd G is the Euclidean distance for matching, G is the Gaussian kernel function; winPixel is the pixel in the sliding window centered on the pixel to be filled; globalWinPixel is the pixel in the global corresponding sliding window centered on the matching pixel; Pixel(x,y) is the pixel value with x and y coordinates.

3. The texture filling method based on glass heating wire coverage area repair according to claim 2, characterized in that, The range of pixels (winPixel) within a sliding window centered on the pixel to be filled is limited to a set size.

4. The texture filling method based on glass heating wire coverage area repair according to claim 2, characterized in that, The global WinPixel property, centered on the matching pixel, defines the area within the corresponding sliding window as a region centered on the matching pixel.

5. The texture filling method based on glass heating wire coverage area repair according to claim 1, characterized in that, Specifically, step S2 involves using a bone extraction-like algorithm to segment the heating line selection area image into multiple smaller heating line selection area images, and then using the SSD algorithm to repair and fill each smaller heating line selection area image.

6. A texture filling method for repairing areas covered by glass heating wires according to claim 5, characterized in that, Step S2 further includes selecting the intersection region in the heating line selection region. When the heating line selection region image is segmented into multiple heating line selection region small images using a bone extraction-like algorithm in step S2, the size of the heating line selection region small image where the intersection region is located is larger than the heating line selection region small image where the intersection region is not located.

7. The texture filling method based on glass heating wire coverage area repair according to claim 1, characterized in that, Step S2 specifically involves determining the sum of pixels convolved around each pixel in the selected area image of the heating line, and then determining whether the corresponding pixel might be at the edge of the crack based on the sum of pixels. If so, the SSD algorithm is used for repair and filling; otherwise, the assignment method is used for filling.

8. A texture filling method for repairing areas covered by glass heating wires according to claim 7, characterized in that, The step of determining whether a pixel might be at the edge of a crack based on the sum of its pixels specifically involves determining whether the sum of its pixels is less than a set threshold. If it is, the corresponding pixel is determined to be at the edge of a crack; otherwise, the corresponding pixel is determined to be impossible at the edge of a crack.

9. A texture filling terminal based on repairing the area covered by a glass heating wire, characterized in that, It includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method described in any one of claims 1-8.

Images