A stamping die surface foreign matter detection method based on block template matching

Abstract

The application discloses a stamping die surface foreign matter detection method based on block template matching, which comprises the steps of image acquisition, image preprocessing, image blocking, foreign matter detection, data processing and judgment, result output and display and the like. The image acquisition covers all the areas to be detected through four groups of cameras and light sources arranged at the corners of the die. The image preprocessing extracts the accurate area to be detected according to color feature segmentation. The image blocking is blocked through a stepped sub-area distribution mode according to the ROI feature, so that the number of invalid areas is reduced and the efficiency of the detection algorithm is improved. The foreign matter detection improves the detection rate of foreign matters through the mode of rough matching first and then precise matching. The data processing and judgment improve the detection rate while reducing the false detection rate of the method through the adaptive sub-area method and the template updating method. The application can solve the problems of low detection rate and high false detection rate of the conventional machine vision detection method, and has a positive significance for improving the automation level of stamping product quality detection.

Description

Technical Field

[0001] This invention relates to the field of machine vision inspection technology, and more specifically, to a method for detecting foreign objects on the surface of stamping dies based on block template matching. Background Technology

[0002] During the stamping process of the washing machine cabinet, many small waste materials of different shapes are generated. If these waste materials remaining on the surface of the mold are not cleaned in time, defects such as indentations, scratches and abrasions will be formed on the surface of the cabinet during the next stamping.

[0003] Currently, there is no effective means to monitor whether waste material on the mold has been cleaned up. The existing method relies on manual visual inspection of stamped products at the end of the production line. If defects are found, the production line is stopped, and workers then locate and clean the waste material. However, this still results in batch scrapping, with each downtime taking about 10 minutes. This method is inefficient, unreliable, and costly, and cannot meet the demands of high-speed automated production. While some conventional machine vision inspection systems are used in industry, they suffer from low detection rates and high false positive rates. To address these issues, we have designed a mold surface foreign object detection technology based on segmented template matching. Summary of the Invention

[0004] The technical problem to be solved by this invention is: to design a method for detecting foreign objects on the surface of stamping dies based on block template matching, which solves the problems of low real-time performance and poor reliability of traditional methods and low detection rate of conventional machine vision inspection methods.

[0005] To achieve the above objectives, the technical solution provided by this invention is: a method for detecting foreign objects on the surface of a stamping die based on segmented template matching, comprising the following steps:

[0006] Step 1: The image acquisition module acquires the original image of the mold surface to be tested and transmits it to the industrial control computer for processing; Step 2: Irregular ROI regions are extracted from the original image to obtain an accurate image of the area to be tested;

[0007] Step 3: Divide the ROI image into small sub-regions using an improved image segmentation algorithm;

[0008] Step 4: Set an appropriate detection threshold, calculate the similarity between corresponding sub-regions by combining coarse matching and fine matching, and compare it with the threshold to achieve foreign object detection;

[0009] Step 5: Based on the similarity results obtained in Step 4, further judgment is made to improve the algorithm's detection rate and reduce the false detection rate;

[0010] Step 6: Output the final detection results. If a foreign object is found, issue an alarm and shut down the machine.

[0011] Furthermore, in step one, four sets of cameras and light sources are placed at the four corners of the mold to acquire images of the area to be tested on the mold surface, which are then transmitted to the industrial control computer for subsequent image processing. Near-infrared light is used as the light source and is placed to one side of the camera to provide illumination. The camera has a built-in infrared filter that responds to this light, eliminating the influence of changes in ambient light on the detection.

[0012] Furthermore, in step two, the ROI region to be extracted is manually specified and outlined in the image with a green line. Accurate ROI extraction is achieved through color features during the extraction process, which is as follows:

[0013] (1) Convert the image to HSV space and extract the green pixels in the image;

[0014] (2) Based on the extracted green pixels, fit an irregular quadrilateral and obtain its four vertices;

[0015] (3) Based on the four vertices obtained, specify the ROI by drawing the outline, and then obtain the final ROI image by cropping.

[0016] Furthermore, the specific process of the image segmentation algorithm in step three is as follows:

[0017] (1) Normalize the width of the ROI to a composite number and determine the number of sub-columns k. Based on this, determine the number of first-level sub-regions as k, denoted as R. i i = 1, 2, ..., k;

[0018] (2) Traverse the first and last rows of the ROI, find the non-zero pixels, and record the x-coordinate value and the corresponding sub-column m and n;

[0019] (3) Traverse each subregion R i The two column boundary lines col i-1 and col i Find the pixels with the smallest and largest non-zero grayscale values ​​at their respective ordinates, and record their ordinate values. and For the sub-region R corresponding to sub-column m m Just record and Similarly, for subregion R n Just record and

[0020] (4) Compare the ordinate values ​​of each corresponding pixel. Take:

[0021]

[0022] Obtain the sub-region R i The origin of coordinates is (col i-1 , min i ), and the height h i = max i - min i . For the sub-region R m , it is easy to obtain that its origin of coordinates is (col m-1 , 0), and the height is max m . Similarly, the origin of coordinates and height of the sub-region R n can be obtained.

[0023] (5) After obtaining the origin of coordinates and width-height parameters of each sub-region, locate the position of the sub-region in the ROI image and participate in the subsequent matching process.

[0024] Furthermore, in the process of template matching, use the image of the mold surface without foreign objects as the template, and the actually collected image as the待测图像 (to-be-tested image), calculate their similarity, and judge whether the image contains foreign objects according to the magnitude of the similarity value.

[0025] Furthermore, the specific process of the foreign object detection algorithm based on template matching in step four is as follows:

[0026] (1) First, perform a rough match on several first-level sub-regions R i divided from the ROI image to obtain the similarity S i ;

[0027] (2) Compare S i with the lower detection threshold t D1 and the upper detection threshold t' D1 of the first-level sub-region respectively. If S i < t D1 , then the similarity is low, and it is determined that there are foreign objects in the image. If S i > t' D1 , then the similarity is very high, and it is determined that there are no foreign objects in the image. If t D1 ≤ S i ≤ t' D1 , then there may be foreign objects, and it is necessary to further divide into second-level sub-regions for fine matching;

[0028] (3) On the basis of each first-level sub-region R i , divide it into four equal parts in a "field" shape as the second-level sub-regions, denoted as Calculate the similarity between each corresponding and compare with the detection threshold t D2 of the second-level sub-region. If < This indicates a low similarity, suggesting the presence of a foreign object in the image. The similarity is high, indicating that there are no foreign objects in the image;

[0029] (4) For sub-regions containing foreign objects, highlight their borders in red.

[0030] Furthermore, the detection threshold in step four was obtained through numerous matching experiments, and was specifically set to t. D1 =0.91,t′ D1 =0.99,t D2 =0.9.

[0031] Furthermore, the specific steps in step five, which involve further determining the similarity, are as follows:

[0032] (1) If the detection passes but there are two adjacent sub-regions with values ​​significantly lower than the normal value, consider that there is a foreign object on the boundary between the two adjacent sub-regions. Regenerate the sub-regions using the adaptive sub-region method and perform detection. The specific steps of the adaptive sub-region method are as follows:

[0033] (1.1) Difference between the image to be tested and the template image to remove interference from background pixels;

[0034] (1.2) Adaptive threshold segmentation is performed using the OTSU method, and the location of foreign objects is highlighted through binarization;

[0035] (1.3) Remove interfering pixels by opening operation in morphological operations, and then extract edges using the Canny operator;

[0036] (1.4) Based on the edge image, starting from the boundary line between adjacent sub-regions, extend the traversal column to both sides, traversing all pixels in each column. When a column is detected on one side as containing no non-zero pixels, stop the traversal and record the position of that column in the image. This coordinate is the new boundary of the sub-region. When the traversal on both sides has stopped, re-segment and generate a new temporary sub-region R based on the two recorded coordinates. temp Perform retesting;

[0037] (2) If the detection passes but the similarity value of a certain sub-region is lower than the normal value, or if the detection fails but can be determined to be a false detection, consider that the background gray level of a certain region of the image to be tested has changed and there is a large difference between the gray level of the corresponding region of the existing template. The existing template needs to be updated. The template update method is as follows:

[0038]

[0039] Where T k+1 The template image after the k-th detection is used as the detection template for the (k+1)-th detection. kLet I be the template image for the k-th detection. k D is the image to be tested during the k-th detection; k This represents the result of the k-th test, where 1 indicates a pass and 0 indicates a fail; d μ μ is the difference in grayscale mean between the template and the corresponding sub-region of the image. T The average grayscale value of the template sub-region; t represents the similarity value calculated for the corresponding sub-region. U To update the threshold, α is the update weight.

[0040] Furthermore, step six introduces an anomaly counting mechanism to determine whether a false alarm is detected before issuing a stop signal. Specifically, each time an alarm fails, the anomaly count is incremented, the system issues an alarm but does not stop. If no operator intervenes, the stamping operation continues. Two more consecutive checks are then performed. If both subsequent checks fail, it indicates the presence of a foreign object on the mold surface, and the system stops. If one of the latter two checks passes, it is considered a false alarm caused by interference, the system stops alarming, the anomaly count is reset to zero, and the production line resumes normal operation.

[0041] Compared with existing methods, the advantages of this invention are as follows: It proposes a foreign object detection method for stamping die surfaces based on block template matching. Compared to conventional template matching-based detection methods, this method utilizes an improved image block division method to divide sub-regions, making these sub-regions more closely match the Region of Interest (ROI), reducing the number of invalid regions, and improving the efficiency of the detection method. Furthermore, the method significantly improves the detection rate by employing a coarse-matching followed by a fine-matching approach. In addition, the improved template update method and anomaly counting mechanism reduce the false detection rate, minimizing downtime costs caused by false detections. The results of this invention have positive implications for improving the automation level of stamping product quality inspection. The relevant detection results can characterize the locations where foreign objects frequently occur, helping to guide improvements in die manufacturing processes. Attached Figure Description

[0042] Figure 1 This is a flowchart illustrating the overall testing process of the present invention;

[0043] Figure 2 This is a schematic diagram of the hardware layout of the image acquisition module of the present invention;

[0044] Figure 3 This is a schematic diagram of the area to be measured on the surface of the mold object to which this invention is applied;

[0045] Figures 4(a) and (b) show the original image collected and the ROI extraction results of the present invention, respectively.

[0046] Figure 5 The image segmentation result is shown in the proposed image segmentation method of this invention.

[0047] Figure 6 This is a flowchart of the foreign object detection method of the present invention;

[0048] Figures 7(a)-7(c) These are the original image, the local image of the sub-region to be adaptively generated, and the final generated image, respectively.

[0049] Figures 8(a)-8(e) This is a partial image showing the foreign object detection results of the present invention.

[0050] In the figure: 1. Upper mold, 2. Lower mold, 3. Surface of the mold to be tested, 4. Light source, 5. Camera, 6. Mold worktable, 7. Area to be tested, 8. Mold surface. Detailed Implementation

[0051] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the embodiments of the present invention.

[0052] Example: See Figure 1 The method for detecting foreign objects on the surface of a stamping die based on adaptive block template matching in this embodiment includes the following steps:

[0053] Add template

[0054] Before detection, an initial template needs to be added. The selection of the initial template is very important. It is necessary to ensure that there are no foreign objects in the template image and that the brightness and position of the template are appropriate.

[0055] For image acquisition, please refer to the following: Figure 2 and Figure 3 The system uses four sets of cameras and light sources positioned at the corners of the mold to capture images of the area to be tested on the mold surface.

[0056] Preprocessing

[0057] Preprocessing includes ROI extraction and image filtering. Referring to Figure 4, accurate ROI extraction is achieved by segmenting the green pixels in the image. Gaussian filtering is then used to smooth the image, reducing the impact of noise on the detection results.

[0058] Divide into sub-regions

[0059] The specific steps for dividing the region into sub-regions are as follows:

[0060] (1) The extracted ROI size is 1181×225. The width of the ROI is rounded to a total of 1184, and the number of sub-columns is determined to be 16. Therefore, the number of first-level sub-regions is determined to be 16, denoted as R. i i = 1, 2, ..., 16;

[0061] (2) Traverse the first and last rows of the ROI, find the non-zero pixels, and record the horizontal coordinate value and the corresponding sub-column m and n. In this embodiment, m = 2 and n = 14.

[0062] (3) Traverse each subregion R i The two column boundary lines col i-1 and col i Find the pixels with the smallest and largest non-zero grayscale values ​​at their respective ordinates, and record their ordinate values. and For the sub-region R corresponding to sub-column m m Just record and Similarly, for subregion R n Just record and

[0063] (4) Compare the ordinate values ​​of each corresponding pixel. Take:

[0064]

[0065] Obtain subregion R i The origin of the coordinate system is (col) i-1 min i ), high h i =max i -min i For subregion R m It is easy to find that its origin is (col). m-1 ,0), height is max m Similarly, we can obtain subregion R. n The origin and altitude of the coordinate system.

[0066] (5) After obtaining the coordinate origin and width and height parameters of each sub-region, locate the position of the sub-region in the ROI image and participate in the subsequent matching process.

[0067] See Figure 5 The improved image segmentation method divides sub-regions that fit the ROI, reducing interference from redundant areas.

[0068] Template matching

[0069] During template matching, an image of the mold surface without foreign objects is used as the template, and the actually acquired image is used as the test image. Their similarity is calculated, and the presence of foreign objects in the image is determined based on the similarity score. (See also...) Figure 6 The specific steps of the foreign object detection method based on block template matching are as follows:

[0070] (1) First, divide the ROI image into several first-level sub-regions \(R\). i Perform rough matching to obtain the similarity \(S\). i ;

[0071] (2) Compare \(S\). i with the lower detection threshold \(t\) of the first-level sub-region D1 and the upper detection threshold \(t'\). D1 If \(S\). i \(< t\). D1 Then the similarity is low, and it is determined that there is a foreign object in the image. If \(S\). i \(> t'\). D1 Then the similarity is very high, and it is determined that there is no foreign object in the image. If \(t\). D1 \(\leq S\). i \(\leq t'\). D1 Then there may be a foreign object, and it is necessary to further divide the second-level sub-regions for fine matching.

[0072] (3) Based on each first-level sub-region \(R\). i Divide it into four equal parts in a "field" shape as the second-level sub-regions, denoted as Calculate the similarity between each corresponding. and compare it with the detection threshold \(t\) of the second-level sub-region. If. It indicates that the similarity is low, and it is determined that there is a foreign object in the image. If. D2 It indicates that the similarity is high, and it is determined that there is no foreign object in the image. 说明相似度较低,判定图像中存在异物,若 说明相似度较高,判定图像中不存在异物;

[0073] The detection thresholds in the template matching process are obtained through a large number of experiments. Specifically, \(t\). D1 \( = 0.91\), \(t'\). D1 \( = 0.99\), \(t\). D2 \( = 0.9\).

[0074] Judgment and processing of detection results

[0075] The specific process of judgment and processing of detection results is as follows:

[0076] (1) If the detection passes but there are two adjacent sub-regions with values significantly lower than the normal value, it is considered that there is a foreign object on the boundary between the two adjacent sub-regions. The specific steps of the adaptive sub-region method for re-generating sub-regions and performing detection are as follows:

[0077] (1.1) Differentiate the待测图像 (to-be-tested image) and the template image to eliminate the interference of background pixels.

[0078] (1.2) Perform adaptive threshold segmentation by the OTSU method and highlight the location of the foreign object through binarization.

[0079] (1.3) Remove interfering pixels by opening operation in morphological operations, and then extract edges using the Canny operator;

[0080] (1.4) Referring to Figure 7, based on the edge image, starting from the boundary line between adjacent sub-regions, extend the column to both sides, traversing all pixels in each column. When a column is detected on one side without any non-zero pixels, stop the traversal and record the position of that column in the image. This coordinate is the new boundary of the sub-region. When the traversal on both sides has stopped, re-segment and generate a new temporary sub-region R based on the two recorded coordinates. temp Perform retesting;

[0081] (2) If the detection passes but the similarity value of a certain sub-region is lower than the normal value, or if the detection fails but can be determined to be a false detection, consider that the background gray level of a certain region of the image to be tested has changed and there is a large difference between the gray level of the corresponding region of the existing template. The existing template needs to be updated. The template update method is as follows:

[0082]

[0083] The parameters in the update method were determined experimentally, and the update threshold t was used. U Set it to 0.95, and update the weight α to 0.4.

[0084] The final result is shown in Figure 8. After the detection is completed, the final detection result image is output. If there are no foreign objects in the detection image, "Detection passed" is output. If foreign objects are detected in the image, the border of the sub-region where the foreign object is located is highlighted in red, "Detection failed" is output, the system issues an alarm signal and controls the production line to stop.

[0085] Before issuing a stop signal, an anomaly counting mechanism is used to determine if it is a false alarm. Each time a detection fails, the anomaly count is incremented, the system issues an alarm but does not stop. Two more consecutive checks are then performed. If both fail, it indicates the presence of a foreign object on the mold surface, and the system stops. If one of the two subsequent checks passes, it is considered a false alarm caused by interference, the system stops alarming, the anomaly count is reset to zero, and the production line resumes normal operation.

[0086] The above description of the present invention and its embodiments is not restrictive. The accompanying drawings are merely one embodiment of the present invention, and the actual application is not limited thereto. If those skilled in the art, inspired by this description, design similar structures and embodiments without departing from the spirit of the present invention, such designs should fall within the protection scope of the present invention.

Claims

1. A method for detecting foreign matter on a surface of a press die based on patch template matching, characterized by, The method comprises the following steps: Step one: the image acquisition module acquires the original mold surface image to be tested and transmits it to the industrial computer for processing; Step two: extract the irregular ROI region in the original image to obtain the accurate image of the region to be tested; Step three: divide the ROI image into small-sized sub-regions through an improved image blocking algorithm; Step four: set a suitable detection threshold, calculate the similarity between the corresponding sub-regions through coarse matching combined with fine matching, and compare it with the threshold to realize foreign matter detection; Step five: further process and judge according to the similarity result obtained in step four; Step six: output the final detection result, and issue an alarm and stop when there is foreign matter; In step three, the specific process of the image blocking algorithm is as follows: (1) The width dimension of the ROI is regularized to a multiple, and the number of sub-column divisions k is determined, thereby determining the number of first-level sub-regions as k, denoted as R i i = 1, 2, …, k; (2) traverse the first and last two rows of the ROI, find the non-zero pixel points in them, and record their horizontal coordinate values and the corresponding sub-columns m and n; (3) Traverse each sub-region R i Two column boundary lines col i-1 and col i , respectively find the minimum and maximum pixel points of the respective gray value, record their y-coordinate values and For sub-region R m corresponding to sub-column m, only record and Similarly for sub-region R n , only record and (4) compare the size of the vertical coordinate values of each corresponding pixel point, and take the minimum value: The coordinate origin of the sub-region R i is (col i-1 , min i ), and the height h i = max i -min i . For the sub-region R m , the coordinate origin is (col m-1 , 0), and the height is max m . The coordinate origin and the height of the sub-region R n can be obtained in the same way. (5) after obtaining the coordinate origin and width and height parameters of each sub-region, locate the position of the sub-region in the ROI image to participate in the subsequent matching process.

2. The foreign object detection method for a stamping die surface based on block template matching according to claim 1, characterized in that, The specific process of the foreign matter detection algorithm in step four is as follows: (1) Firstly, a plurality of first-level sub-regions R i are obtained by performing coarse matching on the ROI image i ; (2) S i The lower limit detection threshold t of the first-level sub-region D1 and the upper limit detection threshold t′ D1 Compare them separately, if S i <t D1 If the similarity is low, it is determined that there is a foreign object in the image. i >t′ D1 If the similarity is very high, it is determined that there are no foreign objects in the image. D1 ≤S i ≤t′ D1 There may be foreign objects, requiring further subdivision into secondary sub-regions for precise matching; (3) On the basis of each first-level sub-region R i , divide it into four equal parts in a "field" shape as the second-level sub-regions, denoted as j = 1, 2, 3, 4, calculate the similarity between each corresponding and and compare with the detection threshold t of the second-level sub-region D2 . If , it indicates that the similarity is low and it is determined that there is a foreign object in the image. If , it indicates that the similarity is high and it is determined that there is no foreign object in the image; (4) for the sub-regions with foreign matter, display their frames in red.

3. The foreign object detection method for a stamping die surface based on block template matching according to claim 1, characterized in that, The specific operation of further judgment according to the similarity result in step five is as follows: (1) if the detection passes but there are two adjacent sub-regions with values significantly lower than the normal value, consider that there is foreign matter on the boundary between the two adjacent sub-regions, and re-generate the sub-regions through the adaptive sub-region method and perform detection. The specific steps are as follows: (1.1) difference the image to be tested and the template image to eliminate the interference of background pixels; (1.2) perform adaptive threshold segmentation through the OTSU method, and highlight the foreign matter through binarization processing; (1.3) remove the interference pixels through opening operation in morphological operation, and then extract the edges using the Canny operator; (1.4) According to the edge image, extend from the starting column of the adjacent sub-region boundary line to both sides, traverse all the pixel points in each column, stop the traversal when detecting that a column does not contain non-zero pixel points on one side, record the coordinates of the column in the image, which is the new boundary of the sub-region, and when the traversal on both sides stops, generate a new temporary sub-region R according to the recorded two coordinates temp re-detecting; (2) if the detection passes but there is a sub-region with a similarity value lower than the normal value, or the detection does not pass but it can be determined as a false detection, consider that the background gray scale of a region of the image to be tested has changed, and there is a large difference in gray scale between the corresponding region of the existing template, so the existing template needs to be updated. The template updating method is as follows: where T k+1 is the template image after the kth detection, as the detection template for the k+1th detection, T k is the template image for the kth detection, I k is the image to be detected for the kth detection; D k is the result of the kth detection, 1 indicates that the detection passes, and 0 indicates that the detection fails; d μ is the difference between the average gray values of the corresponding sub-regions of the template and the image, μ T is the average gray value of the sub-region of the template; is the similarity value calculated for the corresponding sub-region, t U is the update threshold, and a is the update weight.

4. The foreign object detection method for a stamping die surface based on block template matching according to claim 1, characterized in that, In step six, an abnormality counting mechanism is designed to determine whether it is a false detection before issuing a stop signal. The specific operation is as follows: every time the detection does not pass, the abnormality count is increased by one, the system issues an alarm but does not stop, if there is no operator to handle it, the stamping operation continues, and then the detection is performed twice again. If both of the latter two detections do not pass, it means that there is foreign matter on the mold surface, and the system stops at this time. If one of the latter two detections passes, it is considered as a false detection caused by interference factors, the system stops alarming, the abnormality count is reset to zero, and the production line operates normally.

Images