Circuit board element detection method based on improved multi-target template matching

Abstract

The invention provides a circuit board element detection method based on improved multi-target template matching, which comprises the following steps of: firstly, collecting image data of a circuit board to be detected by using an industrial camera; secondly, preprocessing the collected image to improve the image quality and obtain more obvious to-be-extracted features; segmenting the preprocessed image into a plurality of small grids, and performing edge detection and optimization in each grid; image pyramids with different resolutions are generated in each grid, and proper template levels are dynamically selected according to the sizes and positions of elements, so that the matching flexibility and accuracy are improved; and finally, combining the detection results of all the grids to form a final detection result. According to the method, the limitations of long time consumption and low precision of circuit board element detection in an industrial production process are overcome, and the matching accuracy and efficiency of an algorithm in a complex scene are improved.

Description

Technical Field

[0001] The invention belongs to the field of electronic manufacturing and detection, and in particular relates to a circuit board component detection method based on improved multi-target template matching. Background Art

[0002] Circuit board component detection is a key link in the electronics manufacturing industry, which aims to identify and locate the installation status and quality of various components on the circuit board, such as missing components, misalignment, damage, and poor solder joints. Traditional circuit board component detection methods mainly rely on manual visual inspection. Although this method is intuitive, it has problems such as low efficiency, susceptibility to human factors, and unstable detection results. With the development of computer vision and image processing technology, algorithm-based automatic detection methods have gradually become mainstream. These methods obtain circuit board images through image acquisition devices, and use image processing and pattern recognition technology to automatically detect the status of components, which greatly improves the speed and accuracy of detection. However, existing automatic detection methods still have certain limitations when facing complex backgrounds and multiple types of components.

[0003] In recent years, multi-target template matching, as a technology for realizing target detection by finding the area in the image that is most similar to the predefined template, has been widely used in various detection fields. This method usually includes the following steps: first, extract the template from the known target sample; then, search for the area most similar to the template in the image to be detected by sliding the window; finally, determine the position and state of the target according to the matching result. Although multi-target template matching has improved the detection efficiency to a certain extent, there are still some problems. First, the computational complexity of multi-target template matching is large, especially when processing high-resolution images, the computing resources are consumed significantly. Secondly, this method still has room for improvement in detection accuracy, especially in the case of complex backgrounds and the coexistence of multiple types of components, it is easy to have false detection and missed detection. Therefore, in order to further improve the efficiency and accuracy of detection and meet the needs of the electronic manufacturing industry for high precision and fast response, it is urgent to improve the existing multi-target template matching method. Summary of the invention

[0004] The purpose of the present invention is to provide a circuit board component detection method based on improved multi-target template matching, so as to improve the accuracy, efficiency and anti-interference ability of existing detection technology in complex environments.

[0005] To achieve the above object, the present invention adopts a circuit board component detection method based on improved multi-target template matching, which specifically includes the following steps:

[0006] Step 1: Data acquisition: Use a high-resolution industrial camera and a stable lighting system to collect image data of the circuit board;

[0007] Step 2: Image preprocessing: Perform comprehensive image preprocessing operations on the collected images to improve image quality and feature extraction effects;

[0008] Step 3, grid division: divide the preprocessed image into multiple small grids, each grid as an independent processing unit;

[0009] Step 4: Feature extraction and optimization: Perform edge detection and optimization within each grid to extract a more realistic template outline;

[0010] Step 5: Hierarchical template matching: Generate image pyramids of different resolutions in each grid and perform multi-level template matching to improve matching flexibility and accuracy;

[0011] Step 6: Merge results: Merge the matching results of each grid to form the final detection result.

[0012] In step 2, the image preprocessing method sequentially performs grayscale conversion, detail enhancement and noise removal on the collected image. First, the image is converted into a grayscale image to reduce the amount of calculation. Then, a guided filtering method is used to enhance the image details and smooth the edges. Furthermore, a Gaussian filtering method is used to remove image noise.

[0013] In step 3, the grid division method independently calculates the gray value of each grid through grid division, skips the area where the gray value differs too much from the average gray value of the template, and reduces unnecessary calculations.

[0014] In step 4, the feature extraction and optimization method performs edge detection in each grid, extracts edges using the Sobel operator, and optimizes the edge detection results using the hysteresis threshold method. The extracted edges are fitted with a straight line using the weighted least squares method to obtain a more realistic contour.

[0015] In step 5, the hierarchical template matching method generates image pyramids of different resolutions in each grid and generates templates of different sizes to be suitable for detecting target components of different scales. Matching is performed at different levels, and the appropriate template level is dynamically selected according to the size and position of the target component to improve the flexibility and accuracy of matching.

[0016] In step 6, the result merging method maps the intermediate level matching result of each grid back to the coordinate system of the original image to form a global detection result. The detection results of all grids are merged to form the final detection result.

[0017] Compared with the prior art, the advantages of the present invention are:

[0018] 1. Introduce grid division method to reduce the amount of calculation and improve detection efficiency.

[0019] The present invention divides the image into small processing units, reduces the amount of data, reduces the computational complexity, improves the processing speed, and supports parallel processing. At the same time, it can retain local features and avoid feature distortion, and is particularly suitable for circuit board detection with complex backgrounds and coexistence of multiple types of components.

[0020] 2. Improve the multi-target template matching method to improve detection accuracy.

[0021] The present invention adopts a hierarchical template matching method, which generates templates of different sizes by generating image pyramids of different resolutions, suitable for detecting target components of different scales. Matching is performed at different levels, and the appropriate template level is dynamically selected according to the size and position of the target component, which improves the flexibility and accuracy of matching.

[0022] 3. Comprehensive image preprocessing and feature optimization operations to improve the ability to resist interference in complex environments.

[0023] The present invention effectively improves the image quality and enhances the adaptability to complex backgrounds and lighting conditions through a comprehensive image preprocessing method, including grayscale conversion, detail enhancement and noise removal. In addition, the Sobel operator is used to extract edges in the feature extraction and optimization steps, and the edge detection results are optimized by the hysteresis threshold method and the weighted least squares method, which further improves the anti-interference ability in complex environments.

[0024] In summary, by introducing the grid division method and hierarchical template matching technology to improve the traditional multi-target template matching method, the accuracy and efficiency of circuit board component detection are significantly improved. The grid division method reduces the amount of calculation and improves the detection efficiency; the hierarchical template matching improves the flexibility and accuracy of matching; the improved image preprocessing method enhances the adaptability to complex environments. These innovations enable the present invention to meet the needs of high-precision and fast-response automated detection in the electronics manufacturing industry. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 A flowchart of a circuit board component detection method based on improved multi-target template matching;

[0026] Figure 2 Extract and optimize schematics for circuit board component features based on meshing;

[0027] Figure 3 Flowchart of hierarchical template matching. DETAILED DESCRIPTION

[0028] The circuit board component detection method based on improved multi-target template matching of the present invention will be described in more detail below in conjunction with a schematic diagram, wherein a preferred embodiment of the present invention is shown, and it should be understood that those skilled in the art can modify the present invention described herein while still achieving the advantageous effects of the present invention. Therefore, the following description should be understood as being widely known to those skilled in the art, and not as a limitation of the present invention.

[0029] like Figures 1 to 3 , a circuit board component detection method based on improved multi-target template matching is designed to improve the accuracy and speed of circuit board component detection. This method optimizes the multi-target template matching method by introducing a grid division method and an improved image pyramid method, thereby improving the efficiency and accuracy of component detection. Specifically, the following steps are included:

[0030] Step 1: Data acquisition: Use a high-resolution industrial camera and a stable lighting system to collect image data of the circuit board;

[0031] Step 2: Image preprocessing: Perform comprehensive image preprocessing operations on the collected images to improve image quality and feature extraction effects;

[0032] Specifically, the collected images are processed in sequence, including grayscale conversion, detail enhancement and noise removal. First, the image is converted into a grayscale image to reduce the amount of calculation. Then, the guided filtering method is used to enhance the image details and smooth the edges. Furthermore, the Gaussian filtering method is used to remove image noise.

[0033] Step 3: Grid division: Divide the preprocessed image into multiple small grids, with each grid serving as an independent processing unit.

[0034] Through grid division, the grayscale value of each grid is calculated independently, and the areas where the grayscale value is too different from the average grayscale value of the template are skipped to reduce unnecessary calculations.

[0035] Step 4: Feature extraction and optimization: edge detection and optimization are performed within each grid.

[0036] Specifically, edge detection is performed in each grid, edges are extracted using the Sobel operator, and the edge detection results are optimized using the hysteresis threshold method. The extracted edges are fitted with a straight line using the weighted least squares method to obtain a more realistic contour.

[0037] Step 5: Hierarchical template matching: Generate image pyramids of different resolutions in each grid and perform multi-level template matching;

[0038] Specifically, image pyramids of different resolutions are generated in each grid, and templates of different sizes are generated to suit the detection of target components of different scales. Matching is performed at different levels, and the appropriate template level is dynamically selected according to the size and position of the target component to improve the flexibility and accuracy of matching.

[0039] Step 6: Merge results: Merge the matching results of each grid to form the final detection result.

[0040] Specifically, the intermediate level matching results of each grid are mapped back to the coordinate system of the original image to form a global detection result. The detection results of all grids are merged to form the final detection result.

[0041] Figure 2 It is a schematic diagram of circuit board component feature extraction and optimization based on grid division, namely steps 3 to 4.

[0042] Taking a comprehensive preprocessed image containing a circuit board as an example, as shown in the figure, first, an image that has been converted to grayscale, enhanced in details and removed in noise is obtained from the preprocessing step.

[0043] Assume the image size is H×W , choose an appropriate grid size g×g.

[0044] Divide the image into n parts vertically and horizontally. h and n w Grids:

[0045]

[0046] Read the template image and calculate the average grayscale value I of the template image:

[0047]

[0048] Where M and N are the height and width of the template image, respectively, and I(i,j) is the grayscale value of the template image at position (i,j).

[0049] For each grid G kl , calculate its average gray value

[0050]

[0051] Where k and l represent the row and column indices of the grid, respectively, and I(x,y) represents the grayscale value of the image at position (x,y).

[0052] Then, a threshold ε is set to determine whether the average grayscale value of the grid is close to the average grayscale value of the template. kl , compare their average gray value The average gray value of the template

[0053] The constraints are as follows:

[0054]

[0055] For those that meet the constraints, the subsequent template matching operation is performed. Otherwise, the grid is skipped.

[0056] After screening and determining the grids that need to be matched, the Sobel operator is used in each grid to calculate the gradient amplitude G of each pixel point (x,y) and direction θ (x,y) ,This step is used to detect the boundaries and contours of components.

[0057] For each pixel (x, y), check the gradient values ​​of neighboring points in its gradient direction and remove non-maximum values. The specific steps are as follows:

[0058] 1. Determine the gradient direction: First, calculate the gradient direction θ of the pixel point (x, y) (x,y) Then, the gradient directions are quantized into four main directions: 0° (horizontal), 45° (right oblique), 90° (vertical), and 135° (left oblique).

[0059] 2. Find neighboring points: according to the gradient direction θ (x,y) , find the two neighboring points (x1,y1) and (x2,y2) of the pixel (x,y).

[0060] 3. Remove non-maximum points: If and Then keep the pixel (x, y); otherwise, (x,y) Set to 0.

[0061] For example, if the gradient direction is 0° (horizontal), the neighboring points of pixel point P are P1 = (x+1, y) and P2 = (x-1, y). and Then keep pixel P; otherwise, P Set to 0.

[0062] Then, according to the hysteresis threshold method, the high and low thresholds T are set. H and T L , compared with the gradient amplitude, is used to distinguish strong edge points, weak edge points and non-edge points:

[0063] G (x,y) >T H , strong edge point

[0064] {T L <G(x,y) ≤T H , weak edge point (5)

[0065] G (x,y) ≤T L , non-edge point

[0066] Keep strong edge points to reduce missed detection and false detection; discard non-edge points; for each weak edge point, check its neighboring points in the gradient direction. If the gradient value of the neighboring point is greater than T H , then keep the weak edge point; otherwise, discard the weak edge point.

[0067] This step ensures that each edge point is the most significant in the gradient direction to improve edge continuity and clarity.

[0068] Furthermore, the weight w of each edge point is calculated i .

[0069] Preferably, the gradient magnitude G is used i As the weight w i .

[0070] Use the weighted least squares method to fit the extracted edges to obtain a more accurate shape and position of the component. Set the fitting line equation to y = a x + b, and the optimization goal is:

[0071]

[0072] Among them, (x i ,y i ) is the coordinate of the ith edge point. S is the total number of edge points.

[0073] Figure 3 This is a multi-level template matching flow chart based on the improved pyramid, namely steps 5 to 6. As shown in the figure, it mainly includes the following steps:

[0074] Step A1: Generate an image pyramid.

[0075] Image pyramids of different resolutions are generated in each grid to help the algorithm adapt to the detection of target components of different scales. Each layer is obtained by applying Gaussian blur to the previous layer image and then downsampling it.

[0076] Preferably, bilinear interpolation is used for downsampling. The resolution of each layer of the pyramid is gradually reduced, and the resolution of each layer is generally set to half of the previous layer. This process is repeated multiple times until a predetermined minimum resolution is reached, which is generally set to 3-5 layers.

[0077] Step A2: Generate a template pyramid.

[0078] A template image (the same image as mentioned above) is extracted from a known component sample, and a template pyramid corresponding to the image pyramid is generated.

[0079] The template image also needs to go through the same processing flow as generating the image pyramid, that is, Gaussian blur and then downsampling to create multiple copies of the template at different resolutions, ensuring that each level in the template pyramid matches the corresponding level of the image pyramid in resolution.

[0080] The number of layers of the template pyramid should be consistent with the number of layers of the image pyramid.

[0081] Step A3: top-level matching.

[0082] Matching begins with images at the top (lowest resolution) level of the image pyramid.

[0083] Using the top template of the template pyramid, a rough match is performed in the top image of the image pyramid to quickly locate the possible target area. The best matching position is determined by calculating the similarity between the template and the image (such as the normalized cross-correlation coefficient).

[0084] Step A4: gradually match to the lower layers.

[0085] Based on the rough matching obtained at the lower resolution layers, we gradually move to higher resolution layers to improve the matching accuracy.

[0086] Several possible matching points are selected from the top-level matching results as candidates, and then a more refined search is performed in the next layer (higher resolution) image with these candidate points as the center.

[0087] In the process of gradually refining the matching results, preferably, a sliding window method is used to perform a local search around the candidate points to improve the matching accuracy.

[0088] In the process of gradually moving to higher resolution layers, the accuracy of the current matching results is dynamically evaluated. The specific steps are as follows:

[0089] 1. Calculate the matching accuracy of the current level: Use the defined matching accuracy evaluation index (such as the normalized mutual correlation coefficient) to measure the matching accuracy of the current level.

[0090] 2. Comparison and threshold: Compare the calculated matching accuracy evaluation index with the set threshold. If the matching accuracy of the current level has reached or exceeded the set threshold, stop further matching. If the matching accuracy of the current level does not reach the set threshold, continue to move to a higher resolution layer and repeat the above steps.

[0091] 3. Record matching results: record the matching results of the current level, including matching position, matching accuracy and other information.

[0092] Step A5: Mapping back to the coordinate system of the original image.

[0093] Map the intermediate-level matching results of each grid back to the coordinate system of the original image.

[0094] According to the pyramid construction rules, the scale factor from the current level to the original image is calculated, and the coordinates of the matching result are multiplied by the scale factor to obtain the coordinates in the original image, forming a local detection result.

[0095] It should be noted that since there may be some errors in the downsampling process, some corrections may be required when mapping back to the original image to ensure the accuracy of the coordinates.

[0096] Step A6: Merge the results.

[0097] The local detection results of all grids are merged to form a global detection result. The completeness and consistency of the detection results are ensured by merging. For each grid, the matching results within it are summarized. A voting mechanism is used to process the results from different grids to improve the robustness of the detection. When merging, preferably, a voting mechanism method is used to process the results from different grids to ensure the accuracy and consistency of the final result.

[0098] Step A7: Output the final detection result.

[0099] All matched and merged data are sorted out, and information such as the position, size, rotation angle, and matching accuracy evaluation index of the target component are output. The output form is determined according to the specific application requirements to ensure that the format and content of the output results meet the requirements of the actual application.

[0100] The above is only a preferred embodiment of the present invention and does not limit the present invention in any way. Any technician in the relevant technical field, without departing from the scope of the technical solution of the present invention, makes any form of equivalent replacement or modification to the technical solution and technical content disclosed in the present invention, which does not depart from the content of the technical solution of the present invention and still falls within the protection scope of the present invention.

Claims

1. A circuit board component detection method based on improved multi-target template matching, characterized in that: The following steps are involved: Collect image data of the circuit board to be tested; perform preprocessing operations on the circuit board image; Divide the image into multiple grids and perform edge detection and optimization in each grid; Generate image pyramids of different resolutions for multi-level template matching; Finally, the matching results of each grid are merged to form the final detection result.

2. The circuit board component detection method based on improved multi-target template matching according to claim 1 is characterized in that: The collected images are processed in sequence by grayscale conversion, detail enhancement and noise removal.

3. The circuit board component detection method based on improved multi-target template matching according to claim 1 is characterized in that: The preprocessed image is divided into multiple small grids, each grid is used as an independent processing unit, and the gray value of each grid is calculated to discard the area where the gray value is too different from the average gray value of the template.

4. The circuit board component detection method based on improved multi-target template matching according to claim 3 is characterized in that: Edge detection is performed in each grid, the Sobel operator is used to extract the edge, and the edge detection result is optimized by the hysteresis threshold method; finally, the weighted least squares method is used to perform straight line fitting on the extracted edge.

5. The circuit board component detection method based on improved multi-target template matching according to claim 3 is characterized in that: Image pyramids of different resolutions are generated within each grid, generating templates of different sizes.

6. The circuit board component detection method based on improved multi-target template matching according to claim 1, characterized in that: The intermediate-level matching results of each grid are mapped back to the coordinate system of the original image to form a global detection result; the detection results of all grids are merged to form the final detection result.

Images