Special-shaped element rapid identification positioning method and system, and storage medium

Abstract

This invention relates to the field of component identification and localization, specifically disclosing a method and system for rapid identification and localization of irregularly shaped components, as well as a storage medium. The method includes the following steps: clustering the features of the component to be detected according to its characteristic shape and position information to obtain one or more feature groups; calculating the number of pyramid layers and the downsampling coefficient of each pyramid layer, constructing virtual template images from multiple angles; using the virtual template images to perform matching scanning on the search image in a multi-layer pyramid form to obtain a coarse position and angle estimate of the component to be detected as a coarse localization result; based on the coarse localization result, sequentially performing pixel-level localization and size parameter scaling factor calculation, verification of the reliability of the localization result, and sub-pixel-level localization to adjust the center coordinates and angle of the component to be detected to obtain a fine localization result. This scheme has no requirements on the specific shape and structural distribution of the component, has good flexibility and versatility, and is fast in detection.

Description

Technical Field

[0001] This invention relates to the field of component identification and positioning technology, and specifically to a method and system for rapid identification and positioning of irregularly shaped components, as well as a storage medium. Background Technology

[0002] Pick and place machines are a key component in the automated production of circuit boards. Their main function is to automate the rapid placement of various surface mount components. Existing surface mount components come in a wide variety of packages, and irregularly shaped components, which differ from traditional Chip, SOP, and BGA packages, present a challenge for rapid identification due to their large size and shape variations, irregular feature distribution, and numerous subcategories.

[0003] There are currently four main common solutions for identifying irregularly shaped components:

[0004] The first approach is to design a separate recognition algorithm for each type of non-standard component. This approach can solve the problem of not being able to recognize components for devices with a small number or fixed types of components. However, this approach is not very practical, has low development efficiency, and can easily lead to software bloat and operational difficulties when there are many types of components.

[0005] The second approach involves standardizing certain features of the irregularly shaped component and then using identification methods for regular components to complete the identification. While this method does not require additional algorithmic overhead and has some practicality, it relies entirely on the operator's experience, demands high skill from the operator, and is not applicable to all components, resulting in poor versatility.

[0006] The third approach is to use deep learning to complete the recognition and detection. The advantage of this method is its universality, capable of handling the recognition of the vast majority of components. However, this method still suffers from problems such as high model resource consumption and difficulties in model training and actual updates.

[0007] The fourth method is to use image or shape template matching to complete the detection. The advantage of this method is that it can be applied to various scenarios and can guarantee speed and accuracy. The disadvantage is that if each component needs to store a template, it will occupy a lot of space resources. If the template is not stored, the operation is cumbersome if the image is imported to build the template before recognition. In addition, the center and angle of the image and shape template have certain initial values, which need to be obtained separately when the template is created. For the detection of the patch machine, the reliability and accuracy are insufficient.

[0008] In summary, existing methods are insufficient to meet the requirements of pick-and-place machines for rapid identification and positioning of irregularly shaped components. Summary of the Invention

[0009] The purpose of this invention is to overcome the problems existing in the prior art and provide a method and system for rapid identification and positioning of irregularly shaped components, as well as a storage medium, which can quickly and accurately estimate the pose of irregularly shaped patch components.

[0010] To achieve the above objectives, the first aspect of the present invention provides a method for rapid identification and positioning of irregularly shaped components, comprising the following steps:

[0011] Cluster the features of the element to be detected according to the feature shape and position information to obtain one or more feature groups. The information of each feature group includes feature size and position information.

[0012] The number of pyramid layers and the downsampling coefficient of each pyramid layer are calculated using the smallest size among all features. Virtual template images with multiple angles are constructed using the component feature parameters. The virtual template images are used to perform matching scans on the image to be searched in a multi-layer pyramid form to continuously narrow the detection range of the center and angle of the image to be searched, and obtain the rough position and angle estimate of the component to be detected as the coarse localization result.

[0013] Based on the coarse positioning results, pixel-level positioning and size parameter scaling factor calculation are performed sequentially, the reliability of the positioning results is verified, and the center coordinates and angle of the detected element are adjusted at the sub-pixel level to obtain the fine positioning results.

[0014] Preferably, the step of calculating the number of pyramid layers and the downsampling coefficient of each pyramid layer using the smallest size among all features, constructing virtual template images from multiple angles using component feature parameters, and using the virtual template images to perform matching scans on the search image in a multi-layer pyramid form includes:

[0015] The minimum value of the feature length and width in each feature group is calculated to obtain the downsampling coefficient zoomfactor0 of the top layer. Based on zoomfactor0, the number of pyramid layers and the image downsampling coefficients of other pyramid layers are determined.

[0016] Create virtual template images from multiple angles according to the set angle search range and search step size;

[0017] The image to be detected is downsampled by zoom factor 0 to obtain the top-level image to be searched. The multi-angle virtual template is downsampled by the same factor to generate a multi-angle top-level template image set.

[0018] On the top-level image to be searched, the top-level template image at each angle is used sequentially for scanning and matching, and the highest score of each matching and the corresponding scanning position are recorded.

[0019] The maximum value among the highest scores under each angle is counted, and the angle corresponding to the maximum value is taken as the top coarse positioning angle. The top coarse positioning coordinates are calculated based on the scanning position corresponding to the maximum value.

[0020] For the remaining layers of images other than the top layer, the ROI region of the current layer template is determined based on the matching result of the previous layer to obtain the minimum search range. The ROI region is downsampled to obtain the image to be searched. The template images within the specified range of the matching angle of the previous layer are downsampled at the same magnification to generate a multi-angle current layer template image set. The current layer template image set is used for scanning matching to obtain the coarse positioning angle and coarse positioning coordinates of the current layer.

[0021] Preferably, determining the number of pyramid layers and the image downsampling coefficients of each other pyramid layer based on zoomfactor0 includes:

[0022] If zoomfactor0 < 1, then zoomfactor0 = 1;

[0023] If zoomfactor0 < 2, the number of pyramid layers is 1, the pyramid form is not used, and the original image is used directly for coarse positioning.

[0024] If 5 > zoomfactor0 ≥ 2, the number of pyramid layers is 2, the downsampling coefficient of the bottom layer is set to 1, and the original image is used directly for coarse positioning without downsampling;

[0025] If 9 > zoomfactor0 ≥ 5, the number of pyramid layers is 3. The downsampling coefficient of the middle layer is set to half of zoomfactor0 and rounded down, and the downsampling coefficient of the bottom layer is set to 1. The original image is used directly for coarse positioning without downsampling.

[0026] If zoomfactor0 ≥ 9, the number of pyramid layers is 4. The downsampling coefficient of the bottom layer is set to 1. The downsampling coefficients of the top and bottom layers are evenly divided into four parts, and the middle two parts are rounded down to obtain the downsampling coefficients of the middle two layers of the pyramid.

[0027] Preferably, the step of calculating the minimum value of the feature length and width in each feature group to obtain the top-level downsampling coefficient zoomfactor0 is as follows:

[0028] The minimum value is converted into a pixel value using the camera scale, then divided by 1 to 5 and rounded down to obtain the downsampling coefficient zoomfactor0.

[0029] Preferably, the information of each feature group further includes the feature distribution pattern, feature group center, feature spacing, and feature quantity. The process of constructing a virtual template image from multiple angles using component feature parameters includes the following steps:

[0030] Based on the clustered feature groups, calculate the feature group center (FsCenterX, FsCenterY) of the current feature in the template image, the size parameter of the current feature, and the feature spacing of the feature group.

[0031] The center of each feature in the template image is calculated based on the feature distribution pattern, feature group center, feature spacing, and feature quantity.

[0032] If the current feature type is rectangle, then the minimum bounding rectangle of the current feature is obtained based on the feature size, feature center and feature angle in the template image, and the minimum bounding rectangle is used as the outer contour of the current feature;

[0033] If the current feature type is circular, then the outer contour of the current feature is a circle with the feature center in the template image as the center and the average length and width of the feature as the diameter.

[0034] Create a completely black background image based on the size of the template image. Then, using the calculated outer contours of each feature, change the pixel values ​​of the pixels inside the outer contours of each feature in the background image to 255 to complete the drawing of the virtual template image.

[0035] Preferably, the pixel-level positioning and size parameter scaling factor calculation specifically includes:

[0036] Using the coarse positioning angle as the reference angle, construct virtual template images from multiple angles within a specified range;

[0037] Using the coarsely located center coordinates as the reference center, within a specified range of pixels, template images of various angles are sequentially used to perform matching scans on the image to be identified to obtain the positioning result correction value.

[0038] The image to be detected is rotated using the angle of the positioning result correction value as the rotation angle and the center of the positioning result correction value as the rotation center, so that the components in the image are rotated to a horizontal state.

[0039] Using the angle of the positioning result correction value as the rotation angle and the center of the template image as the rotation center, the positioning template image corresponding to the positioning result correction value is rotated, the elements in the template image are rotated to a horizontal state, and the rotated template image is scaled and stretched within a specified range in both the horizontal and vertical directions to construct multiple scaled templates.

[0040] Different scaled template images in the horizontal direction are used sequentially to match the corrected position of the image to be recognized, the matching score is calculated, the maximum value is counted, and the scaling factor corresponding to the maximum value is used as the scaling factor in the horizontal direction; different scaled template images in the vertical direction are used sequentially to match the corrected position of the image to be recognized, the matching score is calculated, the maximum value is counted, and the scaling factor corresponding to the maximum value is used as the scaling factor in the vertical direction.

[0041] Using the angle of the corrected positioning result as the reference angle, virtual template images of multiple angles are constructed according to the scaling factors in the horizontal and vertical directions. Using the center coordinates of the corrected positioning result as the reference center, the template images of each angle are used sequentially to perform matching scans on the image to be identified to obtain pixel-level positioning coordinates and angles.

[0042] Preferably, the reliability verification of the positioning result specifically involves:

[0043] A virtual template image is constructed using pixel-level positioning angles and scaling factors in the horizontal and vertical directions. The Canny algorithm is used to process the virtual template image to obtain a template boundary image that retains only the feature boundary contours. The boundary is then widened using a dilation algorithm to obtain the final template boundary image.

[0044] Using the pixel-level positioning coordinate center as the ROI center and the length and width of the final template boundary image as the ROI length and width, the ROI image is extracted from the image to be detected. The extracted ROI image is processed using the Canny algorithm to obtain a contour image that retains only the boundary contour. The boundary is then widened using the dilation algorithm to obtain the component contour image.

[0045] Perform a logical AND operation between the final template boundary image and the component contour image; count the number of white pixels in the result image. If the number of white pixels is greater than a threshold, the recognition and positioning result is considered reliable; otherwise, the result is considered unreliable.

[0046] Preferably, the sub-pixel positioning adjusts the center coordinates and angle of the element to be detected, specifically as follows:

[0047] A virtual template image is constructed using pixel-level positioning angles and scaling factors in the horizontal and vertical directions. Using the pixel-level positioning coordinate center as a reference, a translational scan is performed on the search image to obtain multiple matching scores in the X direction. The coordinates in the X direction are used as the horizontal axis, and the multiple matching scores are used as the vertical axis. A quadratic function is fitted to calculate the maximum vertical axis point of the fitted curve within the horizontal axis range. The horizontal axis value corresponding to the maximum point is used as the sub-pixel positioning result in the horizontal direction.

[0048] A virtual template image is constructed using pixel-level positioning angles and scaling factors in the horizontal and vertical directions. Using the pixel-level positioning coordinate center as a reference, a translational scan is performed on the search image to obtain multiple matching scores in the Y direction. The Y-direction coordinates are used as the abscissa, and the multiple matching scores are used as the ordinates. A quadratic function is fitted, and the ordinate maxima of the fitted curve within the abscissa range are calculated. The abscissa value corresponding to the maxima is used as the sub-pixel positioning result in the vertical direction.

[0049] Using the pixel-level positioning angle as the baseline value and scaling factors in the horizontal and vertical directions, multiple virtual template images at different angles are constructed. Centered on the pixel-level coordinates obtained by rounding the sub-pixel positioning results, template matching is performed on the image to be searched using the virtual template images at various angles to obtain multiple matching scores. Then, using the virtual template angle value as the abscissa and the multiple matching scores as the ordinate, a quadratic curve fitting is performed. The maximum value of the ordinate of the fitted curve within the range of the abscissa is calculated, and the abscissa value corresponding to the maximum value is the sub-pixel positioning result of the angle.

[0050] A second aspect of the present invention provides a rapid identification and positioning system for irregularly shaped components, comprising:

[0051] The feature group clustering module is configured to cluster the features of the element to be detected according to the feature shape and position information to obtain one or more feature groups. The information of each feature group includes feature size and position information.

[0052] The coarse localization module is configured to calculate the number of pyramid layers and the downsampling coefficient of each pyramid layer using the smallest size among all features. It constructs virtual template images from multiple angles using the element feature parameters. The virtual template images are then used to perform matching scans on the image to be searched in a multi-layer pyramid form to continuously narrow down the detection range of the center and angle of the image to be searched, thereby obtaining a coarse position and angle estimate of the element to be detected as the coarse localization result.

[0053] The fine positioning module is configured to perform pixel-level positioning and size parameter scaling factor calculation, reliability verification of positioning results, and sub-pixel-level positioning and adjustment of the center coordinates and angle of the element under test based on the coarse positioning results, in order to obtain fine positioning results.

[0054] A third aspect of the present invention provides a computer storage medium on which machine-readable storage medium stores instructions for causing the machine to execute the above-described method for rapid identification and positioning of irregularly shaped components.

[0055] The above technical solution utilizes clustering to obtain component parameters, easily representing almost all components and resolving the challenge of uniformly describing heterogeneous components. Combining this with a multi-level pyramid and virtual template matching method completes component identification and localization. This solution is flexible and versatile, as it has no requirements regarding the specific shape and structural distribution of components. The use of artificially constructed virtual templates for feature matching solves the problem of calculating and ensuring the accuracy of component center and angle calculations in traditional algorithms that use images or contours for template matching, since the center and angle of the components are fixed during modeling. Furthermore, the multi-level pyramid scheme accelerates the algorithm, ensuring high detection speed. Therefore, the technical solution proposed in this invention has advantages such as high flexibility, strong versatility, fast detection speed, and high localization accuracy. Attached Figure Description

[0056] Figure 1 This is a schematic diagram of the process for rapid identification and positioning of irregularly shaped components disclosed in an embodiment of the present invention.

[0057] Figure 2 Images of the Micro-B_USB2.0 component in the embodiments and test examples disclosed in this invention;

[0058] Figure 3 This is a schematic diagram illustrating the identification features of the Micro-B_USB2.0 element in the embodiments and test examples disclosed in this invention;

[0059] Figure 4 This is a schematic diagram of a virtual template of the Micro-B_USB2.0 component in the embodiments and test examples disclosed in this invention. Detailed Implementation

[0060] The specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are for illustration and explanation only and are not intended to limit the scope of the present invention.

[0061] The following uses the identification of Micro-B_USB2.0 components in the identification of irregularly shaped components as an example to explain in detail the technical solution and specific implementation of the fast identification and positioning method for irregularly shaped components described in this invention, wherein the Micro-B_USB2.0 component is as follows: Figure 2 As shown.

[0062] The first aspect of this invention proposes a method for rapid identification and positioning of irregularly shaped components, referring to... Figure 1 As shown, it includes the following steps:

[0063] S1. Characterize component features using clustering. Cluster the features of the component to be detected according to their shape and location information. Group multiple features of the same type, uniform distribution, and size into a single feature group. Record information such as feature distribution patterns, feature group centers, feature spacing, number of features, and feature size. Use one or more feature groups to represent the complete component features for subsequent component identification and localization. Figure 3 As shown, six feature groups are used to represent Micro-B_USB2.0 components for subsequent template creation and location identification.

[0064] Further, step S1 includes the following processes:

[0065] S11. Feature Clustering. For the irregularly shaped component to be detected, based on the shape, size, and position information of the features, multiple features with the same shape, similar size, and uniform position are clustered into a feature group. Specifically, if a feature has no other features of the same size and type around it, this feature constitutes a feature group on its own. Following the above method, the features of the irregularly shaped component to be detected are clustered into multiple feature groups.

[0066] S12. Feature Group Parameter Calculation. For each feature group above, calculate the shape type, number of features, length and width dimensions of each feature, horizontal / vertical center distance between adjacent features, arrangement of feature groups, overall center position of feature groups, and minimum bounding rectangle of feature groups.

[0067] S13. Calculation of Component Feature Parameters. The overall size of the component to be detected is obtained using the minimum bounding rectangle of each feature group. The relative coordinates of the center of each feature group with respect to the upper left corner of the overall component size are calculated and used as the center position parameters of each feature group.

[0068] S2. Coarse Localization. A multi-layer pyramid structure combined with virtual template matching is used to obtain a coarse estimate of the position and angle of the component to be detected. First, the number of pyramid layers and the downsampling coefficient of each layer are calculated using the smallest size among all features. Then, virtual template images at multiple angles are constructed using the component's feature parameters. Finally, a multi-layer pyramid structure is used for coarse localization scanning, i.e., scanning layer by layer downwards from the top layer using multi-angle virtual template images, continuously narrowing the detection range of the center and angles, and finally obtaining the coarse localization result.

[0069] Further, step S2 includes the following processes:

[0070] S21. Calculation of Pyramid Levels and Downsampling Coefficients. First, loop through each feature group, calculate the minimum value of the feature length and width in all feature groups, and convert the minimum value to a pixel value using the camera scale. Then, divide the minimum pixel value by 1 to 5 (if the value is too small, the feature may be unclear after downsampling; if the value is too large, the downsampling coefficient is insufficient, the pyramid level is insufficient, and it cannot accelerate the process) and round down to obtain the downsampling coefficient zoomfactor0 for the top layer (highest layer). If zoomfactor0 < 1, then zoomfactor0 = 1. Then, obtain the pyramid level and the image downsampling coefficients for each other pyramid level according to the following rules:

[0071] If zoomfactor0 < 2, the number of pyramid layers is 1, meaning that the pyramid form is not used, and the original image is used directly for coarse positioning.

[0072] If 5 > zoomfactor0 ≥ 2, the number of pyramid layers is 2, and the downsampling coefficient of the second layer (bottom layer) is set to 1, that is, the original image is used and no downsampling is performed;

[0073] If 9 > zoomfactor0 ≥ 5, the number of pyramid layers is 3. The downsampling coefficient of the second layer (middle layer) is set to the downsampling coefficient of the first layer (top layer) divided by 2 and rounded down. The downsampling coefficient of the third layer (bottom layer) is set to 1, that is, the original image is used and no downsampling is performed.

[0074] If zoomfactor0 ≥ 9, the number of pyramid layers is 4. The downsampling coefficient of the fourth layer (bottom layer) is set to 1. The downsampling coefficients of the first layer (top layer) and the fourth layer (bottom layer) are evenly divided into four parts, and the middle two parts are rounded down to obtain the pyramid downsampling coefficients of the middle two layers.

[0075] S22. Multi-angle virtual template creation. Multi-angle virtual templates are created according to the set angle search range and search step size. In this embodiment, the set angle search range is ±30°, and the angle search step size is 1°, that is, starting from -30° and ending at 30°, a virtual template for the current angle is built every 1°. The specific creation process of the virtual template is as follows.

[0076] (1) Calculation of template image size. The formula is as follows:

[0077] ImgW = |W * cosθ| + |H * sinθ| + B (1)

[0078] ImgH = |W * sinθ| + |H * cosθ| + B (2)

[0079] Where B is the image boundary extension value, which is generally a positive even number greater than 2, and for example, the value is 4; ImgW, ImgHθ represents the horizontal and vertical dimensions of the virtual template image, respectively, in pixels; θ represents the angle of the element in the constructed virtual template image, in degrees; W and H represent the theoretical pixel values ​​of the overall length and width of the element at the current angle, respectively, in pixels. The formulas for calculating W and H are as follows:

[0080] .

[0081] BodyX and BodyY represent the overall dimensions of the component, in μm, while ScaleX and ScaleY represent the horizontal and vertical pixel dimensions of the camera, in μm / pixel.

[0082] (2) Calculation of the outer contour of the feature. The image center is used as the element center in the template image (centerX, centerY); the feature group center of the current feature (FsCenterX, FsCenterY) is calculated using the formula; the size parameters of the current feature are calculated using the formula; the feature spacing of the feature group is calculated using the formula; the center of each feature is calculated based on the feature distribution pattern, feature group center, feature spacing, and feature quantity. The minimum bounding rectangle of the feature is obtained using the feature size, feature center, and feature angle. If the feature type is rectangular, the minimum bounding rectangle of the feature is used as the outer contour of the feature; if the feature type is circular, a circle is drawn directly with the feature center and the average length and width as the diameter as the outer contour of the feature.

[0083] FsCenterX = centerX - middx (5)

[0084] FsCenterY = centerY - middy (6)

[0085] in, middx, middy These represent the offset of the feature group center relative to the element center at angle θ, in pixels, and are calculated using the following formula:

[0086] middx = Pdx * cosθ - Pdy * sinθ (7)

[0087] middy = Pdy * sinθ - Pdx * cosθ (8)

[0088] in, Pdx, Pdy These represent the pixel value offset of the feature group center relative to the element center at 0°, in pixels, and are calculated using the following formula:

[0089]

[0090] in, dx, dyThese represent the millimeter-level offset of the feature group center relative to the element center at 0°, in μm. dx = -FeatureCenterX + BodyX / 2, dy = FeatureCenterY - BodyY / 2. FeatureCenterX and FeatureCenterY represent the position parameters of the feature group center, in mm, which is the offset of the feature group center relative to the upper left corner of the entire element.

[0091]

[0092] in, w, h These represent the length and width of the feature, respectively, in pixels. FW, FH These represent the lengths of the feature in the horizontal and vertical directions at 0°, respectively, in μm.

[0093] in, PE, PE These represent the center-to-center distance between adjacent features in a feature group, in pixels. FPE, FPN These represent the center-to-center distances of adjacent features in the horizontal and vertical directions within the feature group at 0°, respectively, in μm.

[0094] (3) Virtual template image drawing. First, create a completely black background image using the calculated image size; then, using the calculated outer contours of each feature, change the pixel values ​​of the pixels located inside each outer contour of the background image to 255 to complete the virtual template image drawing, such as... Figure 4 As shown.

[0095] S23. First layer (top layer) template matching. First, the image to be detected is downsampled by zoom factor 0 to obtain the top layer search image; then, the virtual template images created at multiple angles are downsampled by the same factor to generate a set of top layer template images at multiple angles; then, on the top layer search image, the top layer template images at each angle are used for scanning and matching in sequence, and the highest score and the corresponding scanning position of each matching are recorded. The calculation formula for the scanning score is as shown in formula (15); finally, the maximum value of the highest score at each angle is counted, and the angle corresponding to the maximum value is taken as the top layer coarse positioning angle. The top layer coarse positioning coordinates are obtained by calculating the scanning position corresponding to the maximum value through formulas (16) and (17).

[0096]

[0097] in, R(x, y) Indicates the coordinates of the image I to be searched. (x, y) Positional matching score T(x ’ ,y’ ) Indicates that the template image T is in (x ’ ,y ’ ) The gray value of the coordinate position, I( x + x ’ , y + y ’ ) indicates that the image I to be searched is in ( x + x ’ , y + y ’ The gray value at coordinate position, where M represents the number of non-zero pixels in the template image T.

[0098]

[0099] in, Cx, Cy ... Tw, Th These represent the width and height of the downsampled template image that achieved the highest matching score, respectively. Zoomfactor is the downsampling factor for this layer.

[0100] S24. Remaining layer template matching. First, the search range is narrowed using the results of the previous layer template matching. Specifically, the ROI of the current layer template matching is calculated using formulas (18), (19), (20), and (21), and the search image is cropped. Then, the cropped search image is downsampled to obtain a new image to be searched. Next, the template images within the range of ±5° of the previous layer matching angle are downsampled by the same factor to obtain a new set of template images. Finally, on the new image to be searched, the new template images are used sequentially for scanning and matching. The highest score and the corresponding scanning position of each matching are recorded. The maximum value of the highest score at each angle is counted. The angle corresponding to the maximum value is taken as the coarse positioning angle of this template matching. The coarse positioning coordinates of this template matching are obtained by calculating the scanning position corresponding to the maximum value using formulas (16) and (17).

[0101]

[0102] in, Rectw, Recth, Rectx, Recty These represent the width, height, top-left corner x-coordinate, and top-left corner y-coordinate of the ROI matched by the current layer template, respectively, in pixels. pTw, pTh These represent the width and height of the template with the highest score in the previous layer's template matching, respectively, in pixels. Pzoomfactor represents the downsampling coefficient of the previous layer. pCx, pCy This indicates the center coordinates of the coarse positioning layer above.

[0103] S3. Pixel-level Recognition and Positioning. Using the coarse positioning result as a baseline, the angle and position coordinates are further precisely identified within a small area. First, using the coarse positioning result as a baseline, template matching is performed within a small area to obtain a correction value for the positioning result. Then, using the correction value as a baseline, scaling templates with different scaling ratios in the horizontal and vertical directions are constructed, and the scaling factor is determined. Finally, using the correction value as a baseline, the scaling templates are constructed within a small area for template matching to obtain a more accurate positioning result.

[0104] Furthermore, step S3 includes the following processes:

[0105] S31. Localization Result Correction. First, using the coarse localization angle as the reference angle, virtual template images of multiple angles are constructed within a range of ±3° with a step size of 0.5°. Then, using the center coordinates of the coarse localization as the reference center, within a range of ±5 pixels, the template images of each angle are sequentially used for matching and scanning on the image to be recognized to obtain the localization result correction value.

[0106] S32, Component parameter correction.

[0107] The image to be detected is rotated using the angle of the localization result correction value as the rotation angle and the center of the localization result correction value as the rotation center, so that the components in the image are rotated to a horizontal state. The localization template image with the highest score in S31 is rotated using the angle of the localization result correction value as the rotation angle and the center of the template image as the rotation center.

[0108] Multiple scaled templates are constructed by scaling and stretching the rotated template image in both the horizontal and vertical directions, with a scaling range of ±5% and a scaling step of 0.5%.

[0109] Different scaled template images in the horizontal direction are sequentially used to match the corrected positions of the image to be recognized, and the matching score is calculated. The maximum value is then recorded, and the scaling factor corresponding to the maximum value is used as the scaling factor in the horizontal direction. Similarly, different scaled template images in the vertical direction are sequentially used to match the corrected positions of the image to be recognized, and the matching score is calculated. The maximum value is then recorded, and the scaling factor corresponding to the maximum value is used as the scaling factor in the vertical direction.

[0110] S33, Pixel-level positioning. Using the corrected angle as the base angle, within its ±1° range, virtual template images at multiple angles are constructed with scaling factors in the horizontal and vertical directions, using a step size of 0.05°. Using the corrected center coordinates as the base center, within its ±2 pixel range, the template images at each angle are sequentially used for matching and scanning on the image to be recognized to obtain more accurate positioning coordinates and angles.

[0111] S4. Verification of Location Result. Since template matching always outputs a result, it's difficult to determine the validity of the recognition result. Therefore, a location result verification module is needed. In this invention, pixel-level location results are used to verify the location result by checking the overlap between the virtual template boundary and the actual feature boundary.

[0112] Furthermore, step S4 includes the following processes:

[0113] S41. Template Boundary Image Creation. First, a virtual template image is constructed using the precise angles and scaling factors in the horizontal and vertical directions obtained during the pixel-level positioning stage. Then, the Canny algorithm is used to process the virtual template image to obtain a template boundary image that retains only the feature boundary contours. Finally, the boundary is widened to 3 pixels using a dilation algorithm to obtain the final template boundary image.

[0114] S42. Component Contour Image Creation. First, using the precise center obtained in the pixel-level localization stage as the ROI center, and the length and width of the constructed template boundary image as the length and width of the ROI, the ROI image is extracted from the image to be detected. Then, the ROI image is processed using the Canny algorithm to obtain a contour image that retains only the boundary contour. Finally, the boundary is widened to 3 pixels using the dilation algorithm to obtain the component contour image.

[0115] S43. Verification of Localization Results. Perform a logical AND operation between the template boundary image and the contour image; count the number of white pixels in the result image. If the number of white pixels is greater than a threshold, the localization result is considered reliable; otherwise, the result is considered unreliable. The threshold is based on 0.8 times the number of white pixels in the template boundary image.

[0116] S5. Subpixel positioning. The center coordinates X, Y, and angle θ are treated as independent variables. Subpixel coordinates and more accurate angles are obtained by solving a curve polynomial to fit the extreme values.

[0117] Furthermore, step S5 includes the following processes:

[0118] S51. Calculation of Sub-pixel Localization Results in the Horizontal Direction. First, a virtual template image is constructed using the precise angles and scaling factors in the horizontal and vertical directions obtained during the pixel-level localization stage. Then, using the center of the pixel-level localization result as a reference, a translational scan is performed on the search image within a range of 2 pixels to the left and right, obtaining 5 matching scores in the X direction. Next, a quadratic function is fitted using the X-axis coordinates as the x-axis and the 5 matching scores as the y-axis. Finally, the maximum y-axis value of the fitted curve within the x-axis range is calculated, and the x-axis value corresponding to the maximum value is used as the sub-pixel localization result in the horizontal direction.

[0119] S52. Calculation of Sub-pixel Localization Results in the Vertical Direction. First, a virtual template image is constructed using the precise angles and scaling factors in the horizontal and vertical directions obtained during the pixel-level localization stage. Then, using the center of the pixel-level localization result as a reference, a translational scan is performed on the search image within a range of 2 pixels vertically to obtain 5 matching scores in the Y direction. Next, a quadratic function is fitted using the Y-axis coordinates as the abscissa and the 5 matching scores as the ordinates. Finally, the maximum ordinate point of the fitted curve within the abscissa range is calculated, and the abscissa value corresponding to the maximum point is used as the sub-pixel localization result in the vertical direction.

[0120] S53. Angle Calculation. First, using the precise angle obtained in the pixel-level positioning stage as the baseline value, within a range of ±0.1°, and with a step size of 0.05°, five virtual template images at different angles are constructed using the scaling factors in the horizontal and vertical directions obtained in the pixel-level positioning stage. Then, using the pixel-level coordinates obtained by rounding the sub-pixel positioning results as the center, template matching is performed on the image to be searched using the template images at each angle, obtaining five matching scores. Next, a quadratic function is fitted using the template angle value as the abscissa and the five matching scores as the ordinate. Finally, the maximum value of the ordinate of the fitted curve within the range of the abscissa is calculated, and the abscissa value corresponding to the maximum value is used as the more accurate angle result.

[0121] Based on the same inventive concept, a second aspect of the present invention provides a rapid identification and positioning system for irregularly shaped components, comprising:

[0122] The feature group clustering module is configured to cluster the features of the element to be detected according to the feature shape and position information to obtain one or more feature groups. The information of each feature group includes feature size and position information.

[0123] The coarse localization module is configured to calculate the number of pyramid layers and the downsampling coefficient of each pyramid layer using the smallest size among all features. It constructs virtual template images from multiple angles using the element feature parameters. The virtual template images are then used to perform matching scans on the image to be searched in a multi-layer pyramid form to continuously narrow down the detection range of the center and angle of the image to be searched, thereby obtaining a coarse position and angle estimate of the element to be detected as the coarse localization result.

[0124] The fine positioning module is configured to perform pixel-level positioning and size parameter scaling factor calculation, reliability verification of positioning results, and sub-pixel-level positioning and adjustment of the center coordinates and angle of the element under test based on the coarse positioning results, in order to obtain fine positioning results.

[0125] Based on the same inventive concept, a third aspect of the present invention provides a computer storage medium, comprising:

[0126] A memory on which computer programs are stored;

[0127] A processor is used to execute the computer program in the memory to implement the steps of the above-described method for rapid identification and positioning of irregularly shaped components.

[0128] In summary, the technical solution of this invention utilizes clustering to obtain and characterize component parameters, easily representing almost all components and readily solving the problem of uniformly describing heterogeneous components. Combined with virtual template matching, component identification and localization are achieved. This solution has no requirements regarding the specific shape and structural distribution of components, making it highly flexible and versatile. The use of artificially constructed virtual templates for feature matching solves the problem of calculating and ensuring the accuracy of component center and angle calculations in traditional algorithms that use images or contours for template matching, as the center and angle of the components are fixed during modeling. Furthermore, this method employs multi-level localization methods—coarse localization, pixel-level localization, and sub-pixel localization—ensuring both high accuracy and precision while accelerating localization speed. Additionally, the multi-level pyramid scheme accelerates the algorithm, ensuring excellent detection speed. Therefore, the technical solution proposed in this invention offers advantages such as high flexibility, strong versatility, fast detection speed, and high localization accuracy.

[0129] Test example:

[0130] Verification and experimental analysis of the reliability and effectiveness of a rapid identification and location method for irregularly shaped components.

[0131] To verify the reliability and effectiveness of the method proposed in this invention, the following verification experiment was designed:

[0132] The experiment used images of Micro-B USB 2.0 components captured by a YAMAHA YS12 pick-and-place machine as sample images, and the recognition result of the YAMAHA YS12 pick-and-place machine as the standard value. The algorithm proposed in this invention was used to recognize the sample images, and the difference between the recognition result of the proposed algorithm and the standard value was used as the evaluation value of the experimental result.

[0133] The experiment used a non-standard Micro-B_USB2.0 component for testing. The Micro-B_USB2.0 component is shown below. Figure 1 As shown, its dimensions are 12.350mm x 9.425mm. Fifty images were captured during the experiment, each with a size of 2048 pixels x 573 pixels. The algorithm proposed in this invention was used to identify the 50 images of the Micro-B_USB2.0 component.

[0134] The experimental test results are shown in Table 1 below. Table 1 shows that for Micro-B USB 2.0 components, the algorithm proposed in this invention has a maximum center point deviation of no more than 0.74 pixels and a maximum angle deviation of no more than 0.49° compared to the standard results. This demonstrates the high accuracy of the algorithm proposed in this invention. Table 1 also shows that the standard deviations of the center point deviation and angle deviation for Micro-B USB 2.0 component identification by the proposed algorithm are both less than 0.3, indicating strong stability of the algorithm. Furthermore, Table 1 shows that the average time for Micro-B USB 2.0 component identification by the proposed algorithm is only 41.19 ms, indicating fast identification speed and good real-time performance.

[0135] Table 1. Statistical Table of Micro-B_USB2.0 Component Identification Results

[0136]

[0137] In Table 1, ΔX, ΔY, and ΔR represent the differences between the center point x, center point y, and rotation angle R of the algorithm recognition results and the reference standard values, respectively.

[0138] The preferred embodiments of the present invention have been described in detail above with reference to the accompanying drawings; however, the present invention is not limited thereto. Within the scope of the inventive concept, various simple modifications can be made to the technical solutions of the present invention, including combinations of various specific technical features in any suitable manner. To avoid unnecessary repetition, the present invention will not describe the various possible combinations separately. However, these simple modifications and combinations should also be considered as the content disclosed in the present invention and are all within the protection scope of the present invention.

Claims

1. A method for rapid identification and positioning of irregularly shaped components, characterized in that, Includes the following steps: Cluster the features of the element to be detected according to the feature shape and position information to obtain one or more feature groups. The information of each feature group includes feature size and position information. The number of pyramid layers and the downsampling coefficient of each layer are calculated using the smallest size among all features. Virtual template images at multiple angles are constructed using the element feature parameters. These virtual template images are then used to perform a multi-layered pyramid-shaped matching scan of the search image to continuously narrow down the detection range of the center and angles of the search image, obtaining a coarse position and angle estimate of the element to be detected as the coarse localization result. This includes: The minimum values ​​of feature length and width in each feature group are calculated to obtain the top-level downsampling coefficient zoomfactor0. Based on zoomfactor0, the number of pyramid layers and the image downsampling coefficients of other pyramid layers are determined. Multi-angle virtual template images are created according to the set angle search range and search step size. The image to be detected is downsampled by zoomfactor0 to obtain the top-level search image. The created multi-angle virtual templates are downsampled by the same factor to generate a multi-angle top-level template image set. On the top-level search image, the top-level template images at each angle are used for scanning and matching in sequence, and the highest score and corresponding scanning position of each matching are recorded. The maximum value of the highest score at each angle is calculated, and the angle corresponding to the maximum value is taken as the top-level coarse localization angle. The top-level coarse localization coordinates are calculated based on the scanning position corresponding to the maximum value. Based on the coarse positioning results, pixel-level positioning and size parameter scaling factor calculation are performed sequentially, the reliability of the positioning results is verified, and the center coordinates and angle of the detected element are adjusted at the sub-pixel level to obtain the fine positioning results.

2. The method according to claim 1, characterized in that, The process involves calculating the number of pyramid layers and the downsampling coefficient of each layer using the smallest size among all features, constructing virtual template images from multiple angles using component feature parameters, and then using these virtual template images to perform a multi-layered pyramid-shaped matching scan of the image to be searched, including: For the remaining layers of images other than the top layer, the ROI region of the current layer template is determined based on the matching result of the previous layer to obtain the minimum search range. The ROI region is downsampled to obtain the image to be searched. The template images within the specified range of the matching angle of the previous layer are downsampled at the same magnification to generate a multi-angle current layer template image set. The current layer template image set is used for scanning matching to obtain the coarse positioning angle and coarse positioning coordinates of the current layer.

3. The method according to claim 2, characterized in that, The determination of the pyramid layer number and other image downsampling coefficients for each pyramid layer based on zoomfactor0 includes: If zoomfactor0 < 1, then zoomfactor0 = 1; If zoomfactor0 < 2, the number of pyramid layers is 1, the pyramid form is not used, and the original image is used directly for coarse positioning. If 5 > zoomfactor0 ≥ 2, the number of pyramid layers is 2, the downsampling coefficient of the bottom layer is set to 1, and the original image is used directly for coarse positioning without downsampling; If 9 > zoomfactor0 ≥ 5, the number of pyramid layers is 3. The downsampling coefficient of the middle layer is set to half of zoomfactor0 and rounded down, and the downsampling coefficient of the bottom layer is set to 1. The original image is used directly for coarse positioning without downsampling. If zoomfactor0 ≥ 9, the number of pyramid layers is 4. The downsampling coefficient of the bottom layer is set to 1. The downsampling coefficients of the top and bottom layers are evenly divided into four parts, and the middle two parts are rounded down to obtain the downsampling coefficients of the middle two layers of the pyramid.

4. The method according to claim 2, characterized in that, The minimum value of the feature length and width in each feature group is used to obtain the top-level downsampling coefficient zoomfactor0, specifically: The minimum value is converted into a pixel value using the camera scale, then divided by 1 to 5 and rounded down to obtain the downsampling coefficient zoomfactor0.

5. The method according to claim 2, characterized in that, The information of each feature group also includes the feature distribution pattern, feature group center, feature spacing, and feature quantity. The process of constructing virtual template images from multiple angles using component feature parameters includes the following steps: Based on the clustered feature groups, calculate the feature group center (FsCenterX, FsCenterY) of the current feature in the template image, the size parameter of the current feature, and the feature spacing of the feature group. The center of each feature in the template image is calculated based on the feature distribution pattern, feature group center, feature spacing, and feature quantity. If the current feature type is rectangle, then the minimum bounding rectangle of the current feature is obtained based on the feature size, feature center and feature angle in the template image, and the minimum bounding rectangle is used as the outer contour of the current feature; If the current feature type is circular, then the outer contour of the current feature is a circle with the feature center in the template image as the center and the average length and width of the feature as the diameter. Create a completely black background image based on the size of the template image. Then, using the calculated outer contours of each feature, change the pixel values ​​of the pixels inside the outer contours of each feature in the background image to 255 to complete the drawing of the virtual template image.

6. The method according to any one of claims 1-5, characterized in that, The pixel-level positioning and size parameter scaling factor calculation are specifically as follows: Using the coarse positioning angle as the reference angle, construct virtual template images from multiple angles within a specified range; Using the coarsely located center coordinates as the reference center, within a specified range of pixels, template images of various angles are sequentially used to perform matching scans on the image to be identified to obtain the positioning result correction value. The image to be detected is rotated using the angle of the positioning result correction value as the rotation angle and the center of the positioning result correction value as the rotation center, so that the components in the image are rotated to a horizontal state. Using the angle of the positioning result correction value as the rotation angle and the center of the template image as the rotation center, the positioning template image corresponding to the positioning result correction value is rotated, the elements in the template image are rotated to a horizontal state, and the rotated template image is scaled and stretched within a specified range in both the horizontal and vertical directions to construct multiple scaled templates. Different scaled template images in the horizontal direction are used sequentially to match the corrected position of the image to be recognized, the matching score is calculated, the maximum value is counted, and the scaling factor corresponding to the maximum value is used as the scaling factor in the horizontal direction. Different scaled template images in the vertical direction are used sequentially to match the corrected position of the image to be recognized, the matching score is calculated, the maximum value is counted, and the scaling factor corresponding to the maximum value is used as the scaling factor in the vertical direction. Using the angle of the corrected positioning result as the reference angle, virtual template images of multiple angles are constructed according to the scaling factors in the horizontal and vertical directions. Using the center coordinates of the corrected positioning result as the reference center, the template images of each angle are used sequentially to perform matching scans on the image to be identified to obtain pixel-level positioning coordinates and angles.

7. The method according to claim 6, characterized in that, The reliability of the positioning result is verified as follows: A virtual template image is constructed using pixel-level positioning angles and scaling factors in the horizontal and vertical directions. The Canny algorithm is used to process the virtual template image to obtain a template boundary image that retains only the feature boundary contours. The boundary is then widened using a dilation algorithm to obtain the final template boundary image. Using the pixel-level positioning coordinate center as the ROI center and the length and width of the final template boundary image as the ROI length and width, the ROI image is extracted from the image to be detected. The extracted ROI image is processed using the Canny algorithm to obtain a contour image that retains only the boundary contour. The boundary is then widened using the dilation algorithm to obtain the component contour image. Perform a logical AND operation between the final template boundary image and the component contour image; count the number of white pixels in the result image. If the number of white pixels is greater than a threshold, the recognition and positioning result is considered reliable; otherwise, the result is considered unreliable.

8. The method according to claim 7, characterized in that, The sub-pixel positioning adjustment of the center coordinates and angle of the element to be detected is specifically as follows: A virtual template image is constructed using pixel-level positioning angles and scaling factors in the horizontal and vertical directions. Using the pixel-level positioning coordinate center as a reference, a translational scan is performed on the search image to obtain multiple matching scores in the X direction. The coordinates in the X direction are used as the horizontal axis, and the multiple matching scores are used as the vertical axis. A quadratic function is fitted to calculate the maximum vertical axis point of the fitted curve within the horizontal axis range. The horizontal axis value corresponding to the maximum point is used as the sub-pixel positioning result in the horizontal direction. A virtual template image is constructed using pixel-level positioning angles and scaling factors in the horizontal and vertical directions. Using the pixel-level positioning coordinate center as a reference, a translational scan is performed on the search image to obtain multiple matching scores in the Y direction. The Y-direction coordinates are used as the abscissa and the multiple matching scores are used as the ordinates. A quadratic function is fitted to calculate the maximum ordinate point of the fitted curve within the abscissa range. The abscissa value corresponding to the maximum point is used as the sub-pixel positioning result in the vertical direction. Using the pixel-level positioning angle as the baseline value and scaling factors in the horizontal and vertical directions, multiple virtual template images at different angles are constructed. Centered on the pixel-level coordinates obtained by rounding the sub-pixel positioning results, template matching is performed on the image to be searched using the virtual template images at various angles to obtain multiple matching scores. Then, using the virtual template angle value as the abscissa and the multiple matching scores as the ordinate, a quadratic curve fitting is performed. The maximum value of the ordinate of the fitted curve within the range of the abscissa is calculated, and the abscissa value corresponding to the maximum value is the sub-pixel positioning result of the angle.

9. A rapid identification and positioning system for irregularly shaped components, characterized in that, include: The feature group clustering module is configured to cluster the features of the element to be detected according to the feature shape and position information to obtain one or more feature groups. The information of each feature group includes feature size and position information. The coarse localization module is configured to calculate the number of pyramid layers and the downsampling coefficients of each pyramid layer using the minimum size among all features. It constructs virtual template images from multiple angles using element feature parameters. These virtual template images are then used to perform a multi-layered pyramid-shaped matching scan of the search image, continuously narrowing the detection range of the center and angles of the search image. This yields a coarse estimate of the position and angle of the detected element as the coarse localization result. This includes: calculating the minimum value of the feature length and width in each feature group to obtain the downsampling coefficient zoomfactor0 for the top layer; determining the number of pyramid layers and the images of other pyramid layers based on zoomfactor0. Downsampling coefficient; create multi-angle virtual template images according to the set angle search range and search step size; perform zoom factor 0 downsampling on the image to be detected to obtain the top-level search image, and perform downsampling on the created multi-angle virtual templates at the same factor to generate a multi-angle top-level template image set; on the top-level search image, use the top-level template images of each angle to perform scanning matching in sequence, record the highest score of each matching and the corresponding scanning position; calculate the maximum value among the highest scores under each angle, take the angle corresponding to the maximum value as the top-level coarse positioning angle, and calculate the top-level coarse positioning coordinates based on the scanning position corresponding to the maximum value; The fine positioning module is configured to perform pixel-level positioning and size parameter scaling factor calculation, reliability verification of positioning results, and sub-pixel-level positioning and adjustment of the center coordinates and angle of the element under test based on the coarse positioning results, in order to obtain fine positioning results.

10. A machine-readable storage medium storing instructions for causing a machine to perform the rapid identification and positioning method for irregularly shaped components as described in any one of claims 1-8.

Images