A general visual inspection feature method

Abstract

The application relates to a general visual detection feature method and relates to the technical field of image recognition; the method comprises the following steps: S10, extraction of self-defined features of elements, including the step of pre-processing an image ImageSrc1 to obtain an image ImageSrc2, and features Featuress that need to be checked by the elements are extracted through the step S10; and S20, checking of the self-defined features of the elements. The application can avoid poor product quality and make the machine more intelligent.

Description

Technical Field

[0001] This invention relates to the field of image recognition technology, and more specifically, to a general visual feature detection method. Background Technology

[0002] With the development of the electronics manufacturing industry, surface mount technology (SMT) is advancing rapidly. The positioning accuracy and speed of SMT components are crucial indicators affecting the performance of SMT machines. Computer-based electronic component positioning and inspection methods are characterized by high speed, high precision, and intelligence, not only increasing production flexibility and automation but also significantly improving the intelligence and versatility of production processes.

[0003] Traditional computer vision positioning methods for electronic components in pick-and-place machines include template matching, but it is difficult to inspect the quality of components. The industry standard is to classify components into regular rectangular components, components with lead features, or components with spherical features. If a component has multiple features, it usually requires the development of specific algorithms, which is time-consuming. Summary of the Invention

[0004] To overcome the shortcomings of existing technologies, this invention provides a universal visual inspection feature method that can avoid poor production quality and make machines more intelligent.

[0005] The technical solution adopted by this invention to solve its technical problem is: a general visual detection feature method, the improvement of which includes the following steps:

[0006] S10, Extraction of custom features of the component, including the step of preprocessing the original image ImageSrc1 to obtain the preprocessed image ImageSrc2, and extracting the feature set of the component to be checked through step S10;

[0007] S20, Checking the custom features of the component, step S20 includes:

[0008] S201, Obtain the feature region Rect2 of the component;

[0009] S202. Perform coarse cropping on the preprocessed image ImageSrc2 to obtain the region Rect4 where the coarse cropped image is located;

[0010] S203. Fill the length and width directions of the preprocessed image ImageSrc2 and the region Rect4 where the coarse cropped image is located with pixels to obtain the region Rect5 where the coarse cropped image and the filled image are located respectively.

[0011] In S204, the preprocessed image ImageSrc2 is cropped at the region Rect5 where the padded image is located, to obtain a coarsely cropped image ImageSrc3; the coarsely cropped image ImageSrc3 is rotated according to the angle of the region Rect3 of the coarsely cropped image, to obtain a rotated image ImageSrc4 after coarse cropping, and the image ImageSrc4 is further subjected to center cropping according to the size of Rect3, to obtain an image ImageSrc5 that has been coarsely cropped, rotated, and then center-cropped;

[0012] In S205, the image ImageSrc5 is cropped at the region Rect2 where the feature of the component is located according to the feature extracted in S10, to obtain a binarized feature image ImageFeature2;

[0013] In S206, the image ImageFeature2 is processed to obtain a set Contours of all target contours in the image, and noise point contours with fewer than 10 contour points among all contours are filtered out;

[0014] In S207, an initial minimum center distance MinDistance=9999 is set, and for each feature Feature in Features, the position of the Feature is compared with the center distance of each contour in Contours to find the one with the smallest distance, and the area ratio Rate of the contour is compared at the same time,

[0015] Rate=Min(Feature.Area, Contour.Area) / Max(Feature.Area, Contour.Area), Contours with Rate<0.5 are skipped; and the contour center distance Distance is calculated at the same time,

[0016] If Distance<MinDistance, MinDistance is updated to Distance; where Feature.Area is the area of the feature, Contour.Area is the area of the contour, Feature.center.x is the x-value of the feature center, Feature.center.y is the y-value of the feature center, contour.center.x is the x-value of the contour center, and contour.center.y is the y-value of the contour center;

[0017] In S208, the maximum range of the target offset is set

[0018] max_dis = (Feature.Size.Width + Feature.Size.Height) / 2. If Distance ≤ max_dis, then the contour area is determined, and the feature score is...

[0019] Score = min(Feature.Area, Contour.Area) / Max(Feature.Area, Contour.Area). If the Score is greater than the set acceptance level, the feature check passes; otherwise, the feature check fails.

[0020] If Distance > max_dis, then the feature check is directly deemed to have failed; where Feature.Size.Width is the width of the feature, Feature.Size.Height is the height of the feature; Feature.Area is the area of ​​the feature, and Contour.Area is the area of ​​the contour.

[0021] S209. Repeat the operations from S205 to S208 until all feature checks are completed. The process is complete only if all feature checks pass. If any feature check fails, it is determined to be a problematic component.

[0022] In step S10 of the above technical solution, the image preprocessing includes S101: performing binarization processing on ImageSrc1 using automatic thresholding or adjusting an appropriate threshold, and using morphological opening operation to filter out interference points to obtain ImageSrc2.

[0023] Step S10 in the above technical solution further includes:

[0024] S102. Find all contours in the image ImageSrc2, treat all contour points as a whole, and obtain the region Rect1 where the smallest bounding rectangle is located. Calculate the center and angle.

[0025] S103. ImageSrc1 is subjected to affine transformation based on the center and angle calculated by Rect1 to correct it back to the image center. At the same time, it is cropped according to the size of Rect1 to obtain the template image ImageModel.

[0026] S104. Find Rect2 in the template image ImageModel and crop it again to obtain the feature image ImageFeature1;

[0027] S105. Binarize the feature image ImageFeature1, filter out interference points, remove burrs from the contours, find all contours in the image, and select the feature type for that range.

[0028] S106. Manually remove interference points that do not need to be checked and save the set of features that need to be checked.

[0029] S107. Repeat steps S104 to S106 until all features that need to be checked have been added.

[0030] In step S102 of the above technical solution, contours with fewer than 20 contour points are filtered out.

[0031] In step S105 of the above technical solution, the image ImageFeature1 is binarized using an automatic threshold or by adjusting a suitable threshold, morphological opening operations are used to filter out interference points, and closing operations are used to remove burrs from the contour.

[0032] In step S105 of the above technical solution, the feature types are rectangle and circle.

[0033] In step S201 of the above technical solution, the feature region position Rect2 of the element is obtained by template matching.

[0034] In step S202 of the above technical solution, the size of the image ImageSrc2 that needs to be coarsely cropped is calculated based on Rect3;

[0035] The length and width of Rect4 are Length, the center position of Rect4 is the center position of Rect3, and the angle of Rect4 is 0 degrees.

[0036] In step S202 of the above technical solution, the image ImageSrc2 is filled with 500 pixels in both its length and width directions; and the image Rect4 is filled with 250 pixels in both its length and width directions.

[0037] In step S206 of the above technical solution, the threshold used in step S105 is used to binarize the image ImageFeature2, morphological opening operation is used to filter out interference points, and closing operation is used to remove contour burrs.

[0038] The beneficial effects of the present invention are: the general visual inspection feature method provided by the present invention first extracts custom features of the components, and then checks the custom features of the components, which can avoid poor production quality and make the machine more intelligent. Attached Figure Description

[0039] Figure 1 This is a flowchart of a general visual feature detection method according to the present invention. Detailed Implementation

[0040] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0041] The following will clearly and completely describe the concept, specific structure, and technical effects of the present invention in conjunction with embodiments and accompanying drawings, so as to fully understand the purpose, features, and effects of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, not all of them. Other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are all within the scope of protection of the present invention. Furthermore, all connections / linkages involved in the patent do not simply refer to direct contact between components, but rather to the ability to form a better connection structure by adding or reducing connecting accessories according to specific implementation conditions. The various technical features in this invention can be combined interactively without contradicting each other.

[0042] Reference Figure 1 This invention provides a general visual feature detection method, comprising the following steps:

[0043] S10, Extraction of custom features of the element, including the step of preprocessing the original image ImageSrc1 to obtain a preprocessed image ImageSrc2. The preprocessing of the image includes step S101: performing binarization processing on the image ImageSrc1 using automatic thresholding or adjusting an appropriate threshold and filtering out interference points using morphological opening operation to obtain the image ImageSrc2. The feature set of the element to be checked is extracted through step S10.

[0044] S20, Checking the custom features of the component, step S20 includes:

[0045] S201. Obtain the feature region Rect2 of the component through methods such as template matching.

[0046] S202. Since image translation takes a long time, it is necessary to directly perform coarse cropping on image ImageSrc2. Calculate the size of the image ImageSrc2 to be coarsely cropped, perform coarse cropping on image ImageSrc2, and obtain the region Rect4 where the coarsely cropped image is located.

[0047] S203. Since the calculated Rect4 may exceed the range of ImageSrc2, ImageSrc2 needs to be padded with pixels. Both ImageSrc2 and Rect4 are padded with 500 pixels in their width and height directions to obtain a coarsely cropped image. Because Rect4 has undergone the padding operation, its position has also changed. The x and y coordinates of Rect4 need to be increased by 250 pixels each to obtain Rect5.

[0048] In S204, a ROI operation is performed on the image ImageSrc2 at Rect5 to obtain a coarsely cropped image ImageSrc3. The image ImageSrc3 is rotated according to the angle of Rect3 to obtain a rotated image ImageSrc4 after coarse cropping, and the image ImageSrc4 is then center-cropped according to the size of Rect3 to obtain an image ImageSrc5 that has undergone coarse cropping, rotation and then center cropping. At this point, the obtained image ImageSrc5 is an image with component features similar to those of the initial image.

[0049] In S205, the image ImageSrc5 is cropped at Rect2 according to the features extracted in S10 to obtain a feature image ImageFeature2.

[0050] In S206, the image ImageFeature2 is processed to obtain a set Contours of all target contours in the image, and noise contours with fewer than 10 contour points among all the contours are filtered out. In the processing of the image ImageFeature2, the threshold used in step S105 is adopted to perform binarization processing on the image ImageFeature2, morphological opening operation is used to filter interference points, and closing operation is used to remove burrs of the contours.

[0051] In S207, an initial minimum center distance MinDistance=9999 is set. For each feature Feature in Features, the center distance between the position of the Feature and each contour in Contours is compared to find the one with the smallest distance, and the area ratio Rate of the contour is compared at the same time,

[0052] Rate=Min(Feature.Area, Contour.Area) / Max(Feature.Area, Contour.Area), Contours with Rate<0.5 are skipped; the contour center distance Distance is calculated at the same time,

[0053] If Distance<MinDistance, MinDistance is updated to Distance. Feature.Area is the area of the feature, Contour.Area is the area of the contour, Feature.center.x is the x value of the feature center, Feature.center.y is the y value of the feature center, contour.center.x is the x value of the contour center, and contour.center.y is the y value of the contour center.

[0054] In S208, the maximum range of the target offset is set

[0055] max_dis = (Feature.Size.Width + Feature.Size.Height) / 2. If Distance ≤ max_dis, then the contour area is determined, and the feature score is...

[0056] Score = min(Feature.Area, Contour.Area) / Max(Feature.Area, Contour.Area). If the Score is greater than the set acceptance level, the feature check passes; otherwise, the feature check fails.

[0057] If Distance > max_dis, then the feature check is directly deemed to have failed. Here, Feature.Size.Width is the width of the feature, Feature.Size.Height is the height of the feature; Feature.Area is the area of ​​the feature, and Contour.Area is the area of ​​the contour.

[0058] S209. Repeat the operations from S205 to S208 until all feature checks are completed. The process is complete only if all feature checks pass. If any feature check fails, it is determined to be a problematic component.

[0059] The specific method for extracting the custom features of the components in step S10 further includes:

[0060] S102. Find all contours in the image ImageSrc2, filter out contours with fewer than 20 contour points, treat all contour points as a whole as a single contour, obtain the region Rect1 containing the smallest bounding rectangle, and calculate the center and angle.

[0061] S103. ImageSrc1 is subjected to affine transformation based on the center and angle calculated by Rect1 to correct it back to the image center. At the same time, it is cropped according to the size of Rect1 to obtain the template image ImageModel.

[0062] S104. Find Rect2 in the template image ImageModel and crop it again to obtain the feature image ImageFeature1;

[0063] S105. Binarize the feature image ImageFeature1, filter out interference points, and remove contour burrs. Find all contours within the image and select the feature type for that range, which is rectangle or circle. Specifically, use automatic thresholding or adjust an appropriate threshold to binarize the image ImageFeature1, use morphological opening operations to filter out interference points, and use closing operations to remove contour burrs.

[0064] S106. Manually remove interference points that do not need to be checked and save the set of features that need to be checked.

[0065] S107. Repeat steps S104 to S106 until all features that need to be checked have been added.

[0066] In step S202, the size of the image ImageSrc2 to be coarsely cropped is calculated based on Rect3. Here, the cropping size Length is the length of the hypotenuse of Rect3. At this point, the final image to be cropped is Rect4, the length and width of image Rect4 are Length, the center position of Rect4 is the center position of Rect3, and the angle of Rect4 is 0 degrees.

[0067] The universal visual inspection feature method provided by this invention first extracts custom features from components, and then checks the custom features of the components, which can avoid poor production quality and make the machine more intelligent.

[0068] The above is a detailed description of the preferred embodiments of the present invention. However, the present invention is not limited to the embodiments described. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of the present invention. All such equivalent modifications or substitutions are included within the scope defined by the claims of this application.

Claims

1. A general visual feature detection method, characterized in that, Includes the following steps: S10, Extraction of custom features of the component, including the step of preprocessing the original image ImageSrc1 to obtain the preprocessed image ImageSrc2, and extracting the feature set of the component to be checked through step S10; Step S10 further includes: S102. Find all contours in the image ImageSrc2, treat all contour points as a whole, and obtain the region Rect1 where the smallest bounding rectangle is located. Calculate the center and angle. S103. The image ImageSrc1 is subjected to affine transformation based on the center and angle calculated by Rect1 to correct it back to the image center. At the same time, it is cropped according to the size of Rect1 to obtain the template image ImageModel. S104. Find Rect2 in the template image ImageModel and crop it again to obtain the feature image ImageFeature1; S105. Binarize the feature image ImageFeature1, filter out interference points, remove burrs from the contours, find all contours in the image, and select the feature type for that range. S106. Manually remove interference points that do not need to be checked and save the set of features that need to be checked. S107. Repeat steps S104 to S106 until all features that need to be checked have been added. S20, Checking the custom features of the component, step S20 includes: S201, Obtain the feature region Rect2 of the component; S202. Perform coarse cropping on the preprocessed image ImageSrc2 to obtain the region Rect4 where the coarse cropped image is located. In step S202, calculate the size of the image ImageSrc2 to be coarsely cropped based on the region Rect3 of the coarse cropped image. Here, the cropping size Length is the length of the hypotenuse of Rect3. At this time, the final image to be cropped is Rect4. The length and width of Rect4 are Length, the center position of Rect4 is the center position of Rect3, and the angle of Rect4 is 0 degrees. S203. Fill the length and width directions of the preprocessed image ImageSrc2 and the region Rect4 where the coarse cropped image is located with pixels to obtain the region Rect5 where the coarse cropped image and the filled image are located respectively. S204. The preprocessed image ImageSrc2 is cropped at the region Rect5 of the filled image to obtain the coarsely cropped image ImageSrc3. The coarsely cropped image ImageSrc3 is rotated according to the angle of the region Rect3 of the coarsely cropped image to obtain the coarsely cropped and rotated image ImageSrc4. Image ImageSrc4 is then centered according to the size of the region Rect3 of the coarsely cropped image to obtain the coarsely cropped, rotated and centered image ImageSrc5. S205. Based on the features extracted in S10, ImageSrc5 performs a cropping operation at the feature region Rect2 of the element to obtain the binarized feature image ImageFeature2. S206. Process the image ImageFeature2 to obtain the set of all target contours Contours in the image, and filter out the noise contours with fewer than 10 contour points in all contours. S207, Set the initial minimum center distance For each feature in Features, compare the feature's position with the center distance of each contour in Contours, find the one with the smallest distance, and simultaneously compare the contour area to Rate. Skip contours with a rate < 0.5; simultaneously calculate the contour center distance (Distance). ,like If so, then update MinDistance to Distance; where Feature.Area is the area of ​​the feature, Contour.Area is the area of ​​the contour, Feature.center.x is the x-value of the feature center, Feature.center.y is the y-value of the feature center, contour.center.x is the x-value of the contour center, and contour.center.y is the y-value of the contour center. S208. Set the maximum range of target offset. ,like Then, the contour area is determined, and the feature score is... If the score is greater than the set acceptance level, the feature check passes; otherwise, the feature check fails. If the feature fails the inspection, it is directly determined that the feature check fails. Feature.Size.Width is the width of the feature, Feature.Size.Height is the height of the feature, Feature.Area is the area of ​​the feature, and Contour.Area is the area of ​​the contour. S209. Repeat the operations from S205 to S208 until all feature checks are completed. The process is complete only if all feature checks pass. If any feature check fails, it is determined to be a problematic component.

2. The general visual detection feature method according to claim 1, characterized in that: In step S10, the image preprocessing includes S101: performing binarization on ImageSrc1 using automatic thresholding or adjusted thresholding, and using morphological opening operation to filter out interference points to obtain ImageSrc2.

3. The general visual detection feature method according to claim 1, characterized in that: In step S102, contours with fewer than 20 contour points are filtered out.

4. The general visual detection feature method according to claim 1, characterized in that: In step S105, the image ImageFeature1 is binarized using an automatic threshold or an adjusted threshold, morphological opening operations are used to filter out interference points, and closing operations are used to remove burrs from the contour.

5. The general visual detection feature method according to claim 1, characterized in that: In step S105, the feature types are rectangle and circle.

6. The general visual detection feature method according to claim 1, characterized in that: In step S201, the feature region position Rect2 of the element is obtained by template matching.

7. The general visual detection feature method according to claim 1, characterized in that: In step S202, the image ImageSrc2 is padded with 500 pixels in both its length and width directions; and the image Rect4 is padded with 250 pixels in both its length and width directions.

8. The general visual detection feature method according to claim 4, characterized in that: In step S206, the threshold used in step S105 is used to binarize the image ImageFeature2, morphological opening operation is used to filter out interference points, and closing operation is used to remove burrs from the contour.

Images