A method for centering the screw cap and identifying the angle of the slot based on machine vision

Abstract

A method for centering a screw cap and identifying the angle of a slot based on machine vision, the steps of which are: pre-acquire an image of the screw cap of this type, perform image processing to establish a template, make the focus center of the template image the center of the screw cap, and establish a template database; collect the image to be identified of the screw cap to be located; perform image processing and match it with the template image. Process the matching results to obtain multiple screw cap center coordinates (X 01 ,Y 01 ),(X 02 ,Y 02 ), ...; establish a relative coordinate system according to the obtained screw cap center, establish a new template again, and store it in the template database; establish a straight line recognition area in the relative coordinate system, identify the straight line of the slot, and obtain the starting point and the end point of multiple straight lines; calculate the slope of the straight line and reversely find the angle, and take the median of each straight line angle as the angle between the slot and the horizontal coordinate at this time. The present invention has great significance for industrial automation.

Description

Technical Field

[0001] The present invention relates to the field of positioning technology, and in particular to a method of visual positioning, angle measurement and machine learning. Background Art

[0002] Based on the growing rise of machine vision, automation and artificial intelligence industries, this method uses the development of CCD or CMOS camera technology to collect and process images. In addition, as the functions of image processing software such as Labview, Matlab, and Open-CV are gradually improved and powerful, the image denoising and filtering processing are more mature, which greatly improves the accuracy of this method. The positioning of the center of the screw cap and the angle of the slot is of great significance to the realization of industrial automation, automatic assembly, disassembly and artificial intelligence. Summary of the invention

[0003] In order to make industrial automation more perfect and transfer and disassembly more intelligent and automated, the present invention provides a method for visually locating the center of a screw cap and the angle of a slot, comprising the following steps:

[0004] Step (1): pre-acquire an image of the type of straight screw cap, grayscale the image and then take a screenshot so that the center of its focus is the center of the screw cap, and establish a database containing the straight screw cap image and set it as a template. The specific steps are as follows:

[0005] (1.1) First, the saturation surface of the cap to be identified is grayed, and a screenshot of the cap is taken so that the focal center of the image is the center of the cap. This serves as the basis for the subsequent matching result center, i.e., the center of the cap, and an initial template 1 is established;

[0006] (1.2) Binarize the cap and take a screenshot of the cap so that the focal center of the image is the center of the cap. This serves as the basis for the subsequent matching result center, i.e., the center of the cap, and establishes the initial template 2;

[0007] (1.3) storing the initial template 1 and the initial template 2 in the template database;

[0008] Step (2): using a camera based on a CCD device to collect an image of the rotating cap to be identified in the environment to be positioned;

[0009] Step (3): After graying and binarizing the image to be identified and performing Fourier filtering, the image is matched with the template image in the database. Based on the matching results, the specific steps are as follows:

[0010] (3.1) The image to be identified is grayed out on the saturation plane to make it a grayscale image, and is matched with the initial template 1 in the template database. The matching degree is set to 80% of the full score, and the maximum number of matches is 4. All the matching results are stored in array 1;

[0011] (3.2) Extract the red surface of the original image and grayscale it, then perform binarization, find the black part, and set the upper limit to 75 to obtain a binary image. Then perform Fourier filtering to remove edge discontinuities to make the image smoother. Perform secondary matching between the processed binary image and the template in the template database, and set the matching degree to 80% of the full score, which means that it is identified as the cap to be found, and store all the matching results in array 1;

[0012] Step (4): Calculate the focal center obtained from the matching result to obtain the coordinates of the four cap centers (X 01 ,Y 01 ),(X 02 ,Y 02 ), (X 03 ,Y 03 ), (X 04 ,Y 04 ), the specific steps are as follows:

[0013] (4.1) Integrate the multiple sets of matching results in step (3) into an array;

[0014] (4.2) Subtract the first element from the integrated array in (4.1) to obtain a new array. Among the elements in the new array, the elements that satisfy X≤10, Y≤10 are identified as a class of elements. X, Y are the elements in the array. Tracing back to the original result array, extract it as array 3, take the average, and identify it as the coordinates of the center of one of the screw caps;

[0015] (4.3) Remove array 3 from the integrated array to form another new array. Subtract the first element from this new array and repeat (4.2). Similarly, find the coordinates of the center of all the caps. Sort them in order from small to large according to the distance from the center of each cap to the origin of the coordinate system. Name the cap coordinate No. 1 (X 01 ,Y 01 ), coordinate 2 (X 02 ,Y 02 )…;

[0016] Step (5): Take a screenshot of the image, create a new template and save it in the template database for the next recognition. R in the following steps is the radius of the circumscribed circle of the screw cap. The specific steps are as follows:

[0017] (5.1) Establishing the region of interest (ROI) of the screenshot according to the coordinates of the circle center obtained in step (4);

[0018] (5.2) Take a screenshot with (X01-R, Y01-R) as the upper left vertex of the rectangle and (X01+R, Y01+R) as the lower right vertex of the rectangle, learn the corresponding template, and save it as a new template;

[0019] (5.3) Take a screenshot with (X02-R, Y02-R) as the upper left vertex of the rectangle and (X02+R, Y02+R) as the lower right vertex of the rectangle, and perform corresponding template learning and save it as a new template. Similarly, find the region of interest (ROI) of all the screw caps and create a new template;

[0020] Step (6): Identify the straight lines of the slot and obtain the starting points and end points of multiple straight lines.

[0021] The specific steps are as follows:

[0022] (6.1) Extract the red plane from the original image and convert it into grayscale;

[0023] (6.2) Based on the coordinates of the circle center obtained in step (4), (X0i-R, Y0i-R) is the upper left vertex of the rectangle, and (X0i+R, Y0i+R) is the lower right vertex of the rectangle, a rectangular region of interest (ROI) for line recognition is established;

[0024] (6.3) Setting the values ​​of the recognition parameters: the minimum length of the recognized straight line is set to one third of the length of the slot, and the edge threshold is set to 75;

[0025] (6.4) For the edge of the slot, the straight line recognition is performed and the coordinates of the starting and ending points of all the straight lines obtained after the recognition are output, as shown in Table 1;

[0026] Table 1

[0027] result 1 2 3 4 X coordinate of starting point 1035.13 1011.18 1037.03 998.46 Starting point Y coordinate 636.00 590.03 560.00 554.00 End point X coordinate 1030.38 997.85 1019.42 98.32 End point Y coordinate 675.95 705.03 693.04 639.95

[0028] Step (7): Calculate the slope of the identified straight line based on the starting point and the end point of the straight line and inversely find the angle. Take the median of the angles of each straight line as the angle between the slot and the pixel coordinate X axis at this time. The specific steps are as follows:

[0029] (7.1) According to the coordinates of the starting and ending points of all straight lines obtained in step (6), the slope K of each straight line is calculated using the two-point method. The calculation formula is as follows:

[0030]

[0031] According to the calculated K value, the inverse trigonometric function is used to calculate the straight line angle value β. The calculation formula is as follows:

[0032] β=arctanK

[0033] (7.2) Sort the β values ​​of each straight line and find the median α of the β values ​​of each straight line;

[0034] (7.3) Use the segment method to calculate the angle θ of the slot. The calculation formula is as follows:

[0035]

[0036] The innovation of the present invention in locating the center of the screw cap is that the grayscale image template matching of the S surface is adopted. In order to improve the recognition accuracy and stability, the matching degree is set to 80% of the full score, and the grayscale image and template matching are performed later to locate the center again.

[0037] The innovation of the present invention for simultaneously locating the centers of multiple screw caps is that after multiple matching coordinates are obtained, the coordinates are classified and processed, all matching center coordinates are integrated into an array, and then a new array is obtained by subtracting the first element from the integrated array. Among the elements in the new array, elements that satisfy X≤10, Y≤10 (X, Y are elements in the array) are identified as a class of elements, and traced back to the original result array, extracted as array 3, averaged, and identified as the center coordinate of one of the screw caps. The center coordinates of all the screw caps to be identified are obtained in turn.

[0038] The innovation of the present invention in autonomous machine learning lies in that, after obtaining the coordinates of the center of the circle (X0i, Y0i), the rectangular region of interest, i.e., ROI, of the screenshot is established with (X0i-R, Y0i-R) as the upper left vertex of the rectangle and (X0i+R, Y0i+R) as the lower right vertex of the rectangle, and the template parameters of the image are processed and stored in the template database for subsequent template matching to improve recognition accuracy and stability.

[0039] The invention is innovative in identifying the angle of the slot in order to improve the recognition accuracy and reduce the recognition area. Based on the recognition of the center coordinates (X0i, Y0i) of the cap, (X0i-R, Y0i-R) is used as the upper left vertex of the rectangle, and (X0i+R, Y0i+R) is used as the lower right vertex of the rectangle to establish a rectangular ROI for straight line recognition. Edge straight line detection is performed in this area to detect the edge of the slot, and the detected straight line is output in the form of the coordinates of the starting point and the ending point. According to the starting point and the ending point of the straight line obtained by recognition, according to the two-point formula

[0040]

[0041] Calculate the slope K of each straight line. And inversely calculate its angle β. In order to improve the accuracy of recognition, the β values ​​are sorted from small to large, and the value α is calculated. Due to the situation (x2≠x1) in the two-point formula, the 90° situation cannot be detected. When the angle of the slot is 89.498≤θ≤-89.498, K≥114 or K≤-114, the angle of the slot is considered to be 90°; therefore, the following piecewise function is specially constructed:

[0042]

[0043] Calculate the angle θ of the slot.

[0044] The invention provides a method for visually locating the center of a screw cap and the angle of a slot, which provides a basis for automatic assembly and disassembly and has great significance for industrial automation. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 , which is a flow chart of the present invention.

[0046] Figure 2 It is the initial S-surface grayscale template image of a single screw cap to be identified in the present invention.

[0047] Figure 3 It is a binary template image of a single screw cap to be identified according to the present invention.

[0048] Figure 4 It is the S-surface grayscale template image of the image to be identified in the present invention.

[0049] Figure 5 This is a binarization and positioning result diagram of the image to be identified in the present invention.

[0050] Figure 6 It is a straight line diagram of the edge of a slot in the angle identification of the present invention. DETAILED DESCRIPTION

[0051] The present invention will be further described below in conjunction with the accompanying drawings.

[0052] Reference Figures 1 to 5 , a method for visually locating the center of a screw cap, the specific steps are as follows:

[0053] Step (1): pre-acquire an image of the type of straight screw cap, grayscale the image and then take a screenshot so that the center of its focus is the center of the screw cap, and establish a database containing the straight screw cap image and set it as a template. The specific steps are as follows:

[0054] (1.1) First, the saturation surface of the cap to be identified is grayed, and a screenshot of the cap is taken so that the focal center of the image is the center of the cap. This serves as the basis for the subsequent matching result center, i.e., the center of the cap, and an initial template 1 is established;

[0055] (1.2) Binarize the cap and take a screenshot of the cap so that the focal center of the image is the center of the cap. This serves as the basis for the subsequent matching result center, i.e., the center of the cap, and establishes the initial template 2;

[0056] (1.3) storing the initial template 1 and the initial template 2 in the template database;

[0057] Step (2): using a camera based on a CCD device to collect an image of the rotating cap to be identified in the environment to be positioned;

[0058] Step (3): After graying and binarizing the image to be identified and performing Fourier filtering, the image is matched with the template image in the database. Based on the matching results, the specific steps are as follows:

[0059] (3.1) The image to be identified is grayed out on the saturation plane to make it a grayscale image, and is matched with the initial template 1 in the template database. The matching degree is set to 80% of the full score, and the maximum number of matches is 4. All the matching results are stored in array 1;

[0060] (3.2) Extract the red surface of the original image and grayscale it, then perform binarization, find the black part, and set the upper limit to 75 to obtain a binary image. Then perform Fourier filtering to remove edge discontinuities to make the image smoother. Perform a secondary match between the processed binary image and the template in the template database, and set the matching degree to 80% of the full score, which means that it is identified as the cap to be found, and store all the matching results in array 1;

[0061] Step (4): Calculate the focal center obtained from the matching result to obtain the coordinates of the four cap centers (X 01 ,Y 01 ),(X 02 ,Y 02 ), (X 03 ,Y 03 ), (X 04 ,Y 04 ), the specific steps are as follows:

[0062] (4.1) Integrate the multiple sets of matching results in step (3) into an array;

[0063] (4.2) Subtract the first element (X1, Y1) from the integrated array in (4.1) to obtain a new array. Among the elements in the new array, the elements that satisfy X≤10, Y≤10 are identified as a class of elements. X, Y are the elements in the array. Tracing back to the original result array, extract it as array 3, take the average, and identify it as the coordinates of the center of one of the screw caps;

[0064] (4.3) Remove array 3 from the integrated array to form another new array. Subtract the first element from this new array and repeat (4.2). Similarly, find the center coordinates (X0i, Y0i) of all the caps. Sort them in order from small to large according to the distance from the center of each cap to the origin of the coordinate system. Name the cap coordinate No. 1 (X0i, Y0i). 01 ,Y 01 ), coordinate 2 (X 02 ,Y 02 )…;

[0065] Step (5): Take a screenshot of the image, create a new template and save it in the template database for the next recognition. The R in the following steps is the radius R of the circumscribed circle of the screw cap. The specific steps are as follows:

[0066] (5.1) Establishing the region of interest (ROI) of the screenshot according to the coordinates of the circle center obtained in step (4);

[0067] (5.2) Take a screenshot with (X01-R, Y01-R) as the upper left vertex of the rectangle and (X01+R, Y01+R) as the lower right vertex of the rectangle, and after learning the corresponding template, save it as a new template;

[0068] (5.3) Take a screenshot with (X02-R, Y02-R) as the upper left vertex of the rectangle and (X02+R, Y02+R) as the lower right vertex of the rectangle, and perform corresponding template learning and save it as a new template. Similarly, find the region of interest (ROI) of all the screw caps and create a new template;

[0069] Reference Figure 6 , the identification method of the slot angle is as follows:

[0070] Step (6): Identify the straight lines of the slot and obtain the starting points and end points of multiple straight lines.

[0071] The specific steps are as follows:

[0072] (6.1) Extract the red plane from the original image and convert it into grayscale;

[0073] (6.2) Based on the coordinates of the circle center obtained in step (4), (X0i-R, Y0i-R) is the upper left vertex of the rectangle, and (X0i+R, Y0i+R) is the lower right vertex of the rectangle, a rectangular region of interest (ROI) for line recognition is established;

[0074] (6.3) Setting the values ​​of the recognition parameters: the minimum length of the recognized straight line is set to one third of the length of the slot, and the edge threshold is set to 75;

[0075] (6.4) For the edge of the slot, the straight line recognition is performed and the coordinates of the starting and ending points of all the straight lines obtained after the recognition are output, as shown in Table 1;

[0076] Table 1

[0077] result 1 2 3 4 X coordinate of starting point 1035.13 1011.18 1037.03 998.46 Starting point Y coordinate 636.00 590.03 560.00 554.00 End point X coordinate 1030.38 997.85 1019.42 98.32 End point Y coordinate 675.95 705.03 693.04 639.95

[0078] Step (7): Calculate the slope of the identified straight line based on the starting point and the end point of the straight line and inversely find the angle. Take the median of the angles of each straight line as the angle between the slot and the pixel level at this time. The specific steps are as follows:

[0079] (7.1) According to the coordinates of the starting and ending points of all straight lines obtained in step (6), the slope K of each straight line is calculated using the two-point method. The calculation formula is as follows:

[0080]

[0081] According to the calculated K value, the inverse trigonometric function is used to calculate the straight line angle value β. The calculation formula is as follows:

[0082] β=arctanK

[0083] (7.2) Sort the β values ​​of each straight line and find the median α of the β values ​​of each straight line;

[0084] (7.3) Use the segment method to calculate the angle θ of the slot. The calculation formula is as follows:

[0085]

Claims

1. A method for centering a screw cap and identifying the angle of a slot based on machine vision, characterized in that Step (1): pre-acquire an image of the type of straight screw cap, grayscale the image and then take a screenshot so that the center of its focus is the center of the screw cap, and establish a database containing the straight screw cap image and set it as a template. The specific steps are as follows: (1.1) First, the saturation surface of the cap to be identified is grayed, and a screenshot of the cap is taken so that the focal center of the image is the center of the cap. This serves as the basis for the subsequent matching result center, i.e., the center of the cap, and an initial template 1 is established; (1.2) Binarize the cap and take a screenshot of the cap so that the focal center of the image is the center of the cap. This serves as the basis for the subsequent matching result center, i.e., the center of the cap, and establishes the initial template 2; (1.3) storing the initial template 1 and the initial template 2 in the template database; Step (2): using a camera based on a CCD device to collect an image of the rotating cap to be identified in the environment to be positioned; Step (3): After graying, binarizing, and Fourier filtering the image to be identified, it is matched with the template image in the database, and the result is obtained based on the matching. The specific steps are as follows: (3.1) The image to be identified is grayed out on the saturation plane to make it a grayscale image, and is matched with the initial template 1 in the template database. The matching degree is set to 80% of the full score, and the maximum number of matches is 4. All the matching results are stored in array 1; (3.2) Extract the red surface of the original image and grayscale it, then perform binarization, find the black part, and set the upper limit to 75 to obtain a binary image. Then perform Fourier filtering to remove edge discontinuities to make the image smoother. Perform a secondary match between the processed binary image and the template in the template database, and set the matching degree to 80% of the full score, which means that it is identified as the desired cap, and store all the matching results in array 1; Step (4): Calculate the focal center obtained from the matching result to obtain the coordinates of the four cap centers (X 01 ,Y 01 ),(X 02 ,Y 02 ), (X 03 ,Y 03 ), (X 04 ,Y 04 ), the specific steps are as follows: (4.1) Integrate the multiple sets of matching results in step (3) into an array; the array format is [(X1, Y1), (X2, Y2), (X3, Y3), ..., (Xn, Yn)]; (4.2) Subtract the first element from the integrated array in (4.1) to obtain a new array. Among the elements in the new array, the elements that satisfy X≤10, Y≤10 are identified as a type of element, and X, Y are the elements in the array; trace back to the original result array, extract it as array 3, take the average, and identify it as the coordinates of the center of one of the screw caps; (4.3) Remove array 3 from the integrated array to form another new array. Subtract the first element from this new array and repeat (4.2). Similarly, find the coordinates of the center of all the caps. Sort them in order from small to large according to the distance from the center of each cap to the origin of the coordinate system. Name the cap coordinate No. 1 (X 01 ,Y 01 ), coordinate 2 (X 02 ,Y 02 )…; Step (5): Take a screenshot of the image, create a new template and save it in the template database for the next recognition. In the following steps, R is the radius of the circumscribed circle of the screw cap. The specific steps are as follows: (5.1) Establish the region of interest (ROI) of the screenshot according to the coordinates of the center of the circle obtained in step (4); the coordinate system takes the upper left vertex of the image as the origin, the right is the positive direction of the X axis, and the downward is the positive direction of the Y axis; (5.2) Take a screenshot with (X01-R, Y01-R) as the upper left vertex of the rectangle and (X01+R, Y01+R) as the lower right vertex of the rectangle, learn the corresponding template, and save it as a new template; (5.3) Take a screenshot with (X02-R, Y02-R) as the upper left vertex of the rectangle and (X02+R, Y02+R) as the lower right vertex of the rectangle, and perform corresponding template learning and save it as a new template; and so on, find the rectangular region of interest (ROI) of all the screw caps and establish a new template; Step (6): Identify the straight lines of the slot and obtain the starting points and end points of multiple straight lines. The specific steps are as follows: (6.1) Extract the red plane from the original image and convert it into grayscale; (6.2) Based on the coordinates of the circle center obtained in step (4), (X0i-R, Y0i-R) is the upper left vertex of the rectangle, and (X0i+R, Y0i+R) is the lower right vertex of the rectangle, a rectangular region of interest (ROI) for line recognition is established; (6.3) Setting the values ​​of the recognition parameters: the minimum length of the recognized straight line is set to one third of the length of the slot, and the edge threshold is set to 75; (6.4) Perform straight line recognition on the edge of the slot and output the coordinates of the starting and ending points of all straight lines obtained after recognition; Step (7): Calculate the slope of the identified straight line based on the starting point and the end point of the straight line and inversely find the angle. Take the median of the angles of each straight line as the angle between the slot and the pixel coordinate X axis at this time. The specific steps are as follows: (7.1) Based on the coordinates of the starting and ending points of all the straight lines obtained in step (6), use the two-point method to calculate the slope K of each straight line; According to the calculated K value, the inverse trigonometric function is used to calculate the straight line angle value β. The calculation formula is as follows: β=arctanK (7.2) Sort the β values ​​of each straight line and find the median α of the β values ​​of each straight line; (7.3) The angle θ of the slot is calculated using the piecewise method. Due to the situation of x2≠x1 in the two-point formula, the 90° situation cannot be detected. When the angle of the slot is 89.498≤θ≤-89.498, K≥114 or K≤-114, the angle of the slot is considered to be 90°. Therefore, the following piecewise function is specially constructed:

Images