A real-time positioning method based on multi-template matching based on improved 2D-ICP

Abstract

The present invention belongs to the technical field of online positioning of machine vision template matching, and particularly relates to a multi-template matching real-time positioning method based on improved 2D-ICP. First, an industrial camera is installed and adjusted on a processing platform, a template workpiece is placed and a template image is collected, and a template database is made and saved offline; then, images are collected in real time, and a multi-template matching method and OpenMP parallel programming are used to perform rough matching on the search image, and then an improved 2D-ICP algorithm of the matching measurement function is used for precise matching to obtain the position and posture information of the target with sub-pixel accuracy; and the actual coordinates and angles of the target object on the processing platform are calculated based on the actual installation position and pixel equivalent of the camera; the present invention can realize the matching process of any planar workpiece template and determine the workpiece position and posture, and has the advantages of being fast, accurate and stable.

Description

Technical Field

[0001] The invention belongs to the technical field of machine vision template matching online positioning, and in particular relates to a multi-template matching real-time positioning method based on improved 2D-ICP. Background Art

[0002] With the rapid development of the manufacturing industry, high-speed, high-precision, highly automated and flexible production lines have attracted more and more attention. The application of machine vision technology in the field of automated production has become the preferred solution for solving problems such as detection, measurement, and positioning. Among them, the rapid and accurate positioning of the workpiece is an important step before the workpiece is processed. Its speed determines the processing efficiency, and its accuracy determines the processing quality. At present, the visual plane positioning technology generally adopts the template matching algorithm. The grayscale template matching method uses the grayscale distribution characteristics of the template image to traverse and search in the image to be searched. Its principle is simple and very effective for a relatively ideal environment, but it cannot overcome interference factors such as occlusion and illumination, so it is not suitable for the industrial field. The template matching method based on feature descriptors is to generate feature descriptors in the template image and the image to be searched, and then match the descriptors to obtain matching point pairs, thereby obtaining an affine transformation. However, this method has no high matching rate and the accuracy is too low, so it is only used in the field of computer vision. By generating a template database from different angles and scales, then traversing and searching it with the image to be searched, and finally performing sub-pixel extraction and positioning, the amount of template matching calculation increases sharply with the improvement of accuracy, especially for high-resolution images and precise posture matching at the bottom of the pyramid, which requires a lot of time. Summary of the invention

[0003] The purpose of the present invention is to overcome the defects of the prior art, improve the existing template matching algorithm, improve the efficiency and accuracy of visual positioning, and provide a multi-template matching real-time positioning method based on improved 2D-ICP. The real-time image is roughly matched at the edge using a multi-template offline database to obtain a rough position and angle, and then the 2D-ICP algorithm with an improved matching degree measurement function is used for precise matching to obtain the final position and posture. Finally, the actual coordinates and angles of the workpiece are calculated from the camera physical coordinates and pixel equivalents, which has the advantages of fast speed, high precision, high stability, etc.

[0004] The technical solution for achieving the purpose of the present invention is:

[0005] A multi-template matching real-time positioning method based on improved 2D-ICP includes the following steps:

[0006] Step 1. Adjust the camera: Focus the high-resolution industrial camera on the workpiece plane, fix the working focal length, and adjust the light source and camera exposure time to make the camera image clear with distinct edges.

[0007] Step 2. Make a template database: Place the template workpiece, manually select the area image M containing the template workpiece, and set the number of pyramid layers and angle step according to the size of M. For each angle step in each layer of the pyramid image, use the Canny edge operator to extract the edge of the image at this time, record and save the edge point set P (n, θ, x, y, G x , G y , m), where n and θ represent the pyramid layer number and angle step size of the edge point, x, y are the positions of the edge points, G x , G y , m are the x-direction gradient and y-direction gradient of the edge point, and the gradient modulus. Save the data offline in a file.

[0008] Step 3. Real-time image acquisition: Place the workpiece to be positioned randomly within the camera's field of view (it can be partially outside the field of view), adjust the camera acquisition frame rate to enable real-time image acquisition, and transmit it to the host computer for processing.

[0009] Step 4. Rough matching: Use the image binarization method to extract the region of interest ROI in the image to be searched. According to the number of template data pyramid layers n, use the template data of the nth layer to traverse and search in the nth layer pyramid image of the ROI, and then continue to search in the neighborhood until the bottom layer of the pyramid, and obtain the pixel-level positioning accuracy x s ,y s ,θ s .

[0010] Step 5. Fine matching: In x s ,y s ,θ s 2D-ICP edge point registration is performed near the edge point, and the precise matching result x is obtained by iterating the corresponding edge point distance and the gradient angle of the point pair. r ,y r ,θ r .

[0011] Step 6. Workpiece pose calculation: According to the x obtained by precise matching in step 5 r ,y r ,θ r , combined with the relative pose of the camera and the platform and the camera pixel equivalent, the actual pose information of the workpiece is calculated.

[0012] Furthermore, in step 2, the template database is produced offline. First, the template image area is manually selected, and then the Canny edge extraction is performed on the template image of the nth layer and the θth angle in turn. For the extracted edge points, the noise points and weakly influential points can be manually deleted, and the strongly influential edge points are retained. At the same time, the edge points are downsampled at equal intervals, and the remaining edge points and their information are saved in an offline file.

[0013] Furthermore, the specific method of rough matching in step 4 is as follows: firstly, the real-time image collected in step 3 is binarized, and the ROI is extracted, and the ROI area is used as the area to be searched; then, the image is pyramid-layered according to the parameter n in S2; then, the template data of the nth layer in the template database, i.e., the edge point information, is read, and the ROI area of ​​the nth layer of the image to be searched is traversed and searched through the parallel calculation of the OpenMP compiled code, and the angle search step length of the nth layer is 2 n-1 degrees, and the result interval of the previous layer is the search range; calculate the average value of the sum S of the angle values ​​between all template edge points and the corresponding image pixels. The higher the score, the better the match.

[0014] in

[0015] Where: N is the size of the edge point set P, m, n are the upper left corner pixel coordinates of the ROI area of ​​the image to be searched,

[0016] i is the index of the point set P,

[0017] x i ,y i is the pixel coordinate of the edge index point in the template image,

[0018] are the gradient vector of the edge index point and the gradient vector of the pixel position of the ROI to be searched;

[0019] When calculating the matching degree S for a pixel point in the search ROI area, if , then terminate the calculation of the current point and jump to the next position, where S j is the matching value of the first j edge points, MinScore is the preset minimum matching score, and N is the size of the template edge point set.

[0020] Furthermore, the specific method of fine matching in step 5 is: perform Canny edge extraction in the 0th layer of the pyramid, that is, the original image to be searched, and filter and retain the rough matching result x obtained in step 4 s ,y s ,θ s The edge point set Q near the template data is obtained; the iterative closest point (ICP) algorithm is used for precise matching. Since the point sets P and Q are both two-dimensional points, 2D-ICP is used as the name of the method;

[0021] The steps are as follows: first, use KD-tree to divide the Q point set in the search image, search for the nearest point for each point in the template point set P, and obtain the mapping relationship f(P,Q); then, according to the mapping relationship f(P,Q), calculate the transformation matrix T and R of P→Q, where T is the translation matrix and R is the rotation matrix; then, according to the transformation matrix, calculate the corresponding point set P after the transformation of the point set P 1 ; Now calculate the point set P 1 With Q f(P,Q) The degree of match

[0022] Where N is the point set P 1 size,

[0023] d i is the i-th pair of mapping points f(P i 1 ,Q i ),

[0024] θ i is the i-th pair of mapping points f(P i 1 ,Q i )’s gradient angle;

[0025] Determine whether the matching degree V is less than the preset threshold τ. If so, use the current P 1 The pose is the result of the precise matching. If it is greater than, the first step of the point set P is returned. 1 Search for the nearest point for each point in the image to obtain the mapping relationship f1(P,Q), and iterate until the matching degree V is obtained. n Less than the threshold τ, the precise matching result x is obtained s ,y s ,θ s .

[0026] After adopting the above technical solution, the present invention has the following positive effects:

[0027] (1) The present invention uses manual interaction to generate template data, removes clutter and weakly influential points from edge points, and then performs equal-interval downsampling on the remaining points, thereby greatly reducing the amount of matching calculations and improving the matching speed.

[0028] (2) The present invention uses an image binarization method to perform binary segmentation on the search image and extract the region of interest containing the target, which greatly reduces the scope of the initial traversal search and improves the matching speed through OpenMP parallel programming calculation.

[0029] (3) The present invention adopts a 2D-ICP algorithm with an improved matching measure function to perform iterative searches on both position and angle simultaneously, thereby improving positioning speed and accuracy, wherein the positioning accuracy reaches sub-pixel level. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] In order to make the content of the present invention more clearly understood, the present invention is further described in detail below according to specific embodiments and in conjunction with the accompanying drawings, wherein

[0031] Figure 1 is a flow chart of the method of the present invention;

[0032] Figure 2 Make a template database flow chart for the present invention;

[0033] Figure 3 This is a rough matching flow chart of the present invention;

[0034] Figure 4 This is a precise matching flow chart of the present invention. DETAILED DESCRIPTION

[0035] like Figure 1 As shown, a multi-template matching real-time positioning method based on improved 2D-ICP includes the following steps:

[0036] Step 1. Adjust the camera: Focus the high-resolution industrial camera on the workpiece plane, fix the working focal length, adjust the light source and camera exposure time, so that the camera image is clear and the edges are distinct. The example uses the Daheng Mercury MER-503-36U3M-L model, 500W pixels, and the collected image size is 2448×2048.

[0037] Reference Figure 2 ,Step 2. Make a template database: Make the template database offline. First, manually select the template image area, determine the optimal number of pyramid layers and the angle step of each pyramid search according to the image size and template artifact features; then perform Canny edge extraction on the template image of the nth layer and the θth angle in turn, and record the edge points (x, y, G x , G y , m), where x, y are the pixel positions of edge points, G x , G y is the normalized value of the gradient of the edge point in the x and y directions, and m is the modulus of the edge point gradient. You can manually delete noise points and weakly influential points, retain strongly influential edge points, and at the same time, perform equal-interval downsampling on edge points, and save the remaining edge points and their information in an offline file.

[0038] Step 3. Real-time image acquisition: Place the workpiece to be positioned randomly within the camera's field of view (it can be partially outside the field of view), adjust the camera acquisition frame rate to enable real-time image acquisition, set the frame rate at around 15fps, and transmit it to the host computer for processing.

[0039] Reference Figure 3 , Step 4. Rough matching: First, perform binarization on the real-time image collected in step 3, extract ROI, and use the ROI area as the area to be searched; then, pyramid-layer the image according to the parameter n in step 2; then read the template data of the nth layer in the template database, that is, the edge point information, and perform traversal search in the ROI area of ​​the nth layer of the image to be searched through parallel calculation of the OpenMP compiled code, and the angle search step length of the nth layer is 2 n-1 degrees, and the result interval of the previous layer is the search range; calculate the average value of the sum S of the angle values ​​between all template edge points and the corresponding image pixels. The higher the score, the better the match.

[0040] in

[0041] Where: N is the size of the edge point set P,

[0042] m, n are the pixel coordinates of the upper left corner of the ROI area of ​​the image to be searched,

[0043] i is the index of the point set P,

[0044] x i ,y i is the pixel coordinate of the edge index point in the template image,

[0045] They are the gradient vector of the edge index point and the gradient vector at the pixel position of the ROI to be searched.

[0046] When calculating the matching degree S for a pixel point in the search ROI area, if , then terminate the calculation of the current point and jump to the next position, where S j is the matching value of the first j edge points, MinScore is the preset minimum matching score, and N is the size of the template edge point set.

[0047] Reference Figure 4 , Step 5. Fine matching: Perform Canny edge extraction in the pyramid layer 0, i.e. the original image to be searched, and filter and retain the rough matching result x obtained in step 4 s ,y s ,θ s The edge point set Q near the template data is obtained; the iterative closest point (ICP) algorithm is used for precise matching. Since the point sets P and Q are both two-dimensional points, 2D-ICP is used as the name of the method.

[0048] The steps are as follows: first, use KD-tree to divide the Q point set in the search image, search for the nearest point for each point in the template point set P, and obtain the mapping relationship f(P,Q); then, according to the mapping relationship f(P,Q), calculate the transformation matrix T and R of P→Q, where T is the translation matrix and R is the rotation matrix; then, according to the transformation matrix, calculate the corresponding point set P after the transformation of the point set P 1 ; Now calculate the point set P 1 With Q f(P,Q) The degree of match

[0049] Where N is the point set P 1 size,

[0050] d i is the i-th pair of mapping points f(P i 1 ,Q i ),

[0051] θ i is the i-th pair of mapping points f(P i 1 ,Q i )’s gradient angle;

[0052] Determine whether the matching degree V is less than the preset threshold τ. If so, use the current P 1 The pose is the result of the precise matching. If it is greater than, the first step of the point set P is returned. 1 Search for the nearest point for each point in the image to obtain the mapping relationship f1(P,Q), and iterate until the matching degree V is obtained. n Less than the threshold τ, the precise matching result x is obtained s ,y s ,θ s .

[0053] Step 6. Workpiece pose calculation: According to step 5, the relative pose of the camera and the platform is usually fixed mechanically during installation. The coordinate axes are kept parallel as much as possible during installation. The nine-point calibration method is used to accurately calibrate the camera pose and obtain the rotation matrix R. c and the translation matrix T c , and the camera pixel equivalent d can also be calculated. The x obtained by the precise matching in step 5 r ,y r ,θ r , combined with the relative pose R of the camera and the platform c 、T c And the camera pixel equivalent d, the actual pose information of the workpiece is calculated.

[0054] The specific embodiments described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A multi-template matching real-time positioning method based on improved 2D-ICP, It is characterized in that The following steps are involved: Step 1. Adjust the camera: focus the high-resolution industrial camera on the workpiece plane, fix the working focal length, adjust the light source and camera exposure time, so that the camera image is clear and the edges are distinct; Step 2. Create a template database: Place the template workpiece, manually select the area image M containing the template workpiece, set the number of pyramid layers and angle step according to the size of M; for each angle step in each layer of the pyramid image, use the Canny edge operator to extract the edge of the image at this time, record and save the edge point set P (n, ,x,y, , , m), where n and Represents the number of pyramid layers and angle steps where the edge point is located. x, y are the locations of edge points, , , m are the x-direction gradient and y-direction gradient of the edge point, and the gradient modulus; Then save the data offline in a file; Step 3. Real-time image acquisition: Place the workpiece to be positioned randomly within the camera's field of view or partially outside the field of view, adjust the camera acquisition frame rate to acquire images in real time, and transmit them to the host computer for processing; Step 4. Rough matching: Use the image binarization method to extract the region of interest ROI in the image to be searched. According to the number of layers n of the template data pyramid, use the template data of the nth layer to traverse and search in the nth layer pyramid image of the ROI, and then continue to search in the neighborhood until the bottom layer of the pyramid to obtain pixel-level positioning accuracy. , , ; Step 5. Fine matching: , , 2D-ICP edge point registration is performed nearby, and the precise matching result is obtained by iterating the corresponding edge point distance and the gradient angle of the point pair. , , ; Step 6. Workpiece pose calculation: obtained by precise matching in step 5 , , , combined with the relative pose of the camera and the platform and the camera pixel equivalent, the actual pose information of the workpiece is calculated; The specific method of the rough matching in step 4 is: firstly perform binarization processing on the real-time image collected in step 3, extract ROI, and use the ROI area as the area to be searched; then perform pyramid layering on the image according to the parameter n in step 2; Then read the template data of the nth layer in the template database, i.e. the edge point information, and perform parallel calculations through the OpenMP compiled code to perform a traversal search in the ROI area of ​​the nth layer of the image to be searched. The step length of the angle search of the nth layer is degrees, and the result interval of the previous layer is the search range; calculate the average value of the sum S of the angle values ​​between all template edge points and the corresponding image pixels. The higher the score, the better the match. in , Where: N is the size of the edge point set P, m, n are the pixel coordinates of the upper left corner of the ROI area of ​​the image to be searched, i is the index of the point set P, is the pixel coordinate of the edge index point in the template image, , are the gradient vector of the edge index point and the gradient vector of the pixel position of the ROI to be searched; When calculating the matching degree S for a pixel point in the search ROI area, if If the current point is not reached, the calculation is terminated and the next point is jumped to. in is the matching value of the first j edge points, MinScore is the preset minimum matching score. N is the size of the template edge point set; The specific method of fine matching in step 5 is: perform Canny edge extraction in the 0th layer of the pyramid, that is, the original image to be searched, and filter and retain the rough matching results obtained in step 4 , , The edge point set Q near the template data is obtained; the iterative closest point (ICP) algorithm is used for precise matching; The steps are as follows: first use KD-tree to divide the Q point set in the search image, search for the nearest point for each point in the template point set P, and obtain the mapping relationship ; Then according to the mapping relationship , calculate The transformation matrix T, R, where T is the translation matrix and R is the rotation matrix; then the corresponding point set after the point set P is transformed is calculated according to the transformation matrix ; At this time, the calculation point set and The degree of match , Where N is the point set size, is the i-th pair of mapping points The Euclidean distance of is the i-th pair of mapping points The gradient angle of Determine whether the matching degree V is less than the preset threshold If it is less than, then the current The pose is the result of the exact match. If it is greater than, the first step point set is returned. Search for the nearest point for each point in and get the mapping relationship , iterate in this way until the matching degree is obtained Less than threshold , get the exact matching result , , , which is sub-pixel accuracy.

2. The multi-template matching real-time positioning method based on improved 2D-ICP according to claim 1, Features: In step 2, the template database is prepared offline. First, the template image area is manually selected, and then the nth layer and the Canny edge extraction is performed on the template image of the angle. For the extracted edge points, it is necessary to manually delete the noise and weak influence points, retain the strong influence edge points, and at the same time, the edge points are downsampled at equal intervals, and the remaining edge points and their information are saved in an offline file.

Images