Template Mark Detection Method and Template Position Correction Method Using a Single Camera

Abstract

The present disclosure provides a method for detecting a template mark using a single camera and a method for correcting a template position, which includes the following steps: Step S1: An image of a mark in a template is captured using a single camera, and a binary image is obtained by preprocessing. Step S2: An edge jaggy corner detection is performed on the binary image, and an edge jaggy corner set is obtained. Step S3: An edge detection and a line detection are performed on the binary image in order, and a roughly detected edge line segment set is obtained. Step S4: The roughly detected edge line segment set is checked, and the collinear line segments are left by judging, and a roughly detected collinear line segment set is obtained. Step S5: The edge jaggy corner set is checked, and a line collinearity judgment is performed on the line segments and points in the roughly detected collinear line segment set, and a precisely detected point and a line set are obtained. Step S6: A line fitting is performed on the precisely detected points and the line set, and the inclination angle of each line is calculated by the arctangent function, and the detection of the mark is completed. Furthermore, the position of the template is corrected.

Description

Technical Field

[0001] The present disclosure belongs to the field of optical detection technology, and specifically relates to a template mark detection method and a template position correction method using a single camera.

[0002] (Cross-reference to related applications) The present disclosure claims priority based on a Chinese application with application number 202211651503.6 filed on December 21, 2022, and all of its content is incorporated herein by reference.

Background Art

[0003] In the process of transporting a template by a robot, position correction is required. Generally, two PSDs (Position Sensitive Detectors) are used for optical correction or two cameras are used for image correction. However, when the space above and below the template suction cup is limited, there is not enough space to install the PSD and the camera. When performing straight line detection on a position mark by an optical method, one straight line has a constant slope and it is difficult to correct the slope. Therefore, the accuracy of the slope of the mark obtained by straight line detection is low. In addition, in a grayscale image, the pixel values of adjacent pixel points have a relatively small difference. Therefore, the number of corners obtained by corner detection of the grayscale image is relatively small, and the position information of the mark cannot be determined using the obtained corners.

Summary of the Invention

Problems to be Solved by the Invention

[0004] In view of the above problems, the present disclosure provides a template mark detection method and a template position correction method using a single camera, which solve the technical problems in the conventional method, such as the limited installation of the detection device and the low accuracy of the inclination angle of the obtained mark.

Means for Solving the Problems

[0005] One aspect of the present disclosure provides a method for detecting a template mark using a single camera. The method for detecting a template mark using a single camera includes step S1 of acquiring an image of a mark in a template by a single camera and obtaining a binary image through preprocessing; step S2 of performing edge jaggy corner detection on the binary image to obtain an edge jaggy corner set; step S3 of sequentially performing edge detection and line detection on the binary image to obtain a roughly detected edge line segment set; step S4 of checking and judging the roughly detected edge line segment set, leaving collinear line segments, and obtaining a roughly detected collinear line segment set; step S5 of checking the edge jaggy corner set, performing line and point collinearity determination on the line segments and points in the roughly detected collinear line segment set, and obtaining an accurately detected point and line set; and step S6 of performing linear fitting on the accurately detected point and line set, calculating the inclination angle of each line by an arctangent function, and ending the detection of the mark.

[0006] Furthermore, step S1 includes step S11 of acquiring an image of a mark in a template by a single camera to obtain an initial image; step S12 of converting the initial image into a grayscale image and performing filtering; and step S13 of performing binarization processing on the image obtained in step S12 to obtain a binary image.

[0007] Furthermore, step S2 includes performing detection on pixel points in the binary image using a Shi-Tomasi corner response function to obtain an edge jaggy corner set.

[0008] Furthermore, step S3 includes performing Canny edge detection on the binary image to obtain an edge pattern; and obtaining a plurality of edge line segments in the mark by cumulative probabilistic Hough transform based on the edge pattern, and obtaining a roughly detected edge line segment set composed of the plurality of edge line segments.

[0009] Furthermore, step S4 includes step S41 of arbitrarily selecting two line segments from the set of roughly detected edge line segments to obtain four endpoints, step S42 of calculating the area of the quadrilateral formed by the four endpoints and calculating the slope of one of the line segments and the slopes of the line segments formed by connecting the two endpoints of one line segment and the two endpoints of the other line segment respectively, step S43 of determining that the two line segments are collinear when the area of the quadrilateral is less than a first predetermined threshold and the differences in slopes are both less than a second predetermined threshold, and step S44 of checking the set of roughly detected edge line segments, leaving the collinear line segments among them, and obtaining a set of roughly detected collinear line segments.

[0010] Furthermore, step S5 includes step S51 of arbitrarily selecting one corner from the set of edge jagged corners and forming triangles with the corner and the line segments in the set of roughly detected collinear line segments respectively, step S52 of calculating the area of the triangle, step S53 of determining that the corner and the line segment are collinear when the area of the triangle is less than a third predetermined threshold, and step S54 of checking the set of edge jagged corners and integrating the corners and line segments that are collinear to obtain a set of accurately detected points and lines.

[0011] Furthermore, step S5 further includes step S55 of identifying the same collinear line segments in the set of accurately detected points and lines, leaving any one of them, and removing the other overlapping line segments.

[0012] Furthermore, step S6 includes step S61 of performing linear fitting on the corners that are collinear with the points and lines in the set of accurately detected points and lines and the two endpoints of the corresponding line segments to obtain the equation of the line, and step S62 of calculating the inclination angle of each line using the arctangent function based on the equation of the line.

[0013] Furthermore, the mark in the template includes one of a "rice" shape and a "cross" shape.

[0014] Another aspect of the present disclosure provides a template position correction method based on a single camera. The template position correction method based on a single camera includes: step S1 of acquiring an image of a mark on a template by a single camera and obtaining a binary image through preprocessing; step S2 of performing edge jaggy corner detection on the binary image to obtain an edge jaggy corner set; step S3 of sequentially performing edge detection and line detection on the binary image to obtain a roughly detected edge line segment set; step S4 of checking and judging the roughly detected edge line segment set to leave collinear line segments and obtaining a roughly detected collinear line segment set; step S5 of checking the edge jaggy corner set, performing line and point collinearity determination on the line segments, points in the roughly detected collinear line segment set, and obtaining an accurately detected point and line set; step S6 of performing line fitting on the accurately detected point and line set and calculating the inclination angle of each line by an arctangent function; and step S7 of correcting the position of the template based on the difference between the inclination angle of each line and a standard angle.

Advantages of the Invention

[0015] The template mark detection method and template position correction method using a single camera according to the present disclosure use a single camera to acquire an image, thus reducing the use of space in the station and solving the problem that the installation of the detection device is restricted. Furthermore, by utilizing the binarization of the image, more and more accurate edge corners can be obtained. Additionally, by using the collinear straight lines on both sides of the connected part of the template mark, line fitting is performed to obtain a more accurate inclination angle of the straight line, and by determining the accurate position information of the mark, high-precision detection of the template mark is realized, and the position of the template can be corrected based on the position information of the mark obtained by the detection method.

Brief Description of the Drawings

[0016]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

DETAILED DESCRIPTION OF THE INVENTION

[0017] To make the objectives, technical solutions, and advantages of the present disclosure clearer, the present disclosure will be described in more detail below with reference to the drawings using specific embodiments.

[0018] The terms used herein are only for the purpose of describing specific embodiments and do not limit the present disclosure. The terms such as "including" and "comprising" used herein represent the presence of at least one of the corresponding features, steps, operations, and components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0019] When a direction indication is described in an embodiment of the present disclosure, the direction indication only shows the relative positional relationship, movement status, etc. of each component in a specific posture. When the specific posture changes, the direction indication also changes accordingly.

[0020] The ordinal numbers such as "first", "second", "third", etc. used in the specification and claims are for modifying the corresponding components and do not mean that the corresponding components have ordinal numbers, nor do they represent the order of one component and another component or the order in the manufacturing method. These ordinal numbers are only for clearly distinguishing components with the same name.

[0021] The present disclosure provides a method for detecting a template mark using a single camera. As shown in FIG. 1, the method for detecting a template mark using a single camera includes the following steps. Step S1: Obtain an image of the mark in the template using a single camera, and obtain a binary image through preprocessing. Step S2: Perform edge jaggy corner detection on the binary image to obtain an edge jaggy corner set. Step S3: Perform edge detection and line detection on the binary image in sequence to obtain a rough detection edge line segment set. Step S4: Check and judge the rough detection edge line segment set, and leave the collinear line segments to obtain a rough detection collinear line segment set. Step S5: Check the edge jaggy corner set, and perform line and point collinearity determination on the line segments, points in the rough detection collinear line segment set to obtain an accurate detection point, line set. Step S6: Perform linear fitting on the accurate detection point, line set, calculate the inclination angle of each line using the arctangent function, and complete the detection of the mark.

[0022] The schematic diagram of the station of the device used in this detection method is shown in FIG. 2. The device includes a light source, a lens group, and a CCD camera, and the working process is as follows. The robot transports the template to the template station below the CCD camera, the light from the light source arrives at the mark in the template via the lens group, the reflected light with mark information further arrives at the CCD camera via the lens group, and the image collected by the CCD camera is processed. Since this method uses a single CCD camera to obtain images, it can solve the problem of reduced use of the station space and limited installation of the detection device. Since clamping square grooves are provided around the perimeter of the template station, the rotation range of the template in the square groove is 0.5° or less, and the movement range is ±0.5 mm or less. Within this range, the accuracy of identifying the position of the image mark obtained by the single CCD camera is high, and the position accuracy of the template after subsequent position correction is also correspondingly high.

[0023] Specifically, first, preprocessing and binarization are performed on the acquired image. After binarization, edge jaggedness occurs in the image. By utilizing this feature, more and more accurate edge jaggedness corners can be obtained during corner detection. Then, edge jaggedness corner detection and straight line detection are performed in parallel (the execution of step S2 and step S3 may or may not be simultaneous, and the execution order is not limited). Collinearity detection is performed on the obtained roughly detected edge line segments, leaving two or more collinear line segments, and a set of roughly detected collinear line segments is obtained. Then, the set of edge jaggedness corners is checked, and point and line collinearity determination is performed in the set of roughly detected collinear line segments, leaving collinear edge jaggedness corners, and an accurately detected point and line set is obtained. Furthermore, straight line fitting and inverse trigonometric function operations are performed on the accurately detected point and line set to obtain a more accurate inclination angle of the straight line. According to the above processing process, high-precision detection of the template mark can be realized. That is, the present disclosure utilizes image binarization to obtain more and more accurate edge corners, and further uses the collinear straight lines on both sides of the connected part of the template mark to perform straight line fitting to obtain a more accurate inclination angle of the straight line, and can obtain more accurate position information of the mark.

[0024] In the above embodiment, step S1 includes the following steps. Step S11: An image of the mark in the template is acquired by a single camera to obtain an initial image. Step S12: The initial image is converted into a grayscale image and filtered. Step S13: Binarization processing is performed on the image obtained in step S12 to obtain a binarized image.

[0025] An image is acquired for the marks in the template by a CCD camera to obtain an initial image. The initial image is converted into a grayscale image and filtered. The size of the filtering block is variable, and the result is shown in FIG. 3 (illustrated by a "rice" - shaped mark). Here, the reason for filtering is that the edges of the image are very susceptible to noise interference. Therefore, in order to prevent misdetection of edge information, it is necessary to filter the image to remove noise. The purpose of filtering is to smooth the non - edge regions with relatively weak texture to obtain more accurate edges.

[0026] Then, all pixel points in the grayscale image are checked, the median value of the pixels in this grayscale image is calculated, and binarization is performed on all pixel values based on the median value. For example, pixel values smaller than the median value are set to 255, and pixel values larger than the median value are set to 0. Of course, other values can also be used, but the difference between the two selected values should be large enough to distinguish the image from the blank area. Generally, image binarization means setting the grayscale value of pixel points on the image to 0 or 255. The result is shown in FIG. 4. As can be seen from the drawing, a plurality of jagged edges indicated by the round marks in FIG. 4 are formed on the alignment mark pattern edges after the binarization process.

[0027] In the above - mentioned embodiment, step S2 includes performing detection on the pixel points in the binarized image by using the Shi - Tomasi corner response function to obtain a set of edge jagged corners.

[0028] Edge jagged corner detection is performed on the binarized image to obtain image Shi - Tomasi corner detection data, and a set of edge jagged corners of the image {sp1, sp2, sp3, … sp m} is obtained. The result of the edge jagged corner detection is shown in FIG. 5. In order to obtain sufficient points, the quality coefficient in the Shi - Tomasi corner response function is made as small as possible.

[0029] In the above embodiment, step S3 includes performing Canny edge detection on the binary image to obtain an edge pattern, and obtaining a plurality of edge line segments in the mark by cumulative probability Hough transform based on the edge pattern, and obtaining a set of roughly detected edge line segments composed of the plurality of edge line segments.

[0030] Edge detection is performed on the binary image by a general Canny edge detection algorithm. As the principle for selecting specific numerical values in the Canny edge detection algorithm, in the process of obtaining edges, some weak edges are obtained. These weak edges may be due to the real image or due to noise, and it is necessary to remove the weak edges caused by noise. Based on the relationship between the gradient of the current edge pixel and two threshold values, the attribute of the current edge is judged. When the gradient of the current edge pixel is greater than or equal to the second edge threshold value, the pixel of the current edge is marked as a strong edge, that is, the edge is due to the real image. When the gradient of the current edge pixel is a value between the second edge threshold value and the first edge threshold value, the pixel of the current edge is marked as a weak edge (still retained at this stage). When the gradient of the current edge pixel is less than or equal to the first edge threshold value, the current edge is rejected. After obtaining the weak edges by the above processing, it is determined whether the weak edge is due to the real image or due to noise by judging whether the weak edge is connected to a strong edge. If the weak edge is connected to a strong edge, the weak edge is due to the real image. If the weak edge is not connected to a strong edge, the weak edge is due to noise. The specific values of the first edge threshold value and the second edge threshold value can be selected as needed.

[0031] Line detection is performed on the image after Canny edge detection by a general cumulative probability Hough transform, and based on the edge pattern obtained by Canny edge detection (the edge pattern may have a curved part), edge line segments in the image mark are obtained. After line detection, a set of roughly detected edge line segments {d1, d2, d3, … d nObtain {}, and the detection result of the roughly detected edge line segments is shown in FIG. 6. As can be seen from the portion surrounded by the quadrilateral in FIG. 6, after the cumulative probability Hough transform, multiple edge line segments may be detected from the same position.

[0032] In the above embodiment, step S4 includes the following steps. Step S41: Arbitrarily select two line segments from the set of roughly detected edge line segments to obtain four end points. Step S42: Calculate the area of the quadrilateral formed by the four end points, and calculate the slope of one of the line segments and the slopes of the line segments formed by connecting the two end points of one line segment and the two end points of the other line segment respectively. Step S43: If the area of the quadrilateral is less than the first predetermined threshold and the differences in slopes are all less than the second predetermined threshold, it is determined that the two line segments are collinear. Step S44: Check the set of roughly detected edge line segments, leave the collinear line segments among them, and obtain a set of roughly detected collinear line segments.

[0033] Perform a check on the obtained set of roughly detected edge line segments {d1, d2, d3,... d n} to integrate the collinear line segments. Here, the integration means integrating them into the set of roughly detected collinear line segments. Here, duplicate removal for the collinear line segments is not performed, and a set of roughly detected collinear line segments {cl1, cl 2, cl 3, … cl k} is obtained, that is, each element in the set of roughly detected collinear line segments is a collinear set.

[0034] There are many methods for determining collinearity. Hereinafter, the process of collinearity determination according to the present disclosure will be described. Arbitrarily select two line segments from the set of roughly detected edge line segments {d1, d2, d3,... d n}, and the end points of the two line segments are respectively (P 1, P2) and (P 3, P4). Let the collinearity determination conditions be Ψ1 and Ψ2. If both conditions are satisfied, it is determined that they are collinear.

[0035] Ψ1 determines whether the area of the quadrilateral composed of four endpoints is less than a first predetermined threshold. In the present disclosure, Heron's formula is used to directly obtain the area of a triangle based on the lengths of the three sides of the triangle. The area of the quadrilateral is equal to the sum of the areas of two triangles. Refer to the following formula (1). JPEG0007705488000001.jpg27164

[0036] Let S1 represent the area of triangle P1P2P3, h1 represent the semi-perimeter of triangle P1P2P3, and L pipj be the length of the line segment connecting point P i and point P j points, let S2 represent the area of triangle P2P3P4, h2 represent the semi-perimeter of triangle P2P3P4, η be the first predetermined threshold, for example, η = 400. The calculation formulas for h1, h2, and L pipj refer to the following formulas (2) and (3). JPEG0007705488000002.jpg3387 JPEG0007705488000003.jpg19138

[0037] Ψ2 determines whether the slope of a line segment and the difference between the slopes of the line segments formed by connecting the two endpoints of one line segment and the two endpoints of another line segment are less than a second predetermined threshold. For the sake of convenience of explanation, in the present disclosure, the slope is converted and shown as the angle formed by the line segment and the x-axis. By the arctangent function, the slope can be converted into the angle θ1 formed by the line where endpoints P1 and P2 are located and the x-axis, the angle θ2 formed by the line where endpoints P1 and P3 are located and the x-axis, and the angle θ3 formed by the line where endpoints P2 and P4 are located and the x-axis. Refer to Figure 7. The inclination angles are all expressed in radians. Refer to the following formula (4). JPEG0007705488000004.jpg2060

[0038] For example, the second predetermined threshold δ = 0.16. The calculation formulas for the inclination angles θ1, θ2, and θ3 refer to the following formula (5). JPEG0007705488000005.jpg30152

[0039] In the above embodiments, step S5 includes the following steps. Step S51: Arbitrarily select one corner from the set of edge jagged corners, and the corner and the line segments in the set of roughly detected collinear line segments respectively form a triangle. Step S52: Calculate the area of the triangle. Step S53: If the area of the triangle is less than the third predetermined threshold, it is determined that the corner and the line segment are collinear in terms of point and line. Step S54: Check the set of edge jagged corners, and integrate the corners and line segments that are collinear in terms of point and line to obtain a set of accurately detected points and lines.

[0040] Check the set of edge jagged corners, perform point-line collinearity determination in the set of roughly detected collinear line segments, integrate the collinear corners and line segments into the same set of roughly detected collinear line segments, and obtain a set of accurately detected points and lines JPEG0007705488000006.jpg1058 is obtained.

[0041] The condition for point-line collinearity determination is Ψ3. Ψ3 is the corner sp k and the two endpoints (P i , P j ) of the line segment in the set of roughly detected collinear line segments to determine whether the area of the triangle formed by them is less than the third predetermined threshold. For example, the third predetermined threshold η s = 300, and the calculation method of Ψ3 is the same as the calculation formula for the area of the triangle in Ψ1.

[0042] In the above embodiments, step S5 further includes the following steps. Step S55: Identify the same collinear line segments in the set of accurately detected points and lines, leave any one of them, and remove the other overlapping line segments.

[0043] At this time, duplicate line segments in the same collinear line segments in the accurate detection points and line set are removed, and only one of each set of the same collinear line segments is left. As shown in the part surrounded by the quadrilateral in FIG. 6, there may be a plurality of line segments with very close distances to each other at the same position. After performing the collinearity determination in step S54 on these line segments with each other or with other line segments, two line segments with the same inclination may be obtained, and both of these two line segments are stored in the roughly detected collinear line segment set in step S4. In order to remove duplicate redundant data, in this step, by performing an operation to obtain the union of line segments, duplicate roughly detected line segments in the line segment group including the same roughly detected collinear line segments are removed.

[0044] Based on the above embodiments, step S6 includes the following steps. Step S61: Perform linear fitting on the accurate detection points, the corners where the points and lines in the line set are collinear, and the two endpoints of the corresponding line segments to obtain the equation of the line. Step S62: Calculate the inclination angle of each line using the arctangent function based on the equation of the line. Perform linear fitting on the accurate detection points, the corners where the points and lines in the line set are collinear, and the corresponding line segments. For example, perform linear fitting using the least squares method. Since one line segment can be represented by its two endpoints, when the number of all points in the accurate detection points and line set is N, the equation becomes the following formula (6). The fitting result of the accurate detection points and lines is, as shown in FIG. 8, a linear equation in the form of y = ax + b is obtained. Perform the operation of the arctangent function to obtain the accurate linear inclination angle β. Refer to formula (7). JPEG0007705488000007.jpg39124

[0045] In the above technical solution, the detection deviation of the detection method can be calculated. Step S8 includes the following steps. Step S81: Obtain the detection error of each straight line by subtracting the standard deviation angle and the deviation angle of each straight line. Step S82: Obtain the deflection error of the template conveyance by obtaining the average of the detection errors. Step S83: By obtaining the average value and the standard deviation for the deflection errors corresponding to a plurality of standard deviation angles, the deviation of the detection accuracy of this method can be estimated.

[0046] When the template is arranged by the robot, the mark on the template is displaced by a predetermined angle from the standard position, and the displaced angle is used as the standard deviation angle. The deviation angle of each straight line is obtained by subtracting the inclination angle of each straight line from the standard angle. Therefore, the detection error = standard deviation angle - deviation angle of the straight line, and the deflection error is obtained by obtaining the average of the detection errors. By obtaining the average value and the standard deviation for the deflection errors corresponding to a plurality of standard deviation angles, the detection deviation of this method can be estimated. JPEG0007705488000008.jpg31103 JPEG0007705488000009.jpg106 is the deflection error, m is the number of accurate detection points and line sets of each image, JPEG0007705488000010.jpg1310 is the rotation angle of the robot as the standard deviation angle, JPEG0007705488000011.jpg89 is the deviation angle of the straight line obtained by subtracting the inclination angle of each straight line from the standard angle.

[0047] In the above embodiment, the mark on the template includes one of a "rice" shape and a "cross" shape.

[0048] The detection method according to the present disclosure is applicable not only to a "rice" shaped mark but also to a "cross" shaped mark. As long as it is a mark composed of intersecting straight lines and having no curves, the template mark detection can be performed using the detection method according to the present disclosure.

[0049] The present disclosure further provides a template position correction method based on a single camera. The template position correction method based on a single camera includes the following steps. Step S1: Obtain an image of the marks on the template by a single camera, and obtain a binary image through preprocessing. Step S2: Perform edge jaggy corner detection on the binary image to obtain an edge jaggy corner set. Step S3: Perform edge detection and line detection on the binary image in sequence to obtain a rough detection edge line segment set. Step S4: Check and judge the rough detection edge line segment set, and leave the collinear line segments to obtain a rough detection collinear line segment set. Step S5: Check the edge jaggy corner set, and perform line and point collinearity determination on the line segments, points in the rough detection collinear line segment set to obtain an accurate detection point, line set. Step S6: Perform line fitting on the accurate detection point, line set, and calculate the inclination angle of each line by the arctangent function. Step S7: Perform correction on the position of the template based on the difference between the inclination angle of each line and the standard angle.

[0050] Utilize steps S1 to S6 of the above mark detection method, and further perform more accurate correction on the position of the template based on the difference between the inclination angle obtained by mark detection and the standard angle.

[0051] Based on the characteristics of the alignment marks, more accurate inclination can be obtained by using the collinear straight lines on both sides of the communication part. The corner information obtained by corner detection of the grayscale image is relatively small, and edge jaggy occurs in the binary image. By utilizing this feature, more and more accurate edge corners can be obtained during corner detection. In the present disclosure, the deflection error of the template conveyance position is detected by two lines and edge jaggy corners, and the accuracy of error detection is relatively high.

[0052] The present disclosure will be further described using specific embodiments below. In the following examples, a template mark detection method and a template position correction method using a single camera will be specifically described. However, the following examples are merely for illustrating the present disclosure and do not limit the scope of the present disclosure.

[0053] As shown in FIG. 9, the template mark detection method and the template position correction method using a single camera according to the present disclosure include the following steps that are executed in sequence.

[0054] Step 1: An image is acquired by a single CCD camera to obtain an original image (initial image), the acquired original image is converted into a grayscale image, filtering and binarization are performed to obtain a binarized image. This corresponds to step S1 above.

[0055] Step 2: Edge jaggy corner detection is performed on the binarized image to obtain image corner detection data, and an edge jaggy corner set {sp1, sp2, sp3, … sp m} of the image is obtained. This corresponds to step S2 above.

[0056] Step 3: Edge detection is performed on the binarized image, and line detection is performed on the image after edge detection to obtain a rough detection edge line segment set {d1, d2, d3, … d n}. This corresponds to step S3 above.

[0057] Step 4: In the rough detection edge line segment set {d1, d2, d3, … d n}, line segments are checked, collinear line segments are integrated into a rough detection collinear line segment set, and a rough detection collinear line segment set {cl1, cl 2, cl 3, … cl k} is obtained. This corresponds to step S4 above.

[0058] Step 5: Check the set of edge jagged corners, perform point and line collinearity determination in the roughly detected collinear line segment set, integrate collinear corners and line segments into the same roughly detected collinear line segment set, and obtain the precisely detected point and line set. Obtain JPEG0007705488000012.jpg1157. This corresponds to step S5 above.

[0059] Step 6: Perform linear fitting and inverse trigonometric function operations on the precisely detected point and line set to obtain a more accurate inclination angle of each line in the template mark. This corresponds to step S6 above.

[0060] Step 7: Perform correction on the position of the template based on the difference between the inclination angle of each line and the standard angle. This corresponds to step S7 above.

[0061] Based on steps 1 to 7 above, a specific embodiment is provided below.

[0062] Embodiment 1 The implementation steps of the template mark detection method using a single camera according to this embodiment are as follows.

[0063] Step 11: The mark in the template is in a "rice" shape and is arranged 5 times. Perform 5 detections on the pattern mark of the template. Convert the initial image obtained by a single CCD camera into a grayscale image and perform Blur filtering. The size of the filtering block is variable. In this embodiment, filtering is performed using a 5×5 pixel one. Also tried filtering blocks of 3×3 pixels and 7×7 pixels. As a result, in this case, the accuracy of the inclination angle of the detected mark line decreased, but a relatively accurate inclination angle of the mark line can be obtained.

[0064] Step 12: Check all pixel points in the image, calculate the median pixel value of this grayscale image, perform binarization on all pixel values based on the median pixel value, and obtain a binarized image. In this embodiment, pixel values smaller than the median are set to 255, and pixel values larger than the median are set to 0 (in OpenCV, 0 is black and 255 is white). The result is shown in FIG. 10. (a) to (e) in FIG. 10 respectively correspond to the binarization processing results after 5 times of template placement. As can be seen from FIG. 10, a plurality of jagged edges are formed in the alignment mark pattern after the binarization processing.

[0065] Step 13: Perform edge jagged corner detection on the binarized image, obtain the data of Shi-Tomasi corner detection for the image, and obtain an edge jagged corner set {sp1, sp2, sp3,... sp m}, and the result of the edge jagged corner detection is shown in FIG. 11. (a) to (e) in FIG. 11 respectively correspond to the edge jagged corner detection results after 5 times of template placement. To obtain sufficient points, the quality coefficient in the Shi-Tomasi corner detection algorithm is made as small as possible. In this embodiment, the quality coefficient is set to 0.01, and the minimum Euclidean distance between corners is set to 20, which means that the minimum distance between the finally remaining corners is 20 pixels. The specified neighborhood range when calculating the derivative autocorrelation matrix is 3. As can be seen from FIG. 11, corners on a straight line inclined in the coordinate system cannot be detected by Shi-Tomasi corner detection.

[0066] Step 14: Perform Canny edge detection on the binary image. In this embodiment, since there are no curves or irregular protrusions in the "rice" shape and the pattern is simple and clean, the selected first edge threshold is 1 and the second edge threshold is 10. According to a relatively small threshold, more edge information can be obtained. Perform cumulative probability Hough transform on the image after Canny edge detection. In this embodiment, the selected collinearity threshold is 80, that is, in the polar coordinate system, when the number of curves intersecting at one point exceeds 80, it is determined that the edge points in the Cartesian coordinate system corresponding to these 80 or more curves are collinear, and these collinear edge points together form an edge straight line. Also, to avoid too much data, the minimum value of the length of the edge straight line is set to 50, and line segments shorter than this length are not left or displayed. The Euclidean distance (straight-line distance) allowing the connection of two points is set to 10 pixels. Obtain the set of roughly detected edge line segments {d1, d2, d3, … d n}, and the detection result of the roughly detected edge line segments is shown in FIG. 12. (a) to (e) in FIG. 12 respectively correspond to the results of Canny edge detection and straight-line detection after 5 times of template placement.

[0067] Step 15: Check the line segments in the obtained set of roughly detected edge line segments {d1, d2, d3, … d n}, integrate the collinear line segments into the set of roughly detected collinear line segments, and obtain the set of roughly detected collinear line segments {cl1, cl 2, cl 3, … cl k}.

[0068] Arbitrarily select two line segments from the set of roughly detected edge line segments {d1, d2, d3, … d n}, and the endpoints of the two line segments are respectively (P 1, P2) and (P 3,It is P4), and the collinearity determination conditions are Ψ1 and Ψ2. If both conditions are satisfied, it is determined to be collinear. Ψ1 determines whether the area of the quadrilateral formed by the four endpoints is less than the first predetermined threshold. In this embodiment, Heron's formula is used to directly obtain the area of a triangle based on the lengths of the three sides of the triangle. Ψ2 determines whether the difference between the slope of one line segment and the slopes of the line segments formed by connecting the two endpoints of one line segment to the two endpoints of the other line segment is less than the second predetermined threshold. In this embodiment, the first predetermined threshold η = 400 and the second predetermined threshold δ = 0.16.

[0069] Step 16: Check the edge jaggy corner set, perform point-line collinearity determination in the roughly detected collinear line segment set, integrate the collinear corners and line segments into the same roughly detected collinear line segment set, and obtain the accurately detected point-line set Obtain JPEG0007705488000013.jpg1157. The condition for point-line collinearity determination is Ψ3. Ψ3 is the corner sp k and the two endpoints (P i , P j ) of the line segment in the roughly detected collinear line segment set to determine whether the area of the triangle formed by them is less than the third predetermined threshold. In this embodiment, the third predetermined threshold η s = 300, and the calculation method of Ψ3 is the same as the formula for calculating the area of the triangle in Ψ1.

[0070] Step 17: Perform linear fitting on the corners and corresponding line segments where the points and lines in the accurately detected point-line set are collinear. For example, use the least squares method to perform linear fitting. The fitting results of the accurately detected points and lines are shown in FIG. 13. (a) to (e) in FIG. 13 respectively correspond to the linear fitting results after five arrangements. A plurality of linear equations in the form of y = ax + b are obtained from each image, and the accurate linear inclination angle β is obtained through the operation of the arctangent function.

[0071] Step 18: This step verifies the detection deviation existing in the above detection method. By subtracting the inclination angle of each detected straight line from the standard angle (the desired angle after the template is arranged), the deviation angle of the straight line is obtained. By subtracting the inclination angle of each straight line at the actual position after the template is arranged from the standard angle (the desired angle after the template is arranged), the standard deviation angle is obtained. Therefore, the detection error = standard deviation angle - straight line deviation angle. By obtaining the average of the detection errors, the deflection error is obtained. By obtaining the average value and standard deviation for the deflection errors corresponding to multiple standard deviation angles, the detection deviation of this method can be estimated. The detection errors and deflection errors are shown in Table 1. As can be seen from Table 1, the average value of the detection errors is the deflection error.

Table 1

[0072] By obtaining the average value and standard deviation for the multiple detected deflection errors, the detection deviation of this method can be estimated. As shown in Table 2, since the detection deviation of this method is [-0.00381 ± 0.01259], it meets the accuracy requirements for template conveyance.

Table 2

[0073] After obtaining the inclination angle of each straight line of the template mark by the above method, based on the angle by which the mark deviates from the standard angle, the template is corrected for the angle to position the template mark at the desired arrangement position. Thereby, the template can be arranged at the desired arrangement position.

[0074] As described above, the present disclosure converts the acquired image into a grayscale image, performs Blur filtering and binarization, and performs edge jaggy corner detection and line detection in parallel. Then, collinearity detection is performed on the obtained rough detection line segments, leaving two or more collinear line segments, obtaining a set of rough detection collinear line segments. Then, corner data is checked, point and line collinearity determination is performed in the set of rough detection collinear line segments, collinear corners are integrated, obtaining an accurate detection point and line set. Then, linear fitting and inverse trigonometric function operations are performed on each accurate detection point and line set to obtain a more accurate inclination angle of the straight line. Based on the difference between the inclination angle of each straight line and the standard angle, high-precision correction is performed on the position of the template.

[0075] The above specific embodiments further elaborate on the objectives, technical solutions, and beneficial effects of the present disclosure. The above are only specific embodiments of the present disclosure and do not limit the present disclosure. Any changes, equivalent substitutions, improvements, etc. made without departing from the spirit and principle of the present disclosure shall fall within the protection scope of the present disclosure.

Claims

1. A step S1 of acquiring an image of a mark in a template using a single camera and obtaining a binary image through preprocessing; A step S2 of performing edge jaggy corner detection on the binary image to obtain an edge jaggy corner set; A step S3 of sequentially performing edge detection and line detection on the binary image to obtain a rough detection edge line segment set; A step S4 of checking and determining the rough detection edge line segment set, leaving collinear line segments, and obtaining a rough detection collinear line segment set; A step S5 of checking the edge jaggy corner set, performing collinearity determination on the corners in the edge jaggy corner set and the line segments in the rough detection collinear line segment set, and obtaining an accurate detection point and line set; A step S6 of performing linear fitting on the accurate detection point and line set to obtain a line by fitting, calculating the inclination angle of each line by an arctangent function, and ending the detection of the mark, including A method for detecting a template mark using a single camera, characterized in that.

2. The step S1 includes: A step S11 of acquiring an image of a mark in a template using a single camera to obtain an initial image; A step S12 of converting the initial image into a grayscale image and performing filtering; A step S13 of performing binarization processing on the image obtained in step S12 to obtain a binary image, including A method for detecting a template mark using a single camera according to claim 1, characterized in that.

3. The step S2 includes performing detection on pixel points in the binary image by a Shi-Tomasi corner response function to obtain an edge jaggy corner set. A method for detecting a template mark using a single camera according to claim 1, characterized in that.

4. The step S3 includes: A step of performing Canny edge detection on the binary image to obtain an edge pattern; A step of obtaining a plurality of edge line segments in the mark by cumulative probabilistic Hough transform based on the edge pattern, and obtaining the rough detection edge line segment set composed of the plurality of edge line segments, including A method for detecting a template mark using a single camera according to claim 1, characterized in that.

5. The step S4 includes: A step S41 of arbitrarily selecting two line segments from the rough detection edge line segment set to obtain four endpoints; Calculate the area of the quadrilateral formed by the four endpoints, and calculate the slope of one of the line segments and the slopes of the line segments formed by connecting the two endpoints of one line segment and the two endpoints of the other line segment respectively in step S42; In the case where the area of the quadrilateral is less than the first predetermined threshold and all the differences in the slopes are less than the second predetermined threshold, determine in step S43 that the two line segments are collinear; Check the set of roughly detected edge line segments, leave the collinear line segments among them, and obtain a set of roughly detected collinear line segments in step S44, including The method for detecting a template mark by a single camera according to claim 1, characterized in that.

6. Step S5 is Arbitrarily select one corner from the set of edge jagged corners, and in step S51, the corner and the line segments in the set of roughly detected collinear line segments respectively form triangles; Calculate the area of the triangle in step S52; In the case where the area of the triangle is less than the third predetermined threshold, determine in step S53 that the corner and the line segment are collinear in terms of points and lines; Check the set of edge jagged corners, and integrate the corners and line segments that are collinear in terms of points and lines to obtain a set of accurately detected points and lines in step S54, including The method for detecting a template mark by a single camera according to claim 1, characterized in that.

7. Step S5 further includes step S55 of identifying the same collinear line segments in the set of accurately detected points and lines, leaving any one of them, and removing the other overlapping line segments The method for detecting a template mark by a single camera according to claim 6, characterized in that.

8. Step S6 is Perform linear fitting on the corners and the two endpoints of the corresponding line segments that are collinear in the set of accurately detected points and lines to obtain the equation of the line in step S61; Calculate the inclination angle of each line based on the equation of the line using the arctangent function in step S62, including The method for detecting a template mark by a single camera according to claim 1, characterized in that.

9. The mark in the template includes one of a "rice" shape and a "cross" shape The method for detecting a template mark by a single camera according to claim 1, characterized in that.

10. Perform image acquisition on the mark in the template by a single camera, and obtain a binary image through preprocessing in step S1; Step S2 of performing edge jaggy corner detection on the binary image to obtain an edge jaggy corner set; Step S3 of sequentially performing edge detection and line detection on the binary image to obtain a rough detection edge line segment set; Step S4 of checking and judging the rough detection edge line segment set to leave collinear line segments and obtain a rough detection collinear line segment set; Step S5 of checking the edge jaggy corner set, performing collinearity determination on the corners in the edge jaggy corner set and the line segments in the rough detection collinear line segment set, and obtaining an accurate detection point and line set; Step S6 of performing linear fitting on the accurate detection point and line set to obtain a line by fitting, and calculating the inclination angle of each line by an arctangent function; Step S7 of correcting the position of the template based on the difference between the inclination angle of each line and the standard angle, including A template position correction method based on a single camera, characterized in that.

Images