Method for detecting template marks and correcting template position using a single camera

Abstract

The present disclosure provides a method for detecting a template mark using a single camera and a method for correcting a template position, which includes the following steps: Step S1: An image of a mark in a template is captured using a single camera, and a binary image is obtained by preprocessing. Step S2: An edge jaggy corner detection is performed on the binary image, and an edge jaggy corner set is obtained. Step S3: An edge detection and a line detection are performed on the binary image in order, and a roughly detected edge line segment set is obtained. Step S4: The roughly detected edge line segment set is checked, and the collinear line segments are left by judging, and a roughly detected collinear line segment set is obtained. Step S5: The edge jaggy corner set is checked, and a line collinearity judgment is performed on the line segments and points in the roughly detected collinear line segment set, and a precisely detected point and a line set are obtained. Step S6: A line fitting is performed on the precisely detected points and the line set, and the inclination angle of each line is calculated by the arctangent function, and the detection of the mark is completed. Furthermore, the position of the template is corrected.

Description

[Technical field]

[0001] The present disclosure relates to the field of optical detection technology, and more particularly to a method for detecting template marks and correcting template position using a single camera.

[0002] (CROSS REFERENCE TO RELATED APPLICATIONS) This disclosure claims priority to a Chinese application having application number 202211651503.6, filed on December 21, 2022, the entire contents of which are incorporated herein by reference. [Background technology]

[0003] In the process of transporting the template by the robot, position correction is required. Generally, two PSDs (Position Sensitive Detectors) are used for optical correction or two cameras are used for image correction. However, when the space above and below the template suction cup is limited, there is not enough space to install the PSD and the camera. When detecting a line for a position mark by an optical method, the inclination of a line is constant and it is difficult to correct the inclination, so the accuracy of the inclination of the mark obtained by line detection is low. In addition, the difference between the pixel values ​​of adjacent pixel points in a grayscale image is relatively small, so the number of corners obtained by corner detection of the grayscale image is relatively small, and the position information of the mark cannot be determined using the obtained corners. Summary of the Invention [Problem to be solved by the invention]

[0004] In response to the above problems, the present disclosure provides a template mark detection method and template position correction method using a single camera that solves technical problems in conventional methods, such as limited installation of a detection device and low accuracy of the tilt angle of the resulting mark. [Means for solving the problem]

[0005] One aspect of the present disclosure provides a method for detecting template marks using a single camera, which includes the steps of: acquiring an image of a mark in a template using a single camera, obtaining a binary image through preprocessing in step S1; detecting edge jaggy corners on the binary image to obtain a set of edge jaggy corners in step S2; detecting edges and lines on the binary image in order to obtain a set of roughly detected edge lines; checking the set of roughly detected edge lines, judging whether any lines are collinear, and obtaining a set of roughly detected collinear lines in step S4; checking the set of edge jaggy corners, judging whether any lines are collinear between the lines and points in the set of roughly detected collinear lines, and obtaining a set of precisely detected points and lines; and fitting lines to the precisely detected points and the set of lines, and calculating the inclination angle of each line by an arctangent function to complete the detection of the mark in step S6.

[0006] Furthermore, step S1 includes step S11 of performing image capture of a mark on the template using a single camera to obtain an initial image, step S12 of converting the initial image into a grayscale image and filtering it, and step S13 of performing a binarization process on the image obtained in step S12 to obtain a binary image.

[0007] Further, step S2 includes performing detection using a Shi-Tomasi corner response function for pixel points in the binarized image to obtain a set of edge jaggy corners.

[0008] Furthermore, step S3 includes a step of performing Canny edge detection on the binarized image to obtain an edge pattern, and a step of obtaining a plurality of edge segments in the mark by a cumulative probabilistic Hough transform based on the edge pattern, and obtaining a set of roughly detected edge segments consisting of the plurality of edge segments.

[0009] Furthermore, step S4 includes step S41 of arbitrarily selecting two line segments from the roughly detected edge line segment set and obtaining four end points; step S42 of calculating the area of ​​a rectangle formed by the four end points and calculating the slope of one of the line segments and the slope of the line segments connecting the two end points of the one line segment and the two end points of the other line segment; step S43 of determining that the two line segments are collinear if the area of ​​the rectangle is less than a first predetermined threshold and the difference in slope is both less than a second predetermined threshold; and step S44 of checking the roughly detected edge line segment set, retaining the collinear line segments therein, and obtaining a roughly detected collinear line segment set.

[0010] Further, step S5 includes step S51 of arbitrarily selecting a corner from the edge jaggy corner set, and forming a triangle between the corner and a line segment in the roughly detected collinear line segment set; step S52 of calculating the area of ​​the triangle; step S53 of determining that the corner and the line segment are collinear points and lines if the area of ​​the triangle is less than a third predetermined threshold; and step S54 of checking the edge jaggy corner set and integrating the corners and line segments whose points and lines are collinear to obtain a precisely detected point and line set.

[0011] Furthermore, step S5 further includes step S55 of identifying the same collinear line segments in the line set as the precise detection points, keeping any one of them, and removing the other overlapping line segments.

[0012] Furthermore, step S6 includes step S61 of performing line fitting on the precise detection points, the points in the line set, the corners where the lines are collinear, and the two end points of the corresponding line segments to obtain line equations; and step S62 of calculating the inclination angles of each line by using an arctangent function based on the line equations.

[0013] Furthermore, the marks in the template include one of a "R" shape and a "C" shape.

[0014] Another aspect of the present disclosure provides a template position correction method based on a single camera, which includes the steps of: acquiring an image of a mark on a template by a single camera, obtaining a binary image by preprocessing in step S1; detecting edge jaggy corners on the binary image to obtain an edge jaggy corner set in step S2; detecting edges and lines on the binary image in sequence to obtain a roughly detected edge line segment set in step S3; checking the roughly detected edge line segment set, judging and leaving collinear lines, and obtaining a roughly detected collinear line segment set in step S4; checking the edge jaggy corner set, performing line collinearity judgment on the lines and points in the roughly detected collinear line segment set, and obtaining a precisely detected point and line set in step S5; performing line fitting on the precisely detected points and line set, and calculating the inclination angle of each line by an arctangent function in step S6; and correcting the position of the template based on the difference between the inclination angle of each line and a standard angle in step S7. Effect of the Invention

[0015] The method for detecting template marks using a single camera and the method for correcting the position of the template according to the present disclosure use a single camera to capture images, which can reduce the space required for the station and solve the problem of limited installation of the detection device. In addition, the binarization of the image is used to obtain more and more accurate edge corners, and the collinear straight lines on both sides of the connecting part of the template mark are used to perform straight line fitting to obtain more accurate inclination angles of the straight lines, and the accurate position information of the mark is determined, thereby realizing high-precision detection of the template mark, and the position of the template can be corrected according to the position information of the mark obtained by the detection method. [Brief description of the drawings]

[0016] [Figure 1] 1 is a schematic flow chart of a method for detecting template marks using a single camera according to an embodiment of the present disclosure. [Diagram 2]FIG. 2 is a schematic block diagram of a station of an apparatus used in a detection method according to an embodiment of the present disclosure. [Diagram 3] FIG. 13 is a schematic diagram of a pattern after filtering a mark image according to an embodiment of the present disclosure. [Figure 4] FIG. 13 is a schematic diagram of a pattern after performing binarization processing on a grayscale image according to an embodiment of the present disclosure. [Diagram 5] 10A and 10B are schematic diagrams illustrating results of edge jaggy corner detection performed on a binarized image according to an embodiment of the present disclosure. [Figure 6] 10 is a schematic diagram showing a result of edge detection and straight line detection performed on a binarized image according to an embodiment of the present disclosure; FIG. [Figure 7] FIG. 13 is a schematic diagram illustrating included angles when making collinearity determinations according to an embodiment of the present disclosure. [Figure 8] 13 is a schematic diagram showing the result of performing straight line fitting on a set of precisely detected points and lines according to an embodiment of the present disclosure; FIG. [Figure 9] 1 is a schematic flow chart of a process of single camera based template mark detection according to an embodiment of the present disclosure. [Figure 10] 11A to 11C are schematic diagrams illustrating binarization results for each of five template arrangements according to the first embodiment of the present disclosure. [Figure 11] FIG. 13 is a schematic diagram of the results of edge jaggy corner detection on binarized images for each of five template placements according to the first embodiment of the present disclosure. [Figure 12] FIG. 13 is a schematic diagram showing the results of edge detection and line detection for binarized images for each of five template arrangements according to the first embodiment of the present disclosure. [Figure 13] FIG. 13 is a schematic diagram of the results of straight line fitting to the accurate detection points and line sets for each of the five template placements according to the first embodiment of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0017] In order to make the objectives, technical solutions and advantages of the present disclosure more clear, the present disclosure will be described in more detail hereinafter with reference to the drawings using specific embodiments.

[0018] The terms used herein are merely for the purpose of describing specific embodiments and are not intended to limit the present disclosure. As used herein, the terms "comprises", "comprises", and the like refer to the presence of at least one of the indicated features, steps, operations, and components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0019] When a directional orientation is described in an embodiment of the present disclosure, the directional orientation merely indicates the relative positional relationship, movement status, etc. of each member in a certain posture, and when the certain posture changes, the directional orientation also changes accordingly.

[0020] The ordinal numbers such as "first," "second," and "third" used in the specification and claims are intended to modify the corresponding parts, and do not mean that the parts have ordinal numbers, nor do they represent the order of one part relative to another, or the order in a manufacturing process. These ordinal numbers are merely intended to clearly distinguish parts with the same name.

[0021] The present disclosure provides a method for detecting template marks using a single camera. As shown in FIG. 1, the method for detecting template marks using a single camera includes the following steps: Step S1: An image of a mark in a template is captured using a single camera, and a binarized image is obtained by preprocessing. Step S2: An edge jaggy corner detection is performed on the binarized image, and an edge jaggy corner set is obtained. Step S3: An edge detection and a line detection are sequentially performed on the binarized image, and a roughly detected edge line segment set is obtained. Step S4: The roughly detected edge line segment set is checked, and the collinear line segments are left by judging, and a roughly detected collinear line segment set is obtained. Step S5: The edge jaggy corner set is checked, and a line collinearity judgment is performed on the line segments and points in the roughly detected collinear line segment set, and a precisely detected point and line set are obtained. Step S6: A line fitting is performed on the precisely detected points and line set, and the inclination angle of each line is calculated by the arctangent function, and the detection of the mark is completed.

[0022] The station diagram of the device used in the detection method is shown in Figure 2. The device is equipped with a light source, a lens group, and a CCD camera, and the work process is as follows: A robot transports a template to the template station below the CCD camera, and the light from the light source reaches the mark on the template through the lens group, and the reflected light carrying the mark information reaches the CCD camera through the lens group again, and then the image collected by the CCD camera is processed. This method uses a single CCD camera to capture images, which reduces the space used in the station and solves the problem of limited installation of the detection device. The template station is provided with a square clamping groove on all four sides, so that the rotation range of the template in the square groove is less than 0.5° and the movement range is less than ±0.5mm. Within this range, the accuracy of identifying the position of the image mark captured by the single CCD camera is high, and the position accuracy of the template after the position correction is also high accordingly.

[0023] Specifically, first, the acquired image is pre-processed and binarized, and edge jaggy occurs in the binarized image. By utilizing this feature, more and more accurate edge jaggy corners can be obtained during corner detection. Then, edge jaggy corner detection and line detection are performed in parallel (steps S2 and S3 may or may not be performed simultaneously, and the order of execution is not limited), and collinear detection is performed on the obtained roughly detected edge lines, leaving two or more collinear lines, and a roughly detected collinear line set is obtained. Then, the edge jaggy corner set is checked, and point and line collinearity determination is performed on the roughly detected collinear line set, leaving collinear edge jaggy corners, and an accurately detected point and line set are obtained. Furthermore, line fitting and inverse trigonometric function calculation are performed on the accurately detected point and line set to obtain a more accurate inclination angle of the line. According to the above processing process, highly accurate detection of the template mark can be realized. That is, the present disclosure utilizes image binarization to obtain more and more accurate edge corners, and further uses collinear straight lines on both sides of the connecting portion of the template mark to perform straight line fitting to obtain a more accurate straight line inclination angle, thereby obtaining more accurate position information of the mark.

[0024] In the above embodiment, step S1 includes the following steps: Step S11: Perform image capture of the mark in the template by a single camera to obtain an initial image; Step S12: Convert the initial image into a grayscale image and perform filtering; Step S13: Perform binarization processing on the image obtained in step S12 to obtain a binarized image.

[0025] The CCD camera captures the mark on the template to obtain an initial image. The initial image is converted into a grayscale image and filtered, with a variable size of the filtering block, as shown in FIG. 3 (illustrated by the "rice" mark). The reason for filtering is that the edges of the image are highly susceptible to noise interference, so in order to prevent false detection of edge information, it is necessary to filter the image to remove noise. The purpose of filtering is to smooth the non-edge areas with relatively weak texture to obtain more accurate edges.

[0026] Then, all pixel points in the grayscale image are checked, the pixel median value of the grayscale image is calculated, and all pixel values ​​are binarized based on the pixel median value. For example, pixel values ​​smaller than the median value are set to 255, and pixel values ​​larger than the median value are set to 0. Of course, other values ​​can be used, but the difference between the two selected values ​​should be large enough to distinguish the image from the blank area. In general, image binarization refers to making the grayscale value of pixel points on an image either 0 or 255. The result is shown in FIG. 4, and as can be seen, multiple jaggies, indicated by circles in FIG. 4, are formed on the alignment mark pattern edge after binarization.

[0027] In the above embodiment, step S2 includes performing Shi-Tomasi corner response function detection on pixel points in the binarized image to obtain a set of edge jaggy corners.

[0028] Edge jaggy corner detection is performed on the binarized image to obtain Shi-Tomasi corner detection data, and the edge jaggy corner set of the image {sp1,sp2,sp3,...sp m} is obtained, and the result of edge jaggy corner detection is shown in Fig. 5. In order to obtain enough points, the mass coefficient in the Shi-Tomasi corner response function is made as small as possible.

[0029] In the above embodiment, step S3 includes a step of performing Canny edge detection on the binarized image to obtain an edge pattern, and a step of obtaining a plurality of edge segments in the mark by a cumulative probabilistic Hough transform based on the edge pattern, and obtaining a set of roughly detected edge segments consisting of the plurality of edge segments.

[0030] The binarized image is subjected to edge detection by a general Canny edge detection algorithm, and the principle of selecting specific values ​​in the Canny edge detection algorithm is that in the process of obtaining edges, some weak edges are obtained, and these weak edges are either due to real images or due to noise, and it is necessary to remove the weak edges due to noise. The attribute of the current edge is determined based on the relationship between the gradient of the pixel of the current edge and two thresholds, and if the gradient of the pixel of the current edge is equal to or greater than the second edge threshold, the pixel of the current edge is marked as a strong edge, that is, the edge is due to a real image. If the gradient of the pixel of the current edge is between the second edge threshold and the first edge threshold, the pixel of the current edge is marked as a weak edge (this step is still left). If the gradient of the pixel of the current edge is equal to or less than the first edge threshold, the current edge is rejected. After obtaining the weak edge by the above process, it is determined whether the weak edge is connected to a strong edge to determine whether the weak edge is due to a real image or due to noise. If the weak edge is connected to a strong edge, the weak edge is due to a real image. If a weak edge is not connected with a strong edge, the weak edge is due to noise.The specific values ​​of the first edge threshold and the second edge threshold can be selected according to needs.

[0031] After the Canny edge detection, the image is subjected to line detection using a general cumulative probability Hough transform, and edge segments in the image mark are obtained based on the edge pattern obtained by the Canny edge detection (the edge pattern may have curved parts). After the line detection, a set of roughly detected edge segments {d1, d2, d3, ... d n} is obtained, and the detection result of the roughly detected edge segments is shown in Fig. 6. As can be seen from the rectangular area in Fig. 6, after the cumulative probabilistic Hough transform, multiple edge segments may be detected from the same position.

[0032] In the above embodiment, step S4 includes the following steps: Step S41: Two line segments are arbitrarily selected from the roughly detected edge line segment set, and four end points are obtained. Step S42: The area of ​​a rectangle formed by the four end points is calculated, and the slope of one of the line segments and the slope of a line segment formed by connecting two end points of one line segment with two end points of another line segment are calculated. Step S43: If the area of ​​the rectangle is less than a first predetermined threshold and the difference in slope is both less than a second predetermined threshold, it is determined that the two line segments are collinear. Step S44: The roughly detected edge line segment set is checked, and the collinear line segments are kept to obtain a roughly detected collinear line segment set.

[0033] The obtained rough edge segment set {d1, d2, d3, ... d n}, check the lines and merge the collinear lines. The merge here means merging into a roughly detected collinear line set. Here, overlap removal is not performed for the collinear lines, and the roughly detected collinear line set {cl1,cl 2, cl 3, …cl k} is obtained, that is, each element in the roughly detected collinear segment set is a collinear set.

[0034] There are many ways to determine collinearity, and the process of determining collinearity according to the present disclosure is described below. n}, select two lines arbitrarily, and set the endpoints of the two lines to (P 1, P2) and (P 3, P4), where the conditions for determining collinearity are Ψ1 and Ψ2, and if both conditions are met, the data are determined to be collinear.

[0035] Ψ1 determines whether the area of ​​the quadrangle formed by the four end points is less than a first predetermined threshold. In this disclosure, Heron's formula is used to directly calculate the area of ​​the triangle based on the lengths of the three sides of the triangle. The area of ​​the quadrangle is equal to the sum of the areas of the two triangles, and see the following formula (1). JPEG2025501823000002.jpg27164

[0036] S1 represents the area of ​​triangle P1P2P3, h1 represents the semi-perimeter of triangle P1P2P3, and L pipj is point P i and point P j h1, h2 and L represent the length of the line segment connecting the points, S2 represents the area of ​​the triangle P2P3P4, h2 represents the semi-perimeter of the triangle P2P3P4, and η is a first predetermined threshold, for example, η=400. pipj The calculation formula is shown in the following formulas (2) and (3). JPEG2025501823000003.jpg3387 JPEG2025501823000004.jpg19138

[0037] Ψ2 determines whether the difference between the slope of one line segment and the slope of a line segment formed by connecting two end points of one line segment and two end points of another line segment is less than a second predetermined threshold. For convenience of explanation, in this disclosure, the slope is converted to an angle between the line segment and the x-axis. Using the arctangent function, the slope can be converted to an angle θ1 between the line on which the end points P1 and P2 are located and the x-axis, an angle θ2 between the line on which the end points P1 and P3 are located and the x-axis, and an angle θ3 between the line on which the end points P2 and P4 are located and the x-axis. See FIG. 7. The slope angles are all expressed in radians and are shown in the following formula (4). JPEG2025501823000005.jpg2060

[0038] For example, the second predetermined threshold value δ=0.16, and the inclination angles θ1, θ2, and θ3 are calculated by referring to the following formula (5). JPEG2025501823000006.jpg30152

[0039] In the above embodiment, step S5 includes the following steps: Step S51: Randomly select a corner from the edge jaggy corner set, and the corner and the line segments in the roughly detected collinear line segment set respectively form a triangle. Step S52: Calculate the area of ​​the triangle. Step S53: If the area of ​​the triangle is less than a third predetermined threshold, it is determined that the corner and the line segment are collinear points and lines. Step S54: Check the edge jaggy corner set, and merge the corners and line segments that are collinear points and lines to obtain a precisely detected point and line set.

[0040] Check the edge jaggy corner set, perform point and line collinearity judgment on the roughly detected collinear line segment set, merge the collinear corners and lines into the same roughly detected collinear line segment set, and accurately detect the point and line set. I get JPEG2025501823000007.jpg1058.

[0041] The condition for determining point and line collinearity is Ψ3. Ψ3 is the corner sp k and the two end points of the line segments in the roughly detected collinear line segment set (P i ,P j ) is less than a third predetermined threshold value. For example, s =300, the calculation method for Ψ3 is the same as the formula for calculating the area of ​​the triangle in Ψ1.

[0042] In the above embodiment, step S5 further includes the following steps: Step S55: Identify the same collinear line segments in the line set as the precise detection points, keep any one of them, and remove the other overlapping line segments.

[0043] At this time, the overlapping line segments in the same collinear line segments in the line set at the precise detection point are removed, and only one of the same collinear line segments in each set is left. As shown in the rectangular area in Fig. 6, there may be a plurality of line segments that are very close to each other at the same position, and after performing the collinearity determination in step S54 for these line segments or these line segments and other line segments, two line segments with the same slope may be obtained, and both of these line segments are stored in the roughly detected collinear line segment set in step S4, and in order to remove redundant overlapping data, the overlapping roughly detected line segments in the line segment group containing the same roughly detected collinear line segments are removed by calculating the union of the line segments in this step.

[0044] Based on the above embodiment, step S6 includes the following steps: Step S61: perform line fitting on the precise detection points, points in the line set, corners where the lines are collinear, and two end points of the corresponding line segments to obtain the line equations; Step S62: calculate the inclination angle of each line by using the arctangent function according to the line equations. Line fitting is performed for the precise detection points, points in the line set, the corners where the lines are collinear, and the corresponding line segments, for example, using the least squares method to perform line fitting. Since a line segment can be represented by its two end points, when the number of all points in the precise detection points and line set is N, the equation is the following Equation (6). The fitting result of the precise detection points and lines is shown in Figure 8, where a straight line equation of the form y=ax+b is obtained, and the arctangent function is calculated to obtain the precise straight line inclination angle β, which is shown in Equation (7). JPEG2025501823000008.jpg39124

[0045] In the above technical solution, the detection deviation of the detection method can be calculated. Step S8 includes the following steps: Step S81: The detection error of each straight line is obtained by subtracting the standard deviation angle of each straight line from the deviation angle. Step S82: The deflection error of the template conveyance is obtained by averaging the detection errors. Step S83: The deviation of the detection accuracy of the method can be estimated by obtaining the average value and standard deviation for the deflection errors corresponding to a number of standard deviation angles.

[0046] When the template is arranged by the robot, the mark on the template is shifted from the standard position by a certain angle, and the shifted angle is taken as the standard deviation angle, and the deviation angle of each straight line is obtained by subtracting the inclination angle of each straight line from the standard angle, so that the detection error = standard deviation angle - straight line deviation angle, and the deviation error is obtained by averaging the detection errors. The detection deviation of this method can be estimated by calculating the average value and standard deviation of the deflection errors corresponding to multiple standard deviation angles. JPEG2025501823000009.jpg31103 JPEG2025501823000010.jpg106 is the deviation error, m is the number of accurately detected points and line sets in each image, JPEG2025501823000011.jpg1310 is the robot's rotation angle as the standard deviation angle, JPEG2025501823000012.jpg89 is the deviation angle of the line obtained by subtracting the inclination angle of each line from the standard angle.

[0047] In the above embodiment, the marks in the template include one of a "R" shape and a "C" shape.

[0048] The detection method according to the present disclosure can be applied not only to the "*" mark but also to the "+" mark. As long as the mark is composed of intersecting straight lines and has no curves, the detection method according to the present disclosure can be used to detect the template mark.

[0049] The present disclosure further provides a template position correction method based on a single camera. The template position correction method based on a single camera includes the following steps: Step S1: An image is captured for a mark in a template by a single camera, and a binarized image is obtained by preprocessing. Step S2: An edge jaggy corner detection is performed for the binarized image, and an edge jaggy corner set is obtained. Step S3: An edge detection and a line detection are sequentially performed for the binarized image, and a roughly detected edge line segment set is obtained. Step S4: The roughly detected edge line segment set is checked, and a collinear line segment is left by judging, and a roughly detected collinear line segment set is obtained. Step S5: The edge jaggy corner set is checked, and a line collinearity judgment is performed for the line segments and points in the roughly detected collinear line segment set, and a precisely detected point and a line set are obtained. Step S6: A line fitting is performed for the precisely detected points and the line set, and the inclination angle of each line is calculated by an arctangent function. Step S7: A correction is performed on the position of the template based on the difference between the inclination angle of each line and a standard angle.

[0050] Using steps S1 to S6 of the above-mentioned mark detection method, more precise correction is made to the position of the template based on the difference between the tilt angle obtained by mark detection and the standard angle.

[0051] In the present disclosure, based on the characteristics of the alignment mark, the collinear straight lines on both sides of the connecting portion can be used to obtain a more accurate inclination, and the corner information obtained by corner detection of the grayscale image is relatively small, and edge jaggies occur in the binarized image, and by utilizing this characteristic, more and more accurate edge corners can be obtained during corner detection. In the present disclosure, the deflection error detection of the template conveying position is performed using two lines and edge jaggies corners, and the accuracy of error detection is relatively high.

[0052] The present disclosure will be further described below using specific embodiments. In the following examples, the template mark detection method and template position correction method using a single camera are specifically described. However, the following examples are merely for illustrating the present disclosure and are not intended to limit the scope of the present disclosure.

[0053] As shown in FIG. 9, the method for detecting template marks and correcting template position using a single camera according to the present disclosure includes the following steps performed in order:

[0054] Step 1: An image is captured by a single CCD camera to obtain an original image (initial image), and the captured original image is converted into a grayscale image, filtered, and binarized to obtain a binarized image. This corresponds to step S1 above.

[0055] Step 2: Perform edge jaggy corner detection on the binarized image to obtain corner detection data of the image, and set edge jaggy corners of the image {sp1, sp2, sp3, … sp m} This corresponds to step S2 above.

[0056] Step 3: Detect edges in the binarized image, and then detect lines in the edge-detected image to obtain a set of roughly detected edge segments {d1, d2, d3, ... d n} This corresponds to step S3 above.

[0057] Step 4: Roughly detect edge segments {d1, d2, d3, … d n}, and merge the collinear segments into a roughly detected collinear segment set {cl1,cl 2, cl 3, …cl k} This corresponds to step S4 above.

[0058] Step 5: Check the edge jaggy corner set, perform point and line collinearity judgment in the roughly detected collinear line segment set, merge the collinear corners and lines into the same roughly detected collinear line segment set, and then precisely detect the point and line set. The result is JPEG2025501823000013.jpg1157. This corresponds to step S5 above.

[0059] Step 6: Perform line fitting and inverse trigonometric function calculation on the precise detection points and line set to obtain a more precise inclination angle of each line in the template mark, which corresponds to step S6 above.

[0060] Step 7: Correct the position of the template based on the difference between the inclination angle of each straight line and the standard angle. This corresponds to step S7 above.

[0061] Based on the above steps 1 to 7, a specific embodiment is provided below.

[0062] Example 1 The implementation steps of the template mark detection method using a single camera according to this embodiment are as follows.

[0063] Step 11: The mark in the template is in the shape of a "rice" character, and is arranged five times. The pattern mark of the template is detected five times. The initial image acquired by a single CCD camera is converted into a grayscale image and subjected to blur filtering, and the size of the filtering block is variable. In this embodiment, filtering is performed with a 5x5 pixel filter. Filtering blocks of 3x3 pixels and 7x7 pixels were also tried, and the result was that in this case, the accuracy of the inclination angle of the straight line of the detected mark was reduced, but a relatively accurate inclination angle of the straight line of the mark could be obtained.

[0064] Step 12: Check all pixel points in the image, calculate the pixel median of this grayscale image, and perform binarization on all pixel values ​​based on the pixel median to obtain a binarized image. In this embodiment, pixel values ​​smaller than the median are set to 255, and pixel values ​​larger than the median are set to 0 (0 is black and 255 is white in OpenCV). The results are shown in Figure 10. (a) to (e) in Figure 10 correspond to the binarization results after five template placements, respectively. As can be seen from Figure 10, multiple jaggies are formed in the alignment mark pattern after binarization.

[0065] Step 13: Perform edge jaggy corner detection on the binarized image, obtain Shi-Tomasi corner detection data for the image, and set edge jaggy corners {sp1, sp2, sp3, … sp m} is obtained, and the result of edge jaggy corner detection is shown in FIG. 11. (a) to (e) in FIG. 11 respectively correspond to the edge jaggy corner detection results after five times of template placement. In order to obtain enough points, the mass coefficient in the Shi-Tomasi corner detection algorithm is made as small as possible. In this embodiment, the mass coefficient is set to 0.01, and the minimum Euclidean distance between corners is set to 20, which means that the minimum distance between the corners to be finally left is 20 pixels. The neighborhood range specified when calculating the derivative autocorrelation matrix is ​​3. As can be seen from FIG. 11, corners on a slanted straight line in the coordinate system cannot be detected by Shi-Tomasi corner detection.

[0066] Step 14: Perform Canny edge detection on the binary image. In this embodiment, since there are no curves or irregular protrusions in the "rice" shape and the pattern is simple and clean, the selected first edge threshold is 1 and the second edge threshold is 10. According to a relatively small threshold, more edge information can be obtained. Perform cumulative probability Hough transform on the image after Canny edge detection. In this embodiment, the selected collinearity threshold is 80, that is, in the polar coordinate system, when the number of curves intersecting at a point exceeds 80, it is determined that the edge points in the Cartesian coordinate system corresponding to these 80 or more curves are collinear, and these collinear edge points together form an edge straight line. Also, to avoid too much data, the minimum value of the length of the edge straight line is set to 50, and line segments shorter than this length are not left or displayed. The Euclidean distance (straight-line distance) allowing the connection of two points is set to 10 pixels. Obtain the set of roughly detected edge line segments {d1, d2, d3, … d n}, and the detection result of the roughly detected edge line segments is shown in FIG. 12. (a) to (e) in FIG. 12 respectively correspond to the results of Canny edge detection and straight-line detection after 5 times of template placement.

[0067] Step 15: Check the line segments in the obtained set of roughly detected edge line segments {d1, d2, d3, … d n}, integrate the collinear line segments into the set of roughly detected collinear line segments, and obtain the set of roughly detected collinear line segments {cl1, cl 2, cl 3, … cl k}.

[0068] Arbitrarily select two line segments from the set of roughly detected edge line segments {d1, d2, d3, … d n}, and the endpoints of the two line segments are respectively (P 1, P2) and (P 3,P4), where the conditions for determining collinearity are Ψ1 and Ψ2, and if both conditions are satisfied, it is determined that the two lines are collinear. Ψ1 determines whether the area of ​​a rectangle formed by four end points is less than a first predetermined threshold. In this embodiment, Heron's formula is used to directly calculate the area of ​​a triangle based on the lengths of the three sides of the triangle. Ψ2 determines whether the difference between the slope of one line segment and the slope of the line segment formed by connecting two end points of one line segment and two end points of another line segment is less than a second predetermined threshold. In this embodiment, the first predetermined threshold η=400, and the second predetermined threshold δ=0.16.

[0069] Step 16: Check the edge jaggy corner set, perform point and line collinearity judgment in the roughly detected collinear line segment set, and merge the collinear corners and lines into the same roughly detected collinear line segment set. JPEG2025501823000014.jpg1157 is obtained. The condition for determining point and line collinearity is Ψ3. Ψ3 is the corner sp k and the two end points of the line segments in the roughly detected collinear line segment set (P i ,P j ) is less than a third predetermined threshold value. s =300, the calculation method for Ψ3 is the same as the formula for calculating the area of ​​the triangle in Ψ1.

[0070] Step 17: Perform line fitting for the precisely detected points, points in the line set, corners where the lines are collinear, and corresponding line segments, for example, by using the least squares method to perform line fitting. The precisely detected points and line fitting results are shown in Figure 13, where (a) to (e) in Figure 13 respectively correspond to the line fitting results after five times of arrangement, and multiple line equations of the form y=ax+b are obtained from each image, and the precise line inclination angle β is obtained by calculating the arctangent function.

[0071] Step 18: This step verifies the detection deviation existing in the above detection method. The deviation angle of the straight line is obtained by subtracting the inclination angle of each detected straight line from the standard angle (desired angle after the template is placed), and the standard deviation angle is obtained by subtracting the inclination angle of each straight line in the actual position after the template is placed from the standard angle (desired angle after the template is placed), so that the detection error = standard deviation angle - straight line deviation angle, and the deflection error is obtained by averaging the detection errors. The detection deviation of this method can be estimated by calculating the average value and standard deviation for the deflection errors corresponding to multiple standard deviation angles. The detection error and deflection error are shown in Table 1. As can be seen from Table 1, the average value of the detection errors is the deflection error. [Table 1]

[0072] The detection deviation of this method can be estimated by calculating the average value and standard deviation of the detected deflection errors. As shown in Table 2, the detection deviation of this method is [-0.00381±0.01259], which meets the accuracy requirements for template transportation. [Table 2]

[0073] After obtaining the inclination angle of each straight line of the template mark by the above-mentioned method, the angle of the template is corrected based on the angle of the mark that deviates from the standard angle, and the template mark is positioned at the desired placement position, thereby enabling the template to be positioned at the desired placement position.

[0074] As described above, the present disclosure converts the acquired image into a grayscale image, performs blur filtering and binarization, and performs edge jaggy corner detection and line detection in parallel, performs collinearity detection on the obtained roughly detected line segments, leaves two or more collinear line segments, obtains a roughly detected collinear line segment set, checks the corner data, performs point and line collinearity judgment on the roughly detected collinear line segment set, merges the collinear corners, obtains precisely detected points and line sets, and performs line fitting and inverse trigonometric function calculation on each precisely detected point and line set to obtain more precise line inclination angles, and performs high-precision correction on the position of the template based on the difference between the inclination angle of each line and the standard angle.

[0075] The above specific examples further illustrate the objectives, technical solutions and beneficial effects of the present disclosure. The above are only specific examples of the present disclosure and do not limit the present disclosure. As long as they do not deviate from the spirit and principle of the present disclosure, any modifications, equivalent replacements, improvements, etc., fall within the scope of protection of the present disclosure.

Claims

1. Step S1: acquiring an image of a mark on a template using a single camera and obtaining a binary image through pre-processing; A step S2 of detecting edge jaggy corners on the binarized image to obtain a set of edge jaggy corners; Step S3 of detecting edges and lines in the binarized image to obtain a set of roughly detected edge lines; Step S4: checking the roughly detected edge line segment set and judging to leave the collinear line segments to obtain a roughly detected collinear line segment set; Step S5: checking the edge jaggy corner set to perform line and point collinearity judgment in the roughly detected collinear line segment set to obtain an accurately detected point and line set; and step S6 of performing straight line fitting on the precise detection points and line set, and calculating the inclination angle of each of the straight lines by an arctangent function, thereby completing the detection of the mark. A method for detecting a template mark using a single camera.

2. The step S1 is Step S11: performing image acquisition of a mark on the template by a single camera to obtain an initial image; Step S12: converting the initial image into a grayscale image and filtering it; and step S13 of performing a binarization process on the image obtained in step S12 to obtain a binarized image.

2. The method for detecting template marks using a single camera according to claim 1.

3. The step S2 includes performing detection using a Shi-Tomasi corner response function for pixel points in the binarized image to obtain a set of edge jaggy corners.

2. The method for detecting template marks using a single camera according to claim 1.

4. The step S3 is performing Canny edge detection on the binarized image to obtain an edge pattern; obtaining a plurality of edge segments in the mark by a cumulative probabilistic Hough transform based on the edge pattern, and obtaining the roughly detected edge segment set consisting of the plurality of edge segments.

2. The method for detecting template marks using a single camera according to claim 1.

5. The step S4 is A step S41 of arbitrarily selecting two line segments from the set of roughly detected edge line segments to obtain four end points; A step S42 of calculating the area of ​​a quadrangle formed by the four end points, and calculating the slope of one of the line segments and the slope of each of the line segments formed by connecting the two end points of the one line segment and the two end points of the other line segment; a step S43 of determining that the two line segments are collinear if the area of ​​the rectangle is less than a first predetermined threshold and the differences in the slopes are both less than a second predetermined threshold; and a step S44 of checking the roughly detected edge segment set, leaving collinear segments therefrom, to obtain a roughly detected collinear segment set.

2. The method for detecting template marks using a single camera according to claim 1.

6. The step S5 is A step S51 of arbitrarily selecting one corner from the edge jaggy corner set, and forming a triangle with the corner and a line segment in the roughly detected collinear line segment set; A step S52 of calculating the area of ​​the triangle; step S53 of determining that the corner and the line segment are points and lines are collinear if the area of ​​the triangle is less than a third predetermined threshold; and step S54 of checking the edge jaggy corner set and integrating the corners and line segments where the points and lines are collinear to obtain a set of accurately detected points and lines.

2. The method for detecting template marks using a single camera according to claim 1.

7. The step S5 further includes a step S55 of identifying the same collinear line segments in the line set as the precise detection points, leaving any one of them, and removing the other overlapping line segments.

7. The method for detecting template marks using a single camera according to claim 6.

8. The step S6 is Step S61: performing line fitting on the precise detection points, the points in the line set, the corners where the lines are collinear, and the two end points of the corresponding line segments to obtain an equation of the line; and a step S62 of calculating an inclination angle of each of the straight lines using an arctangent function based on the equation of the straight line.

2. The method for detecting template marks using a single camera according to claim 1.

9. The mark in the template includes one of a "R" shape and a "C" shape.

2. The method for detecting template marks using a single camera according to claim 1.

10. Step S1: acquiring an image of a mark on a template using a single camera and obtaining a binary image through pre-processing; A step S2 of detecting edge jaggy corners on the binarized image to obtain a set of edge jaggy corners; Step S3 of detecting edges and lines in the binarized image to obtain a set of roughly detected edge lines; Step S4: checking the roughly detected edge line segment set and judging to leave the collinear line segments to obtain a roughly detected collinear line segment set; Step S5: checking the edge jaggy corner set to perform line and point collinearity judgment in the roughly detected collinear line segment set to obtain an accurately detected point and line set; Step S6: fitting a straight line to the precise detection points and the set of lines, and calculating the inclination angle of each of the straight lines by an arctangent function; and performing a correction on the position of the template based on a difference between the inclination angle of each of the straight lines and a standard angle (S7). A method for template position correction based on a single camera, comprising:

Images