Template mark detection method based on single camera and template position correction method

Abstract

The disclosure provides a single-camera-based template mark detection method and a template position correction method, which comprises the following steps: S1, using a single camera to collect images of marks on a template, and obtaining a binary image after preprocessing; S2, performing edge sawtooth corner point detection on the binary image to obtain a set of edge sawtooth corner points; S3, sequentially performing edge detection and straight line detection on the binary image to obtain a set of coarse detection edge line segments; S4, traversing the set of coarse detection edge line segments, judging and retaining collinear line segments to obtain a set of coarse detection collinear line segments; S5, traversing the set of edge sawtooth corner points, performing point and line collinearity judgment on the line segments in the set of coarse detection collinear line segments to obtain a set of fine detection point lines; S6, performing straight line fitting on the set of fine detection point lines, and then using an inverse tangent function to calculate the inclination angles of the straight lines to complete the detection of the marks. Further, the position of the template is corrected.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of optical detection, in particular to a single-camera-based template mark detection method and a template position correction method. BACKGROUND

[0002] Position correction is needed during the transmission of the template by the mechanical hand. Generally, two PSDs are used for optical correction or two cameras are used for image correction. However, in the case that the space above and below the template suction cup is limited, there is not enough space for the installation of PSDs and cameras. When the optical method is used to detect the straight line of the position mark, it is difficult to correct the slope of the single straight line because the slope is fixed. Therefore, the slope precision of the mark detected by the straight line detection is low. In addition, the difference between the pixel values of adjacent pixels in the gray scale image is small. Therefore, the number of corner points obtained by the corner point detection based on the gray scale image is small. Therefore, the position information of the mark cannot be determined by using the obtained corner points. SUMMARY

[0003] (I) Technical problems to be solved

[0004] In view of the above problems, the present disclosure provides a single-camera-based template mark detection method and a template position correction method, which are used to solve the technical problems such as the limited installation of the detection device and the low precision of the inclined angle of the mark in the traditional method.

[0005] (II) Technical solutions

[0006] In one aspect, the present disclosure provides a single-camera-based template mark detection method, which comprises the following steps: S1, using a single camera to collect images of the mark on the template, and obtaining a binary image after preprocessing; S2, performing edge sawtooth corner point detection on the binary image to obtain a set of edge sawtooth corner points; S3, performing edge detection and straight line detection on the binary image in sequence to obtain a set of coarsely detected edge line segments; S4, traversing the set of coarsely detected edge line segments, judging and retaining the line segments that are collinear, and obtaining a set of coarsely detected collinear line segments; S5, traversing the set of edge sawtooth corner points, performing point and line collinearity judgment on the line segments in the set of coarsely detected collinear line segments, and obtaining a set of finely detected point lines; and S6, performing straight line fitting on the set of finely detected point lines, and using the inverse tangent function to calculate the inclined angle of each straight line to complete the detection of the mark.

[0007] Further, S1 comprises the following steps: S11, using a single camera to collect images of the mark on the template to obtain an initial image; S12, converting the initial image into a gray scale image and performing filtering; and S13, performing binary processing on the image obtained in S12 to obtain a binary image.

[0008] Further, S2 includes: detecting pixel points in the binary image with the Shi-Tomasi corner response function to obtain a set of edge sawtooth corners.

[0009] Further, S3 includes: performing Canny edge detection on the binary image to obtain an edge graph; using the cumulative probability Hough transform based on the edge graph to obtain multiple edge line segments in the mark, and the multiple edge line segments form a set of roughly detected edge line segments.

[0010] Further, S4 includes: S41, randomly selecting two line segments from the set of roughly detected edge line segments to obtain four endpoints; S42, calculating the area of the quadrilateral formed by the four endpoints, calculating the slope of one line segment, and the slopes of the line segments connecting the two endpoints of one line segment to the two endpoints of the other line segment respectively; S43, if the area of the quadrilateral is less than the first preset threshold and the differences in slopes are both less than the second preset threshold, then the two line segments are collinear; S44, traversing the set of roughly detected edge line segments and retaining the collinear line segments therein to obtain a set of roughly detected collinear line segments.

[0011] Further, S5 includes: S51, randomly selecting a corner from the set of edge sawtooth corners, and the corner and the line segments in the set of roughly detected collinear line segments respectively form triangles; S52, calculating the area of the triangles; S53, if the area of the triangle is less than the third preset threshold, then the corner, the line segment point, and the line are collinear; S54, traversing the set of edge sawtooth corners and merging the collinear corners and line segments to obtain a set of precisely detected point-line combinations.

[0012] Further, S5 also includes: S55, selecting the same collinear line segments in the set of precisely detected point-line combinations, retaining any one of them, and removing the redundant line segments.

[0013] Further, S6 includes: S61, performing linear fitting on the collinear corners and the two endpoints of the corresponding line segments in the set of precisely detected point-line combinations to obtain a linear equation; S62, calculating the inclination angle of each line according to the linear equation using the arctangent function.

[0014] Further, the mark on the template includes one of a "cross" shape and a "plus" shape.

[0015] Another aspect of the present disclosure provides a single-camera-based template position correction method, comprising: S1, using a single camera to collect images of marks on a template, and obtaining a binary image after preprocessing; S2, performing edge sawtooth corner point detection on the binary image to obtain a set of edge sawtooth corner points; S3, performing edge detection and straight line detection on the binary image in sequence to obtain a set of coarsely detected edge line segments; S4, traversing the set of coarsely detected edge line segments, judging and retaining collinear line segments in the set to obtain a set of coarsely detected collinear line segments; S5, traversing the set of edge sawtooth corner points, and performing point and line collinearity judgment on the line segments in the set of coarsely detected collinear line segments to obtain a set of finely detected point lines; S6, performing straight line fitting on the set of finely detected point lines, and then using an inverse tangent function to calculate the inclination angles of the straight lines; and S7, correcting the position of the template according to the difference between the inclination angles of the straight lines and standard angles.

[0016] (III) Advantages

[0017] The single-camera-based template mark detection method and the template position correction method of the present disclosure use a single camera to collect images, thereby reducing the occupation of a work station and solving the problem of limited installation of a detection device. Furthermore, more accurate edge corner points are obtained through image binarization, and the straight lines collinear on both sides of the connected part of the template mark are used in combination to obtain more accurate straight line inclination angles after straight line fitting, so as to determine the accurate position information of the mark and realize high-precision detection of the template mark. Finally, the mark position information obtained by using the detection method is used to correct the position of the template. BRIEF DESCRIPTION OF DRAWINGS

[0018] Figure 1 A flowchart of the single-camera-based template mark detection method according to an embodiment of the present disclosure is schematically shown;

[0019] Figure 2 A work station structure schematic diagram of a detection method using device according to an embodiment of the present disclosure is schematically shown;

[0020] Figure 3 A graph after filtering of a mark image according to an embodiment of the present disclosure is schematically shown;

[0021] Figure 4 A graph after binarization processing of a gray scale image according to an embodiment of the present disclosure is schematically shown;

[0022] Figure 5 A result graph of edge sawtooth corner point detection on a binary image according to an embodiment of the present disclosure is schematically shown;

[0023] Figure 6 A result graph of edge detection and straight line detection on a binary image according to an embodiment of the present disclosure is schematically shown;

[0024] Figure 7 An angle diagram for collineation judgment is shown schematically according to an embodiment of the present disclosure;

[0025] Figure 8 A result diagram of straight line fitting on the fine detection point line set is shown schematically according to an embodiment of the present disclosure;

[0026] Figure 9 A processing flow diagram of single camera based template mark detection is shown schematically according to an embodiment of the present disclosure;

[0027] Figure 10 A result diagram of binarization processing after the five-time template sheet feeding is shown schematically according to an embodiment 1 of the present disclosure;

[0028] Figure 11 A result diagram of edge sawtooth corner detection on the binarization image after the five-time template sheet feeding is shown schematically according to an embodiment 1 of the present disclosure;

[0029] Figure 12 A result diagram of edge detection and straight line detection on the binarization image after the five-time template sheet feeding is shown schematically according to an embodiment 1 of the present disclosure;

[0030] Figure 13 A result diagram of straight line fitting on the fine detection point line set after the five-time template sheet feeding is shown schematically according to an embodiment 1 of the present disclosure. DETAILED DESCRIPTION

[0031] To make the objects, technical solutions and advantages of the present disclosure clearer, the present disclosure is further described in detail below with reference to the embodiments and the accompanying drawings.

[0032] The terms used herein are merely used to describe specific embodiments, and are not intended to limit the present disclosure. The terms "comprise", "contain" and the like used herein indicate the existence of the described features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components.

[0033] It should be noted that if the present disclosure involves directional indications, the directional indications are only used to explain the relative position relationship, motion condition and the like between the components in a certain posture, and if the certain posture changes, the directional indications also change accordingly.

[0034] The use of ordinal numbers such as "first", "second", "third" and the like in the description and claims shall not be construed to imply that the elements so designated must be in a certain order or that a certain order must be followed in order to manufacture the device.

[0035] The present disclosure provides a single-camera-based template mark detection method, please refer to Figure 1 , comprising: S1, using a single camera to collect images of the marks on the template, and obtaining a binary image after preprocessing; S2, performing edge sawtooth corner detection on the binary image to obtain a set of edge sawtooth corner points; S3, performing edge detection and straight line detection on the binary image in turn to obtain a set of coarse detection edge line segments; S4, traversing the set of coarse detection edge line segments, judging and retaining the line segments that are collinear, to obtain a set of coarse detection collinear line segments; S5, traversing the set of edge sawtooth corner points, and performing point and line collinearity judgment with the line segments in the set of coarse detection collinear line segments, to obtain a set of fine detection point lines; S6, performing straight line fitting on the set of fine detection point lines, and then using the arctangent function to calculate the inclination angles of each straight line, to complete the detection of the marks.

[0036] The working position diagram of the detection method using device is shown in Figure 2 , which includes a light source, a mirror group, and a CCD camera. The working process is as follows: the mechanical hand transmits the template to the template working position below the CCD camera, the light emitted by the light source reaches the marks on the template after being transmitted by the mirror group, the reflected light carrying the mark information reaches the CCD camera again through the mirror group, and the images collected by the CCD camera are processed. This method uses a single CCD camera for image collection, reduces the occupation of the working position, and solves the problem of limited installation of the detection device. Since there are clamping square grooves around the template working position, the rotation range and the movement range of the template in the square groove do not exceed 0.5° and ±0.5mm. Within this range, the higher the accuracy of the image mark positioning collected by the single CCD camera, the higher the position accuracy of the subsequent template position correction.

[0037] Specifically, the collected image is first pre-processed and binarized; the edge sawtooth generated after the image binarization can obtain more and accurate edge sawtooth corners during corner detection. Then, the edge sawtooth corner detection and the straight line detection are performed in parallel (that is, the processing of step S2 and step S3 can be synchronized or unsynchronized, and the order of the two is not sequential), the collinear detection is performed on the obtained coarse detection edge line segment, the line segment with two or more collinear lines is reserved, and the coarse detection collinear line segment set is obtained; then the edge sawtooth corner set is traversed, the point and line collinear judgment is performed in the coarse detection collinear line segment set, the edge sawtooth corner with collinear lines is reserved, and the fine detection point line set is obtained; finally, the straight line fitting and the inverse trigonometric function operation are performed on the fine detection point line set to obtain a more accurate straight line tilt angle. According to the above processing process, the high-precision detection of the template mark can be realized. That is, more and accurate edge corners are obtained by image binarization, the straight line with collinear lines on both sides of the connected part of the template mark is used, the more accurate straight line tilt angle is obtained after the straight line fitting, and the more accurate position information of the mark is obtained.

[0038] On the basis of the above embodiment, S1 comprises: S11, using a single camera to collect images of the mark on the template to obtain an initial image; S12, converting the initial image into a gray image and performing filtering; S13, performing binarization processing on the image obtained by S12 to obtain a binarized image.

[0039] The CCD camera collects images of the mark on the template to obtain an initial image; the initial image is converted into a gray image and filtered, and the size of the filtering block is variable, and the result is as shown in Figure 3 The reason for using filtering is that the image edge is very susceptible to noise interference, and in order to avoid detecting false edge information, it is necessary to filter the image to remove noise. The purpose of filtering is to smooth some non-edge regions with weak texture, so as to obtain more accurate edges.

[0040] Then, the pixel median value of the gray image is calculated by traversing all the pixel points in the gray image, and all the pixel values are binarized according to the pixel median value. For example, the pixel value less than the median value is assigned 255, and the pixel value greater than the median value is assigned 0. Of course, other values can also be assigned, but the difference between the two selected values should be large enough to distinguish the image from the blank area. Generally speaking, image binarization is to set the gray value of the pixel point on the image to 0 or 255. The result is as shown in Figure 4 As can be seen from the figure, the edge of the alignment mark pattern after binarization processing generates multiple sawtooths, as shown in Figure 4 the middle circle part.

[0041] Based on the above embodiments, S2 includes: performing Shi-Tomasi corner response function detection on the pixels in the binarized image to obtain a set of jagged corner points.

[0042] Perform jagged corner detection on the binarized image to obtain Shi-Tomasi corner detection data and obtain the set of jagged corner points {sp1, sp2, sp3, ... sp m The edge jagged corner detection results are as follows: Figure 5 As shown. To obtain a sufficient number of points, the quality coefficient in the Shi-Tomasi corner response function should be kept as small as possible.

[0043] Based on the above embodiments, S3 includes: performing Canny edge detection on the binarized image to obtain an edge pattern; using cumulative probability Hough transform based on the edge pattern to obtain multiple edge segments in the marker, and the multiple edge segments form a coarse detection edge segment set.

[0044] The binarized image is edge-detected using the conventional Canny edge detection algorithm. The specific numerical selection theory in the Canny edge detection algorithm is as follows: During the edge acquisition process, some dummy edges are obtained. These dummy edges may originate from the real image or be caused by noise; noise-generated dummy edges need to be discarded. The attribute of the current edge is determined based on the relationship between the gradient value of the current edge pixel and two thresholds. If the pixel gradient of the current edge is greater than or equal to the second edge threshold, the current edge pixel is marked as a strong edge, meaning it originates from the real image. If the pixel gradient of the current edge is between the second and first edge thresholds, the current edge pixel is marked as a dummy edge (which should still be retained). If the pixel gradient of the current edge is less than or equal to the first edge threshold, the current edge is discarded. After obtaining the dummy edges through the above processing, it is determined whether the dummy edge originates from the real image or is caused by noise by judging whether it is connected to the strong edge. If the dummy edge is connected to the strong edge, it is from the real image; if it is not connected, it is caused by noise. The specific values ​​of the first and second edge thresholds can be selected as needed.

[0045] After Canny edge detection, a conventional cumulative probability Hough transform can be used for line detection. The edge segments in the image markers are obtained from the edge pattern obtained by Canny edge detection (which may contain curved portions). After line detection, a coarsely detected set of edge segments {d1, d2, d3, ..., d...} is obtained. n The detection results of coarse edge line segments are as follows: Figure 6 As shown. From Figure 6 As can be seen from the boxed portion, multiple edge segments may be detected at the same location after the cumulative probability Hough transform.

[0046] On the basis of the above embodiment, S4 comprises: S41, selecting two line segments in the coarse detection edge line segment set to obtain four end points; S42, calculating the area of a quadrilateral composed of the four end points, calculating the slope of one line segment and the slope of the line segment connected by the two end points of one line segment and the two end points of another line segment respectively; S43, if the area of the quadrilateral is less than a first preset threshold value and the difference between the slopes is less than a second preset threshold value, the two line segments are collinear; S44, traversing the coarse detection edge line segment set, retaining the line segments that are collinear in the set to obtain a coarse detection collinear line segment set.

[0047] In the obtained coarse detection edge line segment set {d1, d2, d3, …, d n} line segment traversal is performed, and collinear line segments are merged, where the merging refers to merging into the coarse detection collinear line segment set, and the collinear line segments are not temporarily de-duplicated, to obtain a coarse detection collinear line segment set {cl1, cl2, cl3, … cl k}, that is, each element in the coarse detection collinear line segment set is a collinear set.

[0048] There are many ways to judge collinearity, and the collinearity judgment process of the present disclosure is described below. In the coarse detection edge line segment set {d1, d2, d3, …, d n}, two line segments are selected, and the end points of the two line segments are (P1, P2) and (P3, P4) respectively. The collinearity judgment condition is Ψ1 and Ψ2, and when both conditions are met, it is considered to be collinear.

[0049] Ψ1 is whether the area of the quadrilateral composed of the four end points is less than a first preset threshold value. According to the Heron formula, the area of a triangle is directly calculated using the lengths of the three sides of the triangle. The area of a quadrilateral is equal to the sum of the areas of two triangles, as shown in the following formula (1):

[0050]

[0051] where S1 represents the area of triangle P1P2P3, h1 is the semi-perimeter of triangle P1P2P3, represents the length of the line segment between P i and P j , S2 represents the area of triangle P2P3P4, h2 is the semi-perimeter of triangle P2P3P4, and η is the first preset threshold value, for example, η = 400; the calculation formulas of h1, h2 and are shown in the following formulas (2) and (3):

[0052]

[0053]

[0054] Ψ2 is the slope of a line segment, the slope of a line segment formed by connecting two end points of another line segment, whether it is less than a second preset threshold. For convenience of description, the slope is converted into the angle between the line segment and the x-axis in the disclosure; the slope can be converted into the angle θ1 between the straight line passing through the end point P1 and the end point P2 and the x-axis by the arctangent function; the angle θ2 between the straight line passing through the end point P1 and the end point P3 and the x-axis; the angle θ3 between the straight line passing through the end point P2 and the end point P4 and the x-axis. As shown in FIG. 4. Figure 7 The inclination angles are in radian, and the calculation formula of the inclination angles θ1, θ2 and θ3 is as shown in the following formula (5):

[0055]

[0056] For example, the second preset threshold δ = 0.16, and the calculation formula of the inclination angles θ1, θ2 and θ3 is as shown in the following formula (5):

[0057]

[0058] On the basis of the above embodiment, S5 comprises: S51, selecting an arbitrary corner point in the edge sawtooth corner point set, and the corner point and the line segment in the coarse detection collinear line segment set respectively form a triangle; S52, calculating the area of the triangle; S53, if the area of the triangle is less than a third preset threshold, the corner point and the line segment point are collinear; S54, traversing the edge sawtooth corner point set, and the corner point and the line segment that are collinear are merged to obtain a fine detection point line set.

[0059] Traversing the edge sawtooth corner point set, the point and line collinear judgment is performed in the coarse detection collinear line segment set, the collinear corner point and the line segment are merged into the same coarse detection collinear line segment set to obtain a fine detection point line set {fl1, fl2, fl3, … fl k}.

[0060] The condition of the point and line collinear judgment is Ψ3. Ψ3 is whether the area of a triangle formed by the corner point sp k and the two end points (P i , P j ) of the line segment in the coarse detection collinear line segment set is less than a third preset threshold, for example, the third preset threshold η s = 300, and the calculation method of Ψ3 is the same as the calculation formula of the area of the triangle in Ψ1.

[0061] On the basis of the above embodiment, S5 further comprises: S55, selecting the same collinear line segments in the fine detection point line set, retaining an arbitrary one, and removing the redundant line segments.

[0062] At this time, the redundant line segments in the same collinear line segments in the fine detection point line set can also be removed, and only one line segment is retained in each group of the same collinear line segments. As shown in FIG. 5. Figure 6As shown in the middle box, there can be multiple line segments with close distances in the same position, and after the collinearity judgment of these line segments and other line segments in step S54, two line segments with the same slope can be obtained, and both of the two line segments are saved in the coarse detection collinear line segment set in step S4. In order to remove redundant data, the multiple coarse detection line segments containing the same coarse detection collinear line segment are removed by the union set calculation of the line segments in this step.

[0063] On the basis of the above embodiment, S6 comprises: S61, performing straight line fitting on the corner points of the points and lines in the fine detection point line set and the two end points of the corresponding line segments to obtain a straight line equation; and S62, calculating the inclination angles of each straight line according to the straight line equation by using the arctangent function.

[0064] The corner points of the points and lines in the fine detection point line collinear set are subjected to straight line fitting, for example, least square method is used for straight line fitting. Since a line segment can be represented by its two end points, the number of all points in the fine detection point line set is N, and the fitting result of the fine detection point line is shown in the following equation (6). Figure 8 As shown in the equation (6), the straight line equation in the form of y=ax+b is obtained, and the accurate straight line inclination angle β is obtained after the arctangent function calculation, as shown in the equation (7).

[0065]

[0066]

[0067] On the basis of the above technical solution, the detection deviation of the detection method can be calculated. S8 comprises: S81, subtracting the deviation angle from the standard deviation angle of each straight line to obtain the detection error of each straight line; S82, averaging the detection errors to obtain the deflection error of the template transmission; and S83, averaging and calculating the standard deviation of the deflection errors corresponding to multiple standard deviation angles to estimate the detection precision deviation of the method.

[0068] When the manipulator is used for sheet feeding, the mark on the template deviates from the standard position by a certain angle, and the deviation angle is taken as the standard deviation angle. The inclination angle of each straight line is subtracted from the standard angle to obtain the deviation angle of each straight line. Therefore, the detection error = standard deviation angle - deviation angle of the straight line, and the deflection error is obtained by averaging the detection errors. The average and standard deviation of the deflection errors corresponding to multiple standard deviation angles can be used to estimate the detection deviation of the method.

[0069]

[0070] wherein, is the deflection error, m is the number of the fine detection point line sets of each image, and θ 标The rotation angle of the manipulator is used as the standard deviation angle, β 偏 The deviation angle of each straight line is obtained by subtracting the inclination angle of each straight line from the standard angle.

[0071] Based on the above embodiments, the marks on the template include one of a "rice" shape and a "cross" shape.

[0072] The detection method of the present disclosure is not only applicable to the "rice" - shaped mark, but also applicable to the "cross" - shaped mark. As long as the mark composed of intersecting straight lines and without curves can be detected by the detection method of the present disclosure for the template mark.

[0073] The present disclosure also provides a method for correcting the position of a template based on a single camera, including: S1, using a single camera to collect images of the marks on the template, and obtaining a binary image after pre - processing; S2, performing edge sawtooth corner detection on the binary image to obtain an edge sawtooth corner set; S3, performing edge detection and straight - line detection on the binary image in sequence to obtain a set of roughly detected edge line segments; S4, traversing the set of roughly detected edge line segments, judging and retaining the collinear line segments therein to obtain a set of roughly detected collinear line segments; S5, traversing the edge sawtooth corner set, and performing point - line collinearity judgment with the line segments in the set of roughly detected collinear line segments to obtain a set of precisely detected point - line; S6, performing straight - line fitting on the set of precisely detected point - line, and then using the arctangent function to calculate the inclination angles of each straight line; S7, correcting the position of the template according to the difference between the inclination angle of each straight line and the standard angle.

[0074] According to the above - mentioned mark detection methods S1 - S6, further, according to the difference between the inclination angle detected by the mark and the standard angle, the position of the template is corrected more accurately.

[0075] According to the characteristics of the alignment mark of the present disclosure, more accurate slopes can be obtained by using the straight lines collinear on both sides of the connected part; the corner points obtained by corner detection based on the grayscale image are less, and edge sawtooth will occur after image binarization. This feature can just obtain more and accurate edge corner points during corner detection. The present disclosure combines double - line and edge sawtooth corner points to detect the deflection error of the end - transmission position, and has a high error detection accuracy.

[0076] The following further illustrates the present disclosure through specific embodiments. The above - mentioned method for detecting template marks and the method for correcting the position of the template based on a single camera are specifically described in the following embodiments. However, the following embodiments are only used to illustrate the present disclosure, and the scope of the present disclosure is not limited thereto.

[0077] The method for detecting template marks and the method for correcting the position of the template based on a single camera of the present disclosure, as Figure 9 shown, includes the following steps executed in sequence:

[0078] Step 1: image acquisition is performed using a single CCD camera to obtain an original image (initial image); the acquired original image is converted into a gray-scale image, and filtering and binarization are performed to obtain a binary image; which is equivalent to the above step S1.

[0079] Step 2: edge sawtooth corner detection is performed on the binary image to obtain corner detection data of the image, and a set of edge sawtooth corners {sp1, sp2, sp3, … sp mm} of the image is obtained; which is equivalent to the above step S2.

[0080] Step 3: edge detection is performed on the binary image; straight line detection is performed on the image after edge detection to obtain a set of coarse detection edge line segments {d1, d2, d3, …, d n};which is equivalent to the above step S3.

[0081] Step 4: line segment traversal is performed in the set of coarse detection edge line segments {d1, d2, d3, …, d n}, and collinear line segments are merged into a set of coarse detection collinear line segments to obtain a set of coarse detection collinear line segments {cl1, cl2, cl3, … cl k};which is equivalent to the above step S4.

[0082] Step 5: the set of edge sawtooth corners is traversed, and point-line collinearity judgment is performed in the set of coarse detection collinear line segments; for collinear corners and line segments, they are merged into the same set of coarse detection collinear line segments to obtain a set of fine detection point-line segments {fl1, fl2, fl3, … fl k};which is equivalent to the above step S5.

[0083] Step 6: straight line fitting and inverse trigonometric function operation are performed on the set of fine detection point-line segments to obtain more accurate inclination angles of each straight line in the template mark; which is equivalent to the above step S6.

[0084] Step 7: according to the difference between the inclination angles of each straight line and the standard angle, the position of the template is corrected; which is equivalent to the above step S7.

[0085] According to the above steps 1 to 7, one specific embodiment is provided below.

[0086] Embodiment 1:

[0087] The implementation steps of the single-camera-based template mark detection method in this embodiment are as follows:

[0088] Step 11: The marks on the template are in a "rice" shape. The upper piece is placed five times, and the template pattern marks are detected five times. The initial image collected by the single-channel CCD camera is converted into a grayscale image and Blur filtered. The size of the filter block is variable. In this embodiment, a 5*5 pixel filter is used for filtering. At the same time, filter blocks of 3*3 and 7*7 pixels are also tried. The results show that the accuracy of the inclination angle of the marked straight line detected will decrease, but finally, a relatively accurate inclination angle of the marked straight line can still be obtained.

[0089] Step 12: Traverse all pixel points in the image to calculate the pixel median value of this grayscale image, and binarize all pixel values according to the pixel median value to obtain a binarized image. In this embodiment, pixel values less than the median are assigned 255, and pixel values greater than the median are assigned 0 (in opencv, 0 is black and 255 is white). The results are as Figure 10 shown, Figure 10 (a) to (e) in the figure respectively correspond to the results of binarization processing after the template is placed on the upper piece five times. It can be seen from Figure 10 that multiple sawteeth have appeared on the edge of the alignment mark pattern after binarization processing.

[0090] Step 13: Perform edge sawtooth corner detection on the binarized image to obtain the Shi-Tomasi corner detection data of the image, and obtain the edge sawtooth corner set {sp1, sp2, sp3,... sp m} The edge sawtooth corner detection results are as Figure 11 shown, Figure 11 (a) to (e) in the figure respectively correspond to the results of edge sawtooth corner detection after the template is placed on the upper piece five times. In order to obtain enough points, the quality coefficient in the Shi-Tomasi corner detection algorithm is taken as small as possible. In this embodiment, the quality coefficient is 0.01, and the minimum Euclidean distance between corners is set to 20, which means that the minimum distance between the finally retained corners is 20 pixels. The specified neighborhood range when calculating the derivative autocorrelation matrix is 3. It can be seen from Figure 11 that the Shi-Tomasi corner detection cannot find the corners on the inclined straight line in the coordinate system.

[0091] Step 14: Canny edge detection is performed on the binary image; in this embodiment, since the "rice" character has no curve and special-shaped protrusions, the image structure is simple and clean, and thus the first edge threshold is 1 and the second edge threshold is 10, and a smaller threshold can capture more edge information. Accumulative probability Hough transformation is performed on the image after Canny edge detection, and in this embodiment, the selected collinear threshold is 80, that is, in the polar coordinate system, if the number of curves intersecting at a point exceeds 80, then the edge points in the Cartesian coordinate system corresponding to the 80 or more curves are considered to be collinear, and these collinear edge points collectively form an edge straight line. In addition, in order to avoid excessive data, the minimum length of the edge straight line is set to 50, and line segments below this length are not retained and displayed. The Euclidean distance (straight line distance) allowing two points to be connected is set to 10 pixels. A coarse detection edge line segment set {d1, d2, d3, …, d n} is obtained, and the detection result of the coarse detection edge line segment is shown in FIG. 8. Figure 12 Figure 12 In FIG. 8, (a)-(e) respectively correspond to the results of Canny edge detection and straight line detection after five times of template pasting.

[0092] Step 15: In the obtained coarse detection edge line segment set {d1, d2, d3, …, d n}, line segment traversal is performed, and collinear line segments are merged into a coarse detection collinear line segment set to obtain a coarse detection collinear line segment set {cl1, cl2, cl3, … cl k}.

[0093] In the coarse detection edge line segment set {d1, d2, d3, …, d n}, two line segments are selected, and the end points of the two line segments are respectively (P1, P2) and (P3, P4). The collinear judgment condition is Ψ1 and Ψ2, and when both conditions are met, it is considered to be collinear. Ψ1 is whether the area of the quadrilateral composed of the four end points is less than a first preset threshold. In this embodiment, according to the Heron formula, the area of the triangle is directly calculated using the lengths of the three sides of the triangle. Ψ2 is whether the difference between the slope of a line segment and the slope of the line segment formed by connecting the two end points of another line segment with the two end points of the line segment is less than a second preset threshold. In this embodiment, the first preset threshold η = 400, and the second preset threshold δ = 0.16.

[0094] Step 16: Traverse the edge zigzag corner point set, perform point-line collinear judgment in the coarse detection collinear line segment set, and merge the collinear corner points and line segments into the same coarse detection collinear line segment set to obtain a fine detection point-line set {fl1, fl2, fl3, … fl k}. The condition for point-line collinear judgment is Ψ3. Ψ3 is the condition that the distance between the two end points of a line segment and the distance between the two end points of another line segment are less than a third preset threshold. In this embodiment, the third preset threshold is 10 pixels. k ​The two endpoints (P) of the line segment in the set of collinear line segments detected by coarse detection i P j Whether the area of ​​the triangle formed by Ψ3 is less than the third preset threshold. In this embodiment, the third preset threshold ηs = 300, and the calculation method of Ψ3 is the same as the formula for calculating the area of ​​the triangle in Ψ1.

[0095] Step 17: Perform line fitting on the midpoints and corner points of the collinear set of precision detection points and lines, and their corresponding line segments. For example, use the least squares method for line fitting. The fitting result of the precision detection points and lines is as follows: Figure 13 As shown, Figure 13 middle( a (e) correspond to the results of the straight line fitting of the five images respectively. Each image yields multiple straight line equations of the form y = ax + b. After calculating the arctangent function, the accurate straight line tilt angle β is obtained.

[0096] Step 18: This step verifies the detection deviation of the above detection method. The tilt angle of each detected straight line is subtracted from the standard angle (the expected angle after the template is applied) to obtain the straight line deviation angle. The tilt angle of each straight line at the actual position on the template is subtracted from the standard angle (the expected angle after the template is applied) to obtain the standard deviation angle. Therefore, the detection error = standard deviation angle - straight line deviation angle. The deflection error is obtained by averaging the detection errors. The detection deviation of this method can be estimated by calculating the mean and standard deviation of the deflection errors corresponding to multiple standard deviation angles. The detection error and deflection error are shown in Table 1. As can be seen from Table 1, the mean of the detection error is the deflection error.

[0097] Table 1. Deflection error detection results (unit: °)

[0098]

[0099]

[0100] The detection deviation of this method can be estimated by calculating the mean and standard deviation of the deflection errors of multiple detections, as shown in Table 2. Table 2 shows that the detection deviation of this method is [-0.00381±0.01259], which meets the accuracy requirements of template transmission.

[0101] Table 2 Detection Accuracy

[0102] Deflection error 1 -0.00691 2 -0.01527 3 -0.00417 4 0.01746 5 -0.01016 Mean -0.00381 Standard deviation 0.01259

[0103] After obtaining the tilt angle of each straight line of the template mark using the above method, the template angle can be corrected based on the difference between the mark and the standard angle, so that the template mark reaches the desired position on the film, thereby making the template reach the desired position on the film.

[0104] In summary, the present disclosure converts the collected image into a grayscale image and performs Blur filtering and binarization; then parallel edge sawtooth corner detection and line detection are performed, the obtained coarse detection line segments are subjected to collinearity detection, and coarse detection collinear line segment sets with more than two line segments are retained; then the corner point data is traversed, point-line collinearity judgment is performed in the coarse detection collinear line segment set, collinear corner points are merged, and fine detection point-line sets are obtained; finally, straight line fitting and inverse trigonometric function operation are performed on each fine detection point-line set to obtain more accurate straight line inclination angles, and the position of the template is corrected with high precision according to the difference between the inclination angles of each straight line and the standard angle.

[0105] The above-described specific embodiments further illustrate the purposes, technical solutions and beneficial effects of the present disclosure, and it should be understood that the above-described is only a specific embodiment of the present disclosure and is not intended to limit the present disclosure. Any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A single camera based template marker detection method, characterized in that, The application relates to a method for detecting a mark on a template, which comprises the following steps: S1, image acquisition of the mark on the template by using a single camera, and obtaining a binary image after pretreatment; S2, edge sawtooth corner point detection of the binary image, and obtaining an edge sawtooth corner point set; S3, edge detection and straight line detection of the binary image in sequence, and obtaining a coarse detection edge line segment set; S4, traversal of the coarse detection edge line segment set, judgment and reservation of collinear line segments in the coarse detection edge line segment set, and obtaining a coarse detection collinear line segment set; S5, traversal of the edge sawtooth corner point set, point and line collinearity judgment of the edge sawtooth corner point set and the line segments in the coarse detection collinear line segment set, merging of collinear corner points and line segments into the same coarse detection collinear line segment set, and obtaining a fine detection point line set; S6, straight line fitting of the fine detection point line set, and calculation of the inclination angles of the straight lines by using an inverse tangent function, so as to complete the detection of the mark.

2. The single-camera based template marker detection method of claim 1, wherein, The S1 comprises the following steps: S11, image acquisition of the mark on the template by using a single camera, and obtaining an initial image; S12, conversion of the initial image into a gray image and filtering; S13, binary processing of the image obtained in the S12, and obtaining a binary image.

3. The single-camera based template marker detection method of claim 1, wherein, The S2 comprises the following steps: Shi-Tomasi corner point response function detection of pixel points in the binary image, and obtaining an edge sawtooth corner point set.

4. The single-camera based template marker detection method of claim 1, wherein, The S3 comprises the following steps: Canny edge detection of the binary image, and obtaining an edge graph; According to the edge graph, a plurality of edge line segments in the mark are obtained by using a cumulative probability Hough transformation, and the plurality of edge line segments form the coarse detection edge line segment set.

5. The single-camera based template marker detection method of claim 1, wherein, The S4 comprises the following steps: S41, selection of two line segments in the coarse detection edge line segment set, and obtaining four end points; S42, calculation of the area of a quadrangle composed of the four end points, calculation of the slope of one line segment, and calculation of the slopes of the line segments connected by the two end points of one line segment and the two end points of another line segment; S43, if the area of the quadrangle is smaller than a first preset threshold value, and the difference values of the slopes are all smaller than a second preset threshold value, the two line segments are collinear; S44, traversal of the coarse detection edge line segment set, reservation of collinear line segments in the coarse detection edge line segment set, and obtaining a coarse detection collinear line segment set.

6. The single-camera based template marker detection method of claim 1, wherein, The S5 comprises the following steps: S51, selection of a corner point in the edge sawtooth corner point set, and formation of a triangle by the corner point and the line segments in the coarse detection collinear line segment set; S52, calculation of the area of the triangle; S53, if the area of the triangle is smaller than a third preset threshold value, the corner point and the line segments are collinear; S54, traversal of the edge sawtooth corner point set, and merging of the collinear corner points and line segments to obtain a fine detection point line set.

7. The single-camera based template marker detection method of claim 6, wherein, The S5 further comprises the following steps: S55, selection of the same collinear line segments in the fine detection point line set, reservation of any one of the collinear line segments, and removal of redundant line segments.

8. The single-camera based template marker detection method of claim 1, wherein, The S6 comprises the following steps: S61, straight line fitting of the corner points and the two end points of the corresponding line segments in the fine detection point line set which are collinear, and obtaining a straight line equation; S62, calculation of the inclination angles of the straight lines by using an inverse tangent function according to the straight line equation.

9. The single-camera based template marker detection method of claim 1, wherein, The mark on the template comprises one of a "rice” character and a "ten” character.

10. A single camera based template position correction method, characterized in that, The method comprises the steps of: S1, using a single camera to collect images of the mark on the template, and obtaining a binary image after preprocessing; S2, performing edge sawtooth corner detection on the binary image to obtain a set of edge sawtooth corners; S3, sequentially performing edge detection and straight line detection on the binary image to obtain a set of coarse detection edge line segments; S4, traversing the set of coarse detection edge line segments, judging and retaining line segments that are collinear, and obtaining a set of coarse detection collinear line segments; S5, traversing the set of edge sawtooth corners, performing point and line collinearity judgment with the line segments in the set of coarse detection collinear line segments, merging collinear corners and line segments into the same set of coarse detection collinear line segments, and obtaining a set of fine detection point lines; S6, performing straight line fitting on the set of fine detection point lines, and calculating the inclination angles of the straight lines by using an inverse tangent function; S7, correcting the position of the template according to the difference between the inclination angles of the straight lines and standard angles.

Images