A robust localization and recognition method for multi-point coded markers

Abstract

This invention discloses a robust method for locating and recognizing multi-point coded markers, comprising: capturing an image containing multiple coded markers using an industrial camera; preprocessing the image to obtain a binarized image, ensuring that all the contours of the coded markers to be tested are single-edge contours; performing pixel-level edge detection on the binarized image using an adaptive Canny operator to obtain an edge image; detecting connected contours in the edge image and establishing a contour hierarchy structure; extracting and segmenting Regions of Interest (ROIs) that may be coded markers using the contour hierarchy structure and multi-scale discriminant factors to obtain the final multi-point coded marker ROI region; locating and decoding the multi-point coded markers in the final multi-point coded marker ROI region to obtain the encoded values ​​of the multi-point coded markers; and improving the accuracy and robustness of coded marker location and decoding in multi-point coded marker ROI regions under complex background environments.

Description

Technical Field

[0001] This invention belongs to the field of three-dimensional object measurement technology, and more specifically, relates to a robust localization and identification method for multi-point coded marker points. Background Technology

[0002] In the field of large-scale object 3D measurement using close-range photogrammetry, coded markers are often introduced into the system as a means to control global errors by addressing the cumulative error problem of point clouds obtained from multiple scans and stitching. By pasting coded markers within the common field of view of two measurements, matching information of corresponding points can be accurately obtained, exhibiting higher accuracy and robustness compared to stereo matching algorithms based on grayscale or 2D features. To ensure measurement effectiveness, the design of coded markers should meet characteristics such as unique coding, accurate positioning, large coding capacity, and ease of decoding. In summary, the design, identification, and positioning of coded markers are crucial foundations for corresponding point matching and 3D reconstruction in close-range photogrammetry, and their research is of great significance.

[0003] To address the problem of identifying and locating coded markers, patent application CN113313628A discloses a method for identifying ring-shaped coded markers based on affine transformation and the mean pixel method. This method recovers image distortion caused by shooting tilt angle by performing an affine transformation on the corresponding area of ​​the ring-shaped coded marker with a known-size bounding rectangle, and then decodes the coded area using the mean pixel method. Patent application CN109285198A discloses a ring-shaped coded marker design based on binary-to-decimal conversion. It avoids the cyclic shift calculation process by rearranging the binary string, requiring only a single detection and code value extraction through rearrangement. Furthermore, the coded value can be arbitrarily selected without restriction. In addition, patent application CN108764004A discloses a method for decoding and identifying ring-shaped coded markers based on coded ring sampling. By calculating the ellipse parameters of the coded center point, the decoding process can be performed without affine transformation, further improving decoding efficiency. However, the methods disclosed in the aforementioned patent documents all use ring-shaped coded markers. When the shooting angle is large, the smaller coded rings are easily misidentified as the center positioning circle. Furthermore, since the ring-shaped coded points do not have a starting marker, their encoding capacity is small, and they are no longer used in situations requiring a large number of markers. In addition, the above methods all extract edge contours and then set threshold parameters such as area and shape to filter possible coded marker regions. Such methods require adjusting multiple parameters for different environments, introducing additional computational load, and the robustness of coded marker recognition under complex background conditions still needs to be improved. Summary of the Invention

[0004] To address the aforementioned deficiencies or improvement needs of existing technologies, this invention provides a robust localization and recognition method for multi-point coded markers. The method involves capturing an image containing multiple coded markers using an industrial camera; preprocessing the image to obtain a binarized image, ensuring that all the contours of the coded markers to be tested are single-edge contours; performing pixel-level edge detection on the binarized image using an adaptive Canny operator to obtain an edge image; detecting connected contours in the edge image and establishing a contour hierarchy; using the contour hierarchy and multi-scale discriminant factors to extract and segment potential ROI regions of coded markers, obtaining the final multi-point coded marker ROI region; and performing multi-point coded marker localization and decoding on the final multi-point coded marker ROI region to obtain the encoded values ​​of the multi-point coded markers. By introducing a contour hierarchy structure and multi-scale discriminant factors for multi-point coded markers, efficient and robust extraction of multi-point coded marker ROI regions is achieved in complex background environments, improving the accuracy and robustness of coded marker localization and decoding.

[0005] To achieve the above objectives, the present invention provides a robust localization and identification method for multi-point coded markers, comprising the following steps:

[0006] S1: Capture an image containing multiple coded marker points using an industrial camera;

[0007] S2: Preprocess the image to obtain a binarized image, so that the contours of all the coded marker points to be tested are single-edge contours;

[0008] S3: Perform pixel-level edge detection on the binarized image using the adaptive Canny operator to obtain the edge image;

[0009] S4: Detect connected contours in the edge image and establish a contour hierarchy structure. Extract and segment the ROI of possible coding marker points through the contour hierarchy structure and multi-scale discriminant factors to obtain the final multi-point coding marker point ROI region.

[0010] S5: Perform multi-point coding marker localization and decoding on the final multi-point coding marker ROI region to obtain the coding value of the multi-point coding marker.

[0011] Furthermore, the multi-point coding markers in step S1 are designed based on the principle of cross-ratio invariance. They include eight centered reflective markers, five of which are fixed points and three are floating coding points. The five fixed points are denoted as O, A, B, C, and D, with point C being the overall positioning point of the multi-point coding markers. Points O, C, and D are collinear, with C being the middle point, O being the point closer to C, and D being the point farther away. A and B are perpendicular to the line containing O, C, and D and are distributed on both sides of the line. The coordinate system of the multi-point coding markers is established using points O, A, and B. The remaining three coding points are distributed in 20 selectable coding positions, with each pair of points not adjacent to the others.

[0012] Further, in step S2, the image is preprocessed to obtain a binarized image, so that the contours of all the coded marker points to be tested are single-edge contours, including:

[0013] S21: Perform grayscale processing on the image to obtain a grayscale image;

[0014] S22: Perform median filtering on the grayscale image to obtain the filtered image;

[0015] S23: Using the USM unsharpened mask algorithm, the contour edges of the filtered image are sharpened to obtain an image with enhanced edges;

[0016] S24: The image after edge enhancement is binarized to obtain a binarized image; the binarized image is a black and white image that only contains black and white points with gray values ​​of 0 and 255, thereby ensuring that the contours of all the coded marker points to be tested are single-edge contours.

[0017] Furthermore, step S4 includes the following steps:

[0018] S41: Detect connected contours in the edge image to obtain the positional relationship between the edge contours and their contour points;

[0019] S42: Based on the positional relationship between the edge contour and its contour points, establish a contour hierarchy structure of multi-point coding points;

[0020] S43: Obtain the coding marker point region by filtering based on the contour hierarchy structure of the multi-point coding points.

[0021] S44: Filter and remove contours that are too large or too small in the region of the coded marker points based on the contour size information;

[0022] S45: Perform polygon fitting on the contours selected in step S44 according to the contour shape factor, remove contours that do not conform to the shape, and obtain contours that conform to the shape of the region of the coding mark points to be tested.

[0023] S46: Extract and segment the ROI region of the contour that matches the shape of the region of the coding marker to be tested, and obtain the final ROI region of the coding marker.

[0024] Further, in step S5, the multi-point coded marker point localization and decoding are performed on the final multi-point coded marker point ROI region to obtain the encoded value of the multi-point coded marker point, including the following steps:

[0025] S51: In each ROI region of the final encoded marker point ROI region, subpixel edges of polynomial interpolation are extracted;

[0026] S52: Perform ellipse fitting on the sub-pixel edges extracted in step S51 to obtain all ellipse-like contours in the ROI region.

[0027] S53: Further filter the elliptical-like contours obtained in step S52 based on the two filtering factors of ellipse size and ellipse shape to obtain effective elliptical contours.

[0028] S54: Based on the filtering results of step S53, count the elliptical contours identified in each ROI region to obtain the number of elliptical contours, and determine the validity of the ROI region; if valid, proceed to the next step; if invalid, repeat steps S52 to S54 until a valid ROI region is obtained.

[0029] S55: Based on the coordinates of the four vertices of the rectangular edge contour within the effective ROI region extracted in step S54, perform an affine transformation on the coding marker region to obtain several affine images, i.e., images of multi-point coding markers in the frontal view.

[0030] S56: Filter, binarize, perform edge detection, and ellipse fitting on each of the affine images to obtain the center coordinates of each affine image, and search for collinear points among all the center coordinates. If three points are collinear, the middle point is C, the point closer to C is O, and the point farther away is D; if no three points are collinear, it is considered that there are no coding points in the region.

[0031] S57: Based on the invariance of the cross ratio from A and B to the line COD during the affine process, determine the positions of points A and B and establish a coordinate system;

[0032] S58: Based on the positions of the remaining 3 floating coding points in the coordinate system after the affine transformation, the coding value of the multi-point coding mark can be obtained by looking up the corresponding coding table.

[0033] Further, in step S51, sub-pixel edge extraction using polynomial interpolation is performed in each ROI region, including the following steps:

[0034] Eight templates, evenly divided into 0-360° directions, are convolved with the image to obtain gradients in the eight directions.

[0035] By selecting the gradient with the largest magnitude and performing non-maximum suppression, the gradient direction θ and gradient magnitude g of the image edge pixel (i,j) can be obtained. Therefore, the extracted sub-pixel edge coordinates (x,y) can be expressed as:

[0036]

[0037] Where gleft, gright, and g0 represent the gradient values ​​to the left, right, and themselves of the edge pixel, respectively, and ω represents the distance from the adjacent pixel to the edge pixel; ω = 1 when the gradient is 0°, 90°, 180°, and 270°, and ω = 1 at other angles.

[0038] Furthermore, the selection criterion in step S53 is the ellipse size S. E >50, and ellipse shape factor The ellipse is a valid elliptical profile; S E R represents the size of the ellipse outline. EL R is the radius of the major axis of the ellipse. EW The radius of the minor axis of the ellipse is denoted as _____. If no valid ellipse profile is found, repeat steps S52 and S53 until a valid ellipse profile is found.

[0039] Furthermore, step S54 also includes that if the number of elliptical contours is equal to 8, then the ROI is a valid ROI and can be further used for the positioning and decoding of multi-point coded markers; otherwise, the ROI is invalid.

[0040] Furthermore, step S56 also includes determining whether three points are collinear based on the angular error between lines formed by different combinations of two points, wherein the angular error E θ The calculation formula is:

[0041] E θ =|θ(p1,p2)-θ(p2,p3)| (8)

[0042] Let p1, p2, and p3 be any three points; θ(p1, p2) is the angle between the two points p1 and p2 forming a straight line; θ(p2, p3) is the angle between the two points p2 and p3 forming a straight line.

[0043] The angle error threshold is set to T. θ =0.1°, then the criterion for judging three points as collinear is:

[0044]

[0045] Furthermore, step S3, which uses the adaptive Canny operator to perform pixel-level edge detection on the binarized image, includes the following steps:

[0046] The gradient image of the binarized image is obtained using the Canny operator;

[0047] Let T be the segmentation threshold of the gradient image. K Then, according to the segmentation threshold T K All pixels in the gradient image are divided into categories higher than the segmentation threshold T. K P1 and below the segmentation threshold T K Let P1 and P2 be the two classes, and their means be denoted as m1 and m2 respectively. Let m3 be the mean of all pixels in the gradient image. Let Q1 and Q2 be the probabilities of a pixel being classified into P1 and P2 respectively. Then the following formula holds:

[0048] Q1m1 + Q2m2 = m3 (10)

[0049] Q1 + Q2 = 1 (11)

[0050] Current segmentation threshold T K The formula for calculating the inter-class variance of the image is as follows:

[0051] σ 2 =Q1(m1-m3) 2 +Q2(m2-m3) 2 (12)

[0052] The segmentation threshold T is set within the grayscale range of (0, 255). K Perform a traversal to find the class variance σ. 2 The largest T K The value, which is the obtained adaptive gradient threshold, is denoted as T. B ;

[0053] T B The high threshold T during the nonmaximization process of the Canny operator H The value obtained by dividing the threshold by 3 and rounding down is taken as the lower threshold T. L This enables adaptive pixel-level edge detection, resulting in an edge image.

[0054] In summary, compared with the prior art, the above-described technical solutions conceived by this invention can achieve the following beneficial effects:

[0055] 1. A robust localization and recognition method for multi-point coded markers according to the present invention comprises: capturing an image containing multiple coded markers using an industrial camera; preprocessing the image to obtain a binarized image, ensuring that the contours of all coded markers to be tested are single-edge contours; performing pixel-level edge detection on the binarized image using an adaptive Canny operator to obtain an edge image; detecting connected contours in the edge image and establishing a contour hierarchy structure; extracting and segmenting Regions of Interest (ROIs) that may be coded markers using the contour hierarchy structure and multi-scale discriminant factors to obtain the final multi-point coded marker ROI region; locating and decoding the final multi-point coded marker ROI region to obtain the coded values ​​of the multi-point coded markers; by introducing a contour hierarchy structure and multi-scale discriminant factors for multi-point coded markers, efficient and robust extraction of multi-point coded marker ROI regions is achieved in complex background environments, improving the accuracy and robustness of coded marker localization and decoding.

[0056] 2. The robust localization and recognition method of multi-point coded markers of the present invention introduces median filtering and USM algorithm in the image preprocessing process to improve the image signal-to-noise ratio. While reducing image noise, it enhances the contour edges in the image, reducing the problem that some contour edges are difficult to correctly identify due to uneven light intensity.

[0057] 3. The robust localization and recognition method for multi-point coded markers of the present invention introduces an adaptive Canny edge detection operator in the edge detection process. The high and low thresholds required by the Canny operator are automatically calculated based on the gradient map of the edge image, which further improves the robustness of edge detection and reduces the workload of parameter adjustment.

[0058] 4. The robust localization and recognition method of multi-point coded markers of the present invention uses affine transformation to process the filtered ROI region, which restores the image distortion caused by the camera shooting angle, so that the decoding process is carried out in the ROI region that is close to the front view, thereby reducing the amount of computation and improving the decoding accuracy and robustness. Attached Figure Description

[0059] Figure 1 This is a flowchart illustrating a robust localization and identification method for multi-point coded markers according to an embodiment of the present invention.

[0060] Figure 2 This is an image taken by an industrial camera against a complex background in a robust localization and recognition method for multi-point coded markers according to the present invention.

[0061] Figure 3This is a schematic diagram showing the position of the multi-point coded markers in the O-AB coordinate system in the robust localization and identification method of multi-point coded markers of the present invention.

[0062] Figure 4 This is a diagram illustrating the composition of multi-point coded markers in a robust localization and identification method for multi-point coded markers according to the present invention.

[0063] Figure 5 This is a schematic diagram of the image preprocessing process in the robust localization and recognition method for multi-point coded markers of the present invention;

[0064] Figure 6 This is a schematic diagram showing the magnified effect of image binarization in the image preprocessing of the image in the encoded marker part of the robust localization and recognition method of multi-point encoded markers in the present invention.

[0065] Figure 7 This is a schematic diagram of the image preprocessing results in a robust localization and recognition method for multi-point coded markers according to the present invention.

[0066] Figure 8 This is a schematic diagram illustrating the pixel-level edge detection effect of a robust localization and recognition method for multi-point coded markers according to the present invention.

[0067] Figure 9 This is a schematic diagram illustrating the process of extracting and segmenting the Region of Interest (ROI) of a region that may be a coded marker point in a robust localization and identification method for multi-point coded marker points according to the present invention.

[0068] Figure 10 This is a schematic diagram of the hierarchical structure of the multi-point coded point contour in the robust localization and recognition method of multi-point coded markers of the present invention;

[0069] Figure 11 This is a schematic diagram of the hierarchical inclusion relationship of the tree-like outline of the multi-point coded points in the robust localization and recognition method of multi-point coded markers of the present invention;

[0070] Figure 12 This is a schematic diagram of the results after screening based on the contour hierarchy structure, which is a robust localization and recognition method for multi-point coded markers according to the present invention.

[0071] Figure 13 This is a schematic diagram of the results after filtering based on contour size information, which is a robust localization and recognition method for multi-point coded markers according to the present invention.

[0072] Figure 14 This is a schematic diagram of the results after screening based on the contour shape factor, which is a robust localization and recognition method for multi-point coded markers according to the present invention.

[0073] Figure 15 This is a schematic diagram of the ROI region result of the coded marker points in the edge image after screening, based on the robust localization and recognition method of multi-point coded marker points according to the present invention.

[0074] Figure 16 This is a schematic diagram of the multi-point coded marker localization and decoding process of a robust localization and recognition method for multi-point coded markers according to the present invention.

[0075] Figure 17 This is a schematic diagram of the affine transformation result of the rectangular contour within the effective ROI region in the robust localization and recognition method of multi-point coded markers of the present invention.

[0076] Figure 18 This is a schematic diagram illustrating the detection result of five fixed points after an affine transformation of a multi-point coded marker in a robust localization and recognition method for multi-point coded markers according to the present invention.

[0077] Figure 19 This is a schematic diagram showing the localization and decoding results of all coded markers in an image, based on the robust localization and recognition method for multi-point coded markers according to the present invention. Detailed Implementation

[0078] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0079] like Figure 1 As shown, this invention, taking the use of an industrial camera to collect multi-point coded markers as an example, provides a robust localization and identification method for multi-point coded markers, including the following steps:

[0080] S1: Capture an image containing multiple coded marker points using an industrial camera;

[0081] S2: Preprocess the image to obtain a binarized image, so that the contours of all the coded marker points to be tested are single-edge contours;

[0082] S3: Perform pixel-level edge detection on the binarized image using the adaptive Canny operator to obtain the edge image;

[0083] S4: Detect connected contours in the edge image and establish a contour hierarchy structure. Extract and segment the ROI of possible coding marker points through the contour hierarchy structure and multi-scale discriminant factors to obtain the final multi-point coding marker point ROI region.

[0084] S5: Perform multi-point coding marker localization and decoding on the final multi-point coding marker ROI region to obtain the coding value of the multi-point coding marker.

[0085] Furthermore, in an embodiment of the present invention, step S1 involves image acquisition in a relatively complex background environment (car chassis) to verify the effectiveness of the proposed method. The acquired original images are as follows: Figure 2 As shown, this multi-point coding marker is designed based on the principle of cross-ratio invariance. It consists of eight circular reflective markers, five of which are fixed points and three are floating coding points. The five fixed points are denoted as O, A, B, C, and D, with point C serving as the overall positioning point. Points O, C, and D are collinear, while points A and B are perpendicular to the line containing O, C, and D and distributed on either side of that line. A coordinate system for the multi-point coding marker is established using points O, A, and B. The remaining three coding points are distributed in 20 selectable coding positions, with no two points adjacent to each other. Figure 3 As shown; based on the coordinates of the remaining three encoded points in the O-AB coordinate system, the multi-point encoded marker can be decoded by referring to the encoding template, such as... Figure 4 As shown;

[0086] Furthermore, in the embodiments of the present invention, since the images acquired by the industrial camera are affected by the measurement environment, problems such as image noise and uneven illumination are inevitable. Therefore, certain image preprocessing operations are needed to improve the quality of the acquired images to ensure the positioning and recognition accuracy of multi-point coding points; such as Figure 5 As shown, in step S2, the image is preprocessed to obtain a binarized image, so that the contours of all the coded marker points to be tested are single-edge contours, including:

[0087] S21: Perform grayscale processing on the image to obtain a grayscale image;

[0088] S22: Perform median filtering on the grayscale image to obtain the filtered image;

[0089] S23: Using the USM (Unsharp Mask) algorithm, the contour edges of the filtered image are enhanced (sharpened) to obtain an image with enhanced edges;

[0090] S24: The image after edge enhancement is binarized to obtain a binarized image; the binarized image is a black and white image that only contains black and white points with gray values ​​of 0 and 255, thereby ensuring that the contours of all the coded marker points to be tested are single-edge contours.

[0091] Furthermore, in step S21, since some industrial cameras capture images in color, and color information is not useful for the positioning and recognition of the coded markers, in order to reduce the amount of unrelated information in the image and reduce the computational burden of the algorithm, the image needs to be converted to grayscale first, turning the image into a single-channel image with grayscale values ​​in the range of (0, 255).

[0092] Furthermore, in step S22, considering the limitations of the actual on-site image acquisition environment, various types of noise inevitably exist in the acquired images. To obtain more accurate measurement data and more robust coded marker extraction results, image filtering is necessary to improve the signal-to-noise ratio. Considering that the key to coded marker detection and recognition is extracting the edge contours of objects in the acquired image, this invention chooses to use a median filter to filter the image to obtain better edges. Step S22 also includes: selecting a corresponding filter kernel based on the pixel size occupied by the coded point to be tested in the image acquired within the theoretical working range of the camera, and setting the gray value of each point in the image to the median of the gray values ​​of all pixels within the filter kernel range for that point. This invention, through this method, can effectively eliminate salt-and-pepper noise and impulse noise in the image. Furthermore, due to the edge-preserving characteristics of median filtering, the edge information of the contours of objects in the image is better preserved compared to methods such as mean filtering.

[0093] Furthermore, in step S23, due to the variability of factors such as illumination, the reflective markers in the multi-point coded markers may exhibit inconsistent reflectivity, resulting in uneven brightness of their edge pixels, i.e., large fluctuations in the grayscale values ​​corresponding to the pixels. Directly binarizing the image would make it difficult to select a suitable binarization threshold, leading to a significant difference between the obtained edges and the actual edges. Therefore, this invention employs the USM (Unsharpened Mask) algorithm to enhance (sharpen) the contour edges of the image; step S23 also includes:

[0094] The filtered image is smoothed using a low-pass mean filter to obtain a low-frequency component image P. L ;

[0095] Image P after the filtering process described above O Subtract the low-frequency component image P L The high-frequency image is obtained and used as a template image, denoted as P. M ;

[0096] The template image is multiplied by a weighting coefficient ω and added to the filtered image P. O In the process, the final sharpened image P is obtained. S The final sharpened image P S The calculation formula is:

[0097]

[0098] In a specific embodiment of the present invention, the filter kernel size of the low-pass mean filter is set to 7, and the USM weighting coefficient ω is set to 0.6.

[0099] Further, step S24 also includes: obtaining the approximate grayscale value distribution range of the edge to be detected based on the enhanced edge image, selecting the optimal value as the threshold for image binarization, and performing binarization processing on the enhanced edge image to obtain a final image containing only black and white dots with grayscale values ​​of 0 and 255, such as... Figure 6 As shown, this ensures that the contours of all tested coded marker points are single-edge contours, thus guaranteeing the robustness and accuracy of edge detection; as Figure 7 This is a schematic diagram of the image preprocessing results.

[0100] Furthermore, in step S3, the traditional Canny operator requires manually setting high and low thresholds to filter out detected edges not in the direction of the gradient maximum during non-maximum suppression (NMS) of the gradient edge image. The selection of high and low thresholds usually requires repeated parameter tuning based on actual results, resulting in poor generalization and robustness. To address this problem, this invention applies an adaptive Canny operator to edge detection of encoded marker points. Its basic principle is as follows: after obtaining the gradient image using the traditional Canny operator, the OTSU algorithm is used to calculate the optimal segmentation threshold T between the foreground and background of the gradient image based on the maximum inter-class variance. B Step S3, which uses the adaptive Canny operator to perform pixel-level edge detection on the binarized image, includes the following steps:

[0101] The gradient image of the binarized image is obtained using the Canny operator;

[0102] Let T be the segmentation threshold of the gradient image. K Then, according to the segmentation threshold T K All pixels in the gradient image are divided into categories higher than the segmentation threshold T. K P1 and below the segmentation threshold T K Let P1 and P2 be the two classes, and their means be denoted as m1 and m2 respectively. Let m3 be the mean of all pixels in the gradient image. Let Q1 and Q2 be the probabilities of a pixel being classified into P1 and P2 respectively. Then the following formula holds:

[0103] Q1m1 + Q2m2 = m3 (14)

[0104] Q1 + Q2 = 1 (15)

[0105] Current segmentation threshold TK The formula for calculating the inter-class variance of the image is as follows:

[0106] σ 2 =Q1(m1-m3) 2 +Q2(m2-m3) 2 (16)

[0107] The segmentation threshold T is set within the grayscale range of (0, 255). K Perform a traversal to find the class variance σ. 2 The largest T K The value, which is the obtained adaptive gradient threshold, is denoted as T. B ;

[0108] T B The high threshold T during the nonmaximization process of the Canny operator H The value obtained by dividing the threshold by 3 and rounding down is taken as the lower threshold T. L This enables adaptive pixel-level edge detection, yielding an edge image; the pixel-level edge detection results are as follows: Figure 8 As shown.

[0109] Furthermore, in step S4, due to the complex background of the measurement environment and the presence of numerous interfering features, the efficiency and accuracy of the identification and localization of coded markers may be affected. Therefore, this invention uses a contour hierarchy structure and multi-scale discriminant factors to extract and segment the Region of Interest (ROI) that may be coded markers, such as... Figure 9 As shown, it includes the following steps:

[0110] S41: Detect connected contours in the edge image to obtain the positional relationship between the edge contours and their contour points;

[0111] S42: Based on the positional relationship between the edge contour and its contour points, establish a contour hierarchy structure of multi-point coding points;

[0112] S43: Obtain the coding marker point region by filtering based on the contour hierarchy structure of the multi-point coding points.

[0113] S44: Filter and remove contours that are too large or too small in the region of the coded marker points based on the contour size information;

[0114] S45: Perform polygon fitting on the contours selected in step S44 according to the contour shape factor, remove contours that do not conform to the shape, and obtain contours that conform to the shape of the region of the coding mark points to be tested.

[0115] S46: Extract and segment the ROI region of the contour that matches the shape of the region of the coding marker to be tested, and obtain the final ROI region of the coding marker.

[0116] Furthermore, in step S41, during detection, all continuous contour points in the edge image are first found and saved to a vector, which corresponds to the connected contours in the edge image.

[0117] Further, in step S42, the contour hierarchy structure is a tree-like contour hierarchy containment relationship, that is, the outer contour contains the inner contour, and the inner contour further contains the embedded contour; for example... Figure 10 As shown, taking a standard multi-point coded marker as an example, there are a total of 8 contours. The corresponding contour relationship is that a rectangular contour contains multiple circular sub-contours, and the parent contour of the circular contour is a rectangular contour with no sub-contours. This allows us to establish a corresponding tree-like contour hierarchy; for example... Figure 11 As shown, Roman numeral I represents the rectangular outline; Roman numerals II, III, IV, V, VI, VII, VIII, and IX represent the various circular sub-outlines within the rectangular outline.

[0118] Further, in step S43, to filter out the coded marker point regions, the filtering criteria are set as follows: the contour has sub-contours but no parent contour. Contour filtering is performed according to the sub-hierarchical structure filtering criteria, and the result is as follows. Figure 12 As shown.

[0119] Furthermore, in step S44, since area and perimeter are the most effective reflections of contour size information, this invention selects area screening factors and perimeter screening factors as judgment criteria to screen the size information of the contour. To exclude contours that are too large or too small, the perimeter screening criterion is set according to the number of points contained in each contour as: minimum perimeter threshold L min =100, maximum perimeter threshold L max =800; Based on the area enclosed by each closed contour, the area filtering criterion is set as: minimum area threshold A min =200, maximum perimeter threshold A max =40000; Further contour filtering is performed using contour size information as the filtering criterion, and the results are as follows: Figure 13 As shown.

[0120] Further, in step S45, considering that the shape of the region to be tested as a coded marker is square, and that distortion caused by the shooting angle may cause the outline of the region to be tested as a parallelogram in the image; to filter out contours that do not conform to the shape, polygon fitting is performed on the contours selected in step S44. All square, rectangular, and parallelogram polygon fitting results will have four fitting control vertices. Based on this contour shape factor, further filtering is performed to remove all contours with a fitting vertex count not equal to 4. The result is as follows: Figure 14 As shown.

[0121] Further, in step S46, the minimum bounding rectangle of each contour-fitted polygon in step S45 is calculated, with length and width dimensions L respectively. P W P And set the size of the ROI region to (L P +50,W P +50); Connecting the off-diagonal vertices of the ROI region yields the final encoded ROI region. The ROI region is then marked on the contour map obtained from edge extraction, as shown below. Figure 15 As shown, the area enclosed by the thick white solid line is the ROI region.

[0122] Furthermore, through steps S1 to S4, this invention transforms the problem of multi-point coding point localization and decoding in complex backgrounds into a ROI region free from other feature interference, thereby improving localization accuracy and decoding robustness. Figure 16 As shown, in step S5, the multi-point coded marker point location and decoding are performed on the final multi-point coded marker point ROI region to obtain the encoded value of the multi-point coded marker point, including the following steps:

[0123] S51: In each ROI region of the final encoded marker point ROI region, subpixel edges of polynomial interpolation are extracted;

[0124] S52: Perform ellipse fitting on the sub-pixel edges extracted in step S51 to obtain all ellipse-like contours in the ROI region.

[0125] S53: Further filter the elliptical-like contours obtained in step S52 based on the two filtering factors of ellipse size and ellipse shape to obtain effective elliptical contours.

[0126] S54: Based on the filtering results of step S53, count the elliptical contours identified in each ROI region to obtain the number of elliptical contours, and determine the validity of the ROI region; if valid, proceed to the next step; if invalid, repeat steps S52 to S54 until a valid ROI region is obtained.

[0127] S55: Based on the coordinates of the four vertices of the rectangular edge contour within several effective ROI regions extracted in step S54, perform an affine transformation on the coding marker point region to obtain several affine images, i.e., images of multi-point coding marker points in the frontal view.

[0128] S56: Filter, binarize, perform edge detection, and ellipse fitting on each of the affine images to obtain the center coordinates of each affine image, and search for collinear points among all the center coordinates. If three points are collinear, the middle point is C, the point closer to C is O, and the point farther away is D; if no three points are collinear, it is considered that there are no coding points in the region.

[0129] S57: Based on the invariance of the cross ratio from A and B to the line COD during the affine process, determine the positions of points A and B and establish a coordinate system;

[0130] S58: Based on the positions of the remaining three three-code points in the coordinate system after the affine transformation, the code value of the multi-point code marker point can be obtained by looking up the corresponding code table.

[0131] Further, in step S51, sub-pixel edge extraction using polynomial interpolation is performed in each ROI region, including the following steps:

[0132] Eight templates, evenly divided into 0-360° directions, are convolved with the image to obtain gradients in the eight directions.

[0133] By selecting the gradient with the largest magnitude and performing non-maximum suppression, the gradient direction θ and gradient magnitude g of the image edge pixel (i,j) can be obtained. Therefore, the extracted sub-pixel edge coordinates (x,y) can be expressed as:

[0134]

[0135] Where gleft, gright, and g0 represent the gradient values ​​of the left, right, and themselves of the edge pixel, respectively, and ω represents the distance from the adjacent pixel to the edge pixel (ω = 1 when the gradient is 0°, 90°, 180°, and 270°, and ω = 1 at other angles). ).

[0136] Furthermore, in step S52, considering that circular coded markers will be distorted into ellipses under a non-perfectly perpendicular shooting angle, in order to obtain a higher accuracy contour fitting effect, this invention performs elliptical fitting on the sub-pixel edges extracted in step S51, obtaining the contour size S of all elliptical contours in the ROI region. E ellipse major axis radius R EL The minor axis radius R of the ellipse EW .

[0137] Furthermore, in step S53, to further improve the accuracy and robustness of the encoding marker positioning and decoding, the present invention further filters the fitted ellipse obtained in step S52, mainly setting two filtering factors: ellipse size and ellipse shape; specifically, the filtering criterion is to set the ellipse size S...E >50, and ellipse shape factor The ellipse is a valid ellipse profile; if no valid ellipse profile is found, repeat steps S52 and S53 until a valid ellipse profile is found.

[0138] Furthermore, in step S54, considering that there should be a total of eight reflective markers within the rectangular area of ​​a single coded marker point, the elliptical contours identified within each ROI area are counted based on the filtering results of step S53, thus obtaining the number of elliptical contours N. E If N E If the value is 8, then the ROI is a valid ROI and can be further processed for multi-point coding marker location and decoding; otherwise, the ROI is invalid.

[0139] Further, in step S55, the coordinates of the four vertices of the rectangular edge contour within the effective ROI region extracted in step S54 are used to perform an affine transformation on the encoded marker point region to recover the image distortion caused by the shooting angle, thereby further improving the robustness of decoding; the size of the image after the affine transformation is set to (500, 500), and the selection of the four affine points is determined by the coordinate relationship of each point, thus obtaining an image of multi-point encoded marker points in the frontal view, the result of which is as follows. Figure 17 As shown.

[0140] Further, in step S56, filtering, binarization, edge detection, and ellipse fitting are performed on the affine image to obtain the coordinates of the center points of the circles within the image. These coordinates are used only for marker point decoding and not for marker point localization. A collinearity search is performed on all center point coordinates, and the angular error between lines formed by different combinations of points is used to determine whether three points are collinear. The formula for calculating the angular error is:

[0141] E θ =|θ(p1,p2)-θ(p2,p3)| (18)

[0142] Let p1, p2, and p3 be any three points; let θ(p1, p2) be the angle between points p1 and p2 forming a straight line; let θ(p2, p3) be the angle between points p2 and p3 forming a straight line; and let the angle error threshold be T. θ =0.1, ° Then the criterion for judging three points as collinear is:

[0143]

[0144] If three points are collinear, the middle point is C, the point closer to C is O, and the point farther away is D; if no three points are collinear, it is considered that there are no coded points in that area.

[0145] Furthermore, in step S55, since the affine transformation has a significant impact on image quality, the result of the affine transformation is only used for decoding the multi-point coded marker. If decoding is successful, the location of the multi-point coded marker is determined by the sub-pixel ellipse center obtained in step S53.

[0146] Further, in step S57, points A and B are determined and a coordinate system is established based on the cross ratio relationship. Specifically, since the cross ratio from A and B to the line COD is invariant during the affine transformation, the positions of points A and B can be determined based on their cross ratio values, and a coordinate system is established. The result of detecting five points O, A, B, C, and D after the affine transformation of one multi-point coded point is as follows: Figure 18 As shown.

[0147] Further, step S58 obtains the encoded value of the encoded marker point from the coordinates of the floating point; based on the position of the three floating encoded points in the coordinate system established in step S57 after the affine transformation, the encoded value of the multi-point encoded marker point can be obtained by looking up the corresponding encoding table; finally, the result of locating and decoding all multi-point encoded marker points in the image is as follows: Figure 19 As shown in the figure, the numbers 6, 7, 8, 9, and 10 represent the decoding results of the multi-point coded markers.

[0148] This invention provides a robust localization and recognition method for multi-point coded markers. By introducing a contour hierarchy structure and multi-scale discriminant factors for multi-point coded markers, it achieves efficient and robust extraction of the ROI region of multi-point coded markers in complex background environments, thereby improving the accuracy and robustness of coded marker localization and decoding.

[0149] It should be noted that: Appendix Figure 8 , Figures 12-15 The black and white inversion was done to make the image outlines clearer.

[0150] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A robust localization and identification method for multi-point coded markers, characterized in that: Includes the following steps: S1: Capture an image containing multiple coded marker points using an industrial camera; S2: Preprocess the image to obtain a binarized image, so that the contours of all the coded marker points to be tested are single-edge contours; S3: Perform pixel-level edge detection on the binarized image using the adaptive Canny operator to obtain the edge image; S4: Detect connected contours in the edge image and establish a contour hierarchy structure. Extract and segment the ROI of possible coding marker points through the contour hierarchy structure and multi-scale discriminant factors to obtain the final multi-point coding marker point ROI region. S5: Perform multi-point coding marker localization and decoding on the final multi-point coding marker ROI region to obtain the coding value of the multi-point coding marker; Step S3, which uses the adaptive Canny operator to perform pixel-level edge detection on the binarized image, includes the following steps: The gradient image of the binarized image is obtained using the Canny operator; Let the segmentation threshold of the gradient image be . Then, according to the segmentation threshold All pixels in the gradient image are divided into categories higher than the segmentation threshold. of and below the segmentation threshold of Two classes, their respective means are denoted as... The mean value of all pixels in the gradient image is denoted as . Pixels are divided into and The probabilities are respectively Then the following formula holds true: （4） （5） Current segmentation threshold The formula for calculating the inter-class variance of the image is as follows: （6） The segmentation threshold is set within the grayscale range of (0, 255). Perform a traversal to find the class variance. The largest The value, which is the obtained adaptive gradient threshold, is denoted as . ; Will As a high threshold in the nonmaximization process of the Canny operator The value obtained by dividing the threshold by 3 and taking the integer part is used as the low threshold. This enables adaptive pixel-level edge detection, resulting in an edge image. In step S5, the multi-point coded markers in the final multi-point coded marker ROI region are located and decoded to obtain the encoded values ​​of the multi-point coded markers, including the following steps: S51: In each ROI region of the final encoded marker point ROI region, subpixel edges of polynomial interpolation are extracted; S52: Perform ellipse fitting on the sub-pixel edges extracted in step S51 to obtain all ellipse-like contours in the ROI region. S53: Further filter the elliptical-like contours obtained in step S52 based on the two filtering factors of ellipse size and ellipse shape to obtain effective elliptical contours. S54: Based on the filtering results of step S53, count the elliptical contours identified in each ROI region to obtain the total number of elliptical contours in the ROI region, and determine the validity of the ROI region; if valid, proceed to the next step; if invalid, repeat steps S52 to S54 until a valid ROI region is obtained; S55: Based on the coordinates of the four vertices of the rectangular edge contour within the effective ROI region extracted in step S54, perform an affine transformation on the coding marker region to obtain several affine images, i.e., images of multi-point coding markers in the frontal view. S56: Filter, binarize, perform edge detection, and ellipse fitting on each of the affine images to obtain the center coordinates of each affine image. Then, perform a collinearity search on all center coordinates. If three points are collinear, the middle point is... ,distance The nearest point is The points that are farther away are If three points are not collinear, then the region is considered to have no coding points. S57: According to the affine process to the straight line Cross-ratio invariance, determining the point The location is determined, and a coordinate system is established; S58: Based on the positions of the remaining 3 floating coding points in the coordinate system after the affine transformation, the coding value of the multi-point coding mark can be obtained by looking up the corresponding coding table.

2. The robust localization and identification method for multi-point coded markers according to claim 1, characterized in that: In step S1, the multi-point coding markers are designed based on the principle of cross-ratio invariance. They include eight centered reflective markers, five of which are fixed points and three are floating coding points. The five fixed points are denoted as follows: ,point This is the total location point for the multi-point coded marker. The three points are collinear, with the midpoint being... ,distance The nearest point is The points that are farther away are ; Then perpendicular to The points are located on the straight line and distributed on both sides of the line; the coordinate system of the multi-point coding marker is established by the three points O, A, and B, and the remaining three coding points are distributed in pairs, with no two adjacent points, in a total of 20 selectable coding positions.

3. The robust localization and identification method for multi-point coded markers according to claim 1, characterized in that: In step S2, the image is preprocessed to obtain a binarized image, ensuring that the contours of all tested coded marker points are single-edge contours, including: S21: Perform grayscale processing on the image to obtain a grayscale image; S22: Perform median filtering on the grayscale image to obtain the filtered image; S23: Using the USM unsharpened mask algorithm, the contour edges of the filtered image are sharpened to obtain an image with enhanced edges; S24: The image after edge enhancement is binarized to obtain a binarized image; the binarized image is a black and white image that only contains black and white points with gray values ​​of 0 and 255, thereby ensuring that the contours of all the coded marker points to be tested are single-edge contours.

4. The robust localization and identification method for multi-point coded markers according to claim 1, characterized in that: Step S4 includes the following steps: S41: Detect connected contours in the edge image to obtain the positional relationship between the edge contours and their contour points; S42: Based on the positional relationship between the edge contour and its contour points, establish a contour hierarchy structure of multi-point coding points; S43: Obtain the coding marker point region by filtering based on the contour hierarchy structure of the multi-point coding points. S44: Filter and remove contours that are too large or too small in the region of the coded marker points based on the contour size information; S45: Perform polygon fitting on the contours selected in step S44 according to the contour shape factor, remove contours that do not conform to the shape, and obtain contours that conform to the shape of the region of the coding mark points to be tested. S46: Extract and segment the ROI region of the contour that matches the shape of the region of the coding marker to be tested, and obtain the final ROI region of the coding marker.

5. The robust localization and identification method for multi-point coded markers according to claim 2, characterized in that: In step S51, sub-pixel edge extraction using polynomial interpolation is performed in each ROI region, including the following steps: Eight templates, evenly divided into 0-360° directions, are established and convolved with the image to obtain gradients in the eight directions. By selecting the gradient with the largest magnitude and performing non-maximum suppression, the edge pixels of the image can be obtained. gradient direction With gradient magnitude Therefore, the sub-pixel edge coordinates are extracted. It can be represented as: （1） in, These represent the gradient values ​​to the left, right, and themselves of an edge pixel, respectively. This represents the distance from adjacent pixels to edge pixels; the gradient is... hour From other angles .

6. The robust localization and identification method for multi-point coded markers according to claim 2, characterized in that: The selection criterion in step S53 is the ellipse size. And the ellipse shape factor The ellipse is a valid elliptical outline; Size of the ellipse outline The radius of the major axis of the ellipse is... The radius of the minor axis of the ellipse is denoted as _____. If no valid ellipse profile is found, repeat steps S52 and S53 until a valid ellipse profile is found.

7. The robust localization and identification method for multi-point coded markers according to claim 2, characterized in that: Step S54 also includes that if the number of elliptical contours is equal to 8, then the ROI is a valid ROI and can be further processed by locating and decoding multi-point coded markers; otherwise, the ROI is invalid.

8. The robust localization and identification method for multi-point coded markers according to claim 2, characterized in that: Step S56 further includes determining whether three points are collinear based on the angular error between lines formed by different combinations of two points, wherein the angular error... The calculation formula is: （2） For any three points, for The angle between two points forming a straight line; for The angle between two points forming a straight line; Set the angle error threshold to The criterion for determining whether three points are collinear is: （3）。

Images