A robust detection method and system based on visual marker codes

Abstract

This application relates to the field of visual inspection and image processing technology, and discloses a robust detection method and system based on visually marked codes. The method includes: edge extraction and geometric adaptation repair of the input image to obtain an edge feature set; corner detection based on local gradient changes; perspective transformation estimation and outlier removal of a preliminary corner candidate set; generation of a preliminary marked code instance set; extraction of the geometric center of each instance from the preliminary set; determination of the distance and orientation angle between adjacent geometric centers to obtain a marked code cluster set; detection of missing corners in each marked code; determination of the coordinates of the missing corners and completion of the boundaries; and obtaining a complete marked code boundary set. This application obtains complete edge features through edge repair, eliminates reliable instances by corner detection and perspective transformation to remove outliers, and then clusters them to form marked code clusters. It uses known corners within the clusters to infer the coordinates of missing corners and complete the boundaries through affine transformation, thus improving the accuracy of marked code detection in complex environments.

Description

Technical Field

[0001] This invention belongs to the field of visual inspection and image processing technology, specifically a robust detection method and system based on visual marker codes. Background Technology

[0002] Visual marker code detection technology is widely used in object tracking, identity recognition, and augmented reality. It quickly locates and identifies targets through specific patterns in images, providing an efficient means of interaction for automated systems. Currently, mainstream methods typically rely on complete corner information of the marker code for spatial localization and image correction to achieve accurate detection and decoding of the marker code.

[0003] However, existing detection methods still have significant limitations in complex scenarios. Image distortion and partial occlusion in dynamic environments often lead to the loss of corner information in the marker codes. Traditional methods rely heavily on the overall structural integrity of the marker codes; once corner points are lost, the system struggles to accurately reconstruct the spatial location, easily resulting in recognition failures or inaccurate positioning. Summary of the Invention

[0004] To address the above problems, this invention provides a robust detection method based on visual marker codes, which solves the technical problem of corner point loss caused by visual marker codes in complex environments.

[0005] To achieve the above objectives, the technical solution adopted by the present invention is as follows: Edges are extracted from the input image, and geometric adaptation and repair are performed on the broken edges to obtain a complete set of edge features; Based on the edge feature set, corner points are detected based on local gradient changes to obtain a preliminary candidate set of corner points; Perspective transformation estimation is performed on the preliminary corner candidate set, and outliers are removed to generate a preliminary set of marked code instances; By extracting the geometric center of each instance from the preliminary set of marked code instances, determining the distance and orientation angle between adjacent geometric centers, and grouping instances whose distance and orientation angle satisfy a preset consistency condition into the same cluster, a set of marked code clusters is obtained. The missing corner points of each marker code within the marker code cluster set are detected, and the coordinates of the missing corner points are determined by affine transformation and the boundaries are filled in to obtain a complete marker code boundary set.

[0006] In one embodiment, after detecting the missing corner points of each tag code within the tag code cluster set, determining the coordinates of the missing corner points through affine transformation, and completing the boundary to obtain a complete tag code boundary set, the method further includes: The convex hull enclosing range of each tag code boundary within the cluster is determined based on the complete tag code boundary set, and the overlapping areas between different convex hull boundaries are identified. The overlapping areas are then geometrically adapted and merged to obtain the final target region boundary division result. Based on the final target region boundary division result, the marker code region is separated from the input image, and the pixel distribution features within the marker code region are extracted; When the pixel distribution features conform to the preset encoding mode, the encoding information is extracted by the decoding algorithm to obtain the mark code recognition result; A noise suppression filter is applied to the image corresponding to the mark code recognition result to obtain an enhanced image after removing interfering pixels; When the sharpness of the enhanced image is higher than a preset sharpness threshold, the contrast of the enhanced image is adjusted in conjunction with the illumination compensation mechanism to generate an optimized marker code instance; Multi-dimensional geometric attributes are extracted from the optimized tag code instances, and clustering algorithms are used to group the instances of the multi-dimensional geometric attributes to obtain the grouped instances; When the grouped instances meet the consistency threshold, the geometric attributes within the group are integrated through a fusion algorithm to obtain the final tag code detection output.

[0007] In one embodiment, the step of determining the convex hull enclosing range of each tag code boundary within a cluster based on the complete tag code boundary set, identifying overlapping regions between different convex hull boundaries, and performing geometric adaptation merging processing on the overlapping regions to obtain the final target region boundary division result includes: For the set of boundary points belonging to the same cluster in the complete set of marker code boundaries, the convex hull algorithm is used to determine the convex hull enclosing range of each marker code boundary, and the convex hull boundaries are obtained. Determine the intersection region between the boundaries of each convex hull; If the area of ​​the intersection region is greater than zero, then the intersection region is determined as an overlapping region, and the geometric center point of the overlapping region is extracted. Using the geometric center point as a reference, and combining the curvature of the two boundary lines of the overlapping area, a curve fitting method is used to smoothly merge the two boundary lines into a new boundary line, generating the merged boundary. Verify whether the merged boundary meets the preset requirements for the number and positional consistency of corner points, and obtain the verification results; Corner points are extracted from the two side boundary lines to generate an original set of boundary corner points, and the missing corner points of the merged boundary relative to the preset number of corner points are detected to generate a set of missing corner point identifiers. Based on the original set of boundary corner points, an affine transformation is used to determine the coordinate positions corresponding to the missing corner point identifier set. The corner points are then inserted into the merged boundary to obtain the target region boundary division result.

[0008] In one embodiment, the step of extracting the geometric center of each instance from the preliminary set of marked code instances, determining the distance and orientation angle values ​​between adjacent geometric centers, and grouping instances whose distance and orientation angle values ​​satisfy a preset consistency condition into the same cluster to obtain a set of marked code clusters includes: Extract the geometric center of each instance from the initial set of marked code instances to obtain the set of geometric center points; By determining the Euclidean distance and the direction angle of the line connecting each pair of adjacent center points in the geometric center point set, the distance and direction angle values ​​of adjacent instances are obtained. Instances whose distance value is less than a first preset threshold and whose difference in orientation angle value is less than a second preset threshold are marked as candidate instances of the same type, and density clustering algorithm is used to cluster the candidate instances of the same type to generate an initial cluster. If the spatial location offset of any instance within an initial cluster exceeds a preset offset threshold, the instance is removed from the current cluster, and the cluster boundary is adjusted to obtain a set of labeled code clusters.

[0009] In one embodiment, the step of detecting missing corner points of each tag code within the tag code cluster set, determining the coordinates of the missing corner points through affine transformation, and completing the boundary to obtain a complete tag code boundary set includes: The missing corner identifier is determined by detecting the number of corner points of each marker in the marker code cluster set; Extract the coordinates of known corner points from the cluster to which the missing corner point's tag code belongs, and use them as a reference point set; Based on the reference point set, the affine transformation matrix is ​​solved using the least squares method to obtain the initial affine transformation matrix; The transformation error of the initial affine transformation matrix is ​​determined by geometric constraints; If the transformation error is within a preset tolerance range, then the initial affine transformation matrix is ​​determined to be the target affine transformation matrix; Substitute the standard coordinates corresponding to the missing corner markers into the target affine transformation matrix to determine the actual coordinates corresponding to the missing corner markers, and use Bézier curves to smooth and correct the boundaries to obtain a complete set of marker code boundaries.

[0010] In one embodiment, the step of extracting edges from the input image and performing geometric adaptation repair on broken edges to obtain a complete set of edge features includes: Perform multi-scale edge detection on the input image to generate the original edge distribution map; Local gradient enhancement is performed on the original edge distribution map to extract continuous edge segments; Detect the break points in the continuous edge segments and extract the geometric shape information of the local area at the break point; If the geometric shape information meets the preset geometric adaptation conditions, then the fracture is fitted and connected using a Bézier curve to obtain the repaired edge segment. A morphological closing operation is performed on the repaired edge segments to obtain a complete set of edge features.

[0011] In one embodiment, the step of detecting corner points based on local gradient changes according to the edge feature set to obtain a preliminary corner point candidate set includes: Based on each edge point in the edge feature set, a local gradient change matrix is ​​determined, and a corner response value is determined according to the local gradient change matrix. Points with corner response values ​​greater than a preset corner threshold are marked as potential corner points, and a potential corner point set is generated. Determine whether there is an interruption in the edge segment corresponding to the potential corner point set. If there is an interruption, extract the adjacent edge segments at both ends of the interruption, and use a linear interpolation method to generate connection points to repair the interruption, thereby obtaining the repaired corner point sequence. Based on a preset point set matching accuracy threshold, a reliability assessment is performed on each corner point in the repaired corner point sequence to remove corner points with matching errors greater than the accuracy threshold, thereby obtaining a filtered corner point set. The filtered set of corner points is clustered to merge corner points with distances less than a preset distance threshold into a single corner point, thus obtaining a preliminary candidate set of corner points.

[0012] In one embodiment, the step of performing perspective transformation estimation on the preliminary corner candidate set and removing outliers to generate a preliminary tag code instance set includes: Matrix construction steps: By selecting multiple pairs of corresponding points from the preliminary corner point candidate set, a perspective transformation matrix is ​​constructed as a candidate transformation matrix; Inner point screening and statistics steps: Project the remaining corner points in the preliminary corner point candidate set through the candidate transformation matrix, determine the residual between the projected point and the corresponding actual point, mark the corner points whose residual is less than the preset inner point threshold as inner points, and count the number of inner points; The matrix construction step and the interior point screening and statistics step are executed iteratively. After reaching the preset number of iterations, the candidate transformation matrix with the most interior points is selected as the target transformation matrix, and the corresponding set of interior points is taken as the effective corner point set. Based on the set of valid corner points, a boundary consistency verification is performed using a preset marker code template to eliminate abnormal corner points whose boundary overlap with the preset marker code template is lower than a preset overlap threshold, thereby obtaining a preliminary set of marker code instances.

[0013] Furthermore, to achieve the above objectives, this application also proposes a robust detection device based on visual marker codes, the robust detection device based on visual marker codes comprising: The edge extraction and break repair module is used to extract edges from the input image and perform geometric adaptation repair on broken edges to obtain a complete set of edge features. The local gradient corner detection module is used to detect corners based on local gradient changes according to the edge feature set, and obtain a preliminary corner candidate set. The perspective transformation estimation and anomaly removal module is used to perform perspective transformation estimation on the preliminary corner point candidate set and remove anomalies to generate a preliminary set of marked code instances. The geometric consistency clustering module is used to extract the geometric center of each instance from the preliminary set of marked code instances, determine the distance value and orientation angle value between adjacent geometric centers, and group instances whose distance value and orientation angle value satisfy a preset consistency condition into the same cluster to obtain a set of marked code clusters; The missing corner point completion and boundary repair module is used to detect the missing corner points of each tag code in the tag code cluster set, determine the coordinates of the missing corner points through affine transformation and complete the boundary to obtain a complete tag code boundary set.

[0014] In addition, to achieve the above objectives, this application also proposes a robust detection system based on visual marker codes, the system comprising: an edge extraction and break repair module, a local gradient corner detection module, a perspective transformation estimation and anomaly removal module, a geometric consistency clustering module, and a missing corner completion and boundary repair module; The edge extraction and break repair module is used to extract edges from the input image and perform geometric adaptation repair on broken edges to obtain a complete set of edge features. The local gradient corner detection module is used to detect corners based on local gradient changes according to the edge feature set, and obtain a preliminary corner candidate set. The perspective transformation estimation and anomaly removal module is used to perform perspective transformation estimation on the preliminary corner point candidate set and remove anomalies to generate a preliminary set of marked code instances. The geometric consistency clustering module is used to extract the geometric center of each instance from the preliminary tag code instance set, determine the distance value and orientation angle value between adjacent geometric centers, and group instances whose distance value and orientation angle value satisfy a preset consistency condition into the same cluster to obtain a tag code cluster set; The missing corner point completion and boundary repair module is used to detect the missing corner points of each tag code in the tag code cluster set, determine the coordinates of the missing corner points through affine transformation and complete the boundary to obtain a complete tag code boundary set.

[0015] The proposed technical solution obtains a complete set of edge features by performing edge extraction and break repair on the input image, ensuring the continuity of subsequent corner detection. In the corner detection stage, the local gradient change matrix of each edge point is calculated, and corner features are quantified using a corner response function. A preliminary candidate set of corner points is obtained through threshold comparison, break repair, reliability assessment, and clustering. A preliminary set of marked code instances is then obtained through perspective transformation estimation and outlier removal, effectively filtering out false detections. By extracting the geometric center and grouping instances with consistent distance and direction into the same cluster, a set of marked code clusters is obtained, providing reliable known point dependencies for missing corner inference. Finally, affine transformation is used to determine the coordinates of the missing corner points and complete the boundaries, obtaining a complete set of marked code boundaries. This application can robustly restore complete boundaries even in complex scenarios with broken marked code edges, missing corners, or partial occlusion, improving detection accuracy and stability and ensuring the reliability of target region segmentation. Attached Figure Description

[0016] Figure 1 A flowchart illustrating an embodiment of the robust detection method based on visual marker codes in this application; Figure 2 This is a flowchart illustrating Embodiment 2 of the robust detection method based on visual marker codes in this application; Figure 3 This is a flowchart illustrating Embodiment 3 of the robust detection method based on visual marker codes in this application; Figure 4 This is a schematic diagram of the module structure of the robust detection device based on visual marker codes according to an embodiment of this application. Detailed Implementation

[0017] To enable those skilled in the art to better understand the technical solution, the present invention will be described in detail below with reference to embodiments. The description in this part is only exemplary and explanatory, and should not be used to limit the scope of protection of the present invention in any way.

[0018] Existing detection methods based on visual marker codes have the following drawbacks: First, they cannot effectively repair broken edges when extracting edges from the input image, resulting in an incomplete set of edge features and affecting the accuracy of subsequent corner detection. Second, they rely solely on single corner detection and perspective transformation estimation, lacking utilization of the spatial relationships between multiple marker code instances. When marker codes are partially occluded or corners are missing, it is difficult to infer the missing locations from known points, resulting in the inability to obtain a complete set of marker code boundaries. This is especially problematic in scenarios with dense multiple marker codes, leading to a high failure rate in target region segmentation.

[0019] Therefore, in order to overcome the above-mentioned defects, this application provides a solution: complete edge features are obtained by performing edge extraction and break repair on the input image; preliminary corner point candidates are obtained by detecting corner points based on local gradient changes; preliminary marker code instances are generated by estimating and eliminating abnormal points through perspective transformation; geometric centers are extracted and marker code clusters are obtained by clustering based on the distance and orientation consistency of adjacent instances; finally, missing corner points within the cluster are detected and their coordinates are determined by affine transformation and the boundaries are filled in to obtain a complete marker code boundary set.

[0020] It should be noted that the execution entity of each embodiment of this application can be a computing service system with data processing, network communication, and program execution functions, such as an electronic system capable of implementing the above functions, a robust detection system based on visual marker codes, etc. The following description uses a robust detection system based on visual marker codes (hereinafter referred to as "the System") as an example to illustrate the following embodiments.

[0021] Based on this, embodiments of this application provide a robust detection method based on visually labeled codes, referring to... Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the robust detection method based on visual marker codes in this application.

[0022] In this embodiment, the robust detection method based on visual marker codes includes steps S10 to S50: Step S10: Extract edges from the input image and perform geometric adaptation repair on the broken edges to obtain a complete set of edge features.

[0023] In fields such as intelligent recognition and augmented reality, visual marker codes, as information carriers embedded with specific geometric patterns, are widely used for object tracking, identity recognition, and spatial positioning. In practical applications, changes in lighting, partial occlusion, or image noise often cause the marker codes to have broken edges or missing corner points, thus affecting recognition accuracy. Visual marker codes consist of alternating black and white squares or specific topological structures, and their detection relies on the stable extraction of complete boundary contours and key corner points from the image to support subsequent perspective correction and decoding.

[0024] It's important to note that edge extraction refers to detecting curves formed by connecting pixels with drastic brightness changes in an image. These curves correspond to the contour boundaries of the marker code. Due to noise or low contrast, the detected edges often appear broken. Multi-scale edge detection techniques, such as the Canny algorithm based on Gaussian pyramids, decompose the image into three scale layers (scale factors of 1.0, 0.5, and 0.25). At each scale, Gaussian smoothing (standard deviation σ=2.0) is used to remove noise. Then, the gradient magnitude is calculated, and dual thresholds (low threshold 50, high threshold 150) are set to filter edges. Finally, the results from each scale are merged to capture edge details from coarse to fine. Local gradient enhancement techniques, such as using the Sobel operator to traverse the edge map with a 5×5 window, multiply weak edge points with gradient magnitudes below 30 by an enhancement factor of 1.5, and use non-maximum suppression to make the edges single-pixel, thereby improving the edge continuity in low-contrast areas.

[0025] The geometric adaptation repair strategy addresses edge breaks. Within a 10×10 pixel region around the break point, edge orientation consistency is calculated. If the orientation deviation is less than 15 degrees, a Bézier curve is used to fit the break segment, with control points generated based on the gradient directions at both ends of the break. This repair improves edge smoothness by approximately 20%. Morphological closing operations (3×3 structuring elements) further remove isolated noise points. Through these repairs, the false detection rate for edge detection can be reduced to below 5%, ultimately yielding a complete and continuous set of edge features.

[0026] Step S20: Based on the edge feature set, detect corner points based on local gradient changes to obtain a preliminary corner point candidate set.

[0027] It's important to note that corner points are points in an image where two edges intersect or where curvature abruptly changes; in visual tagging, they correspond to the vertices of a quadrilateral boundary. Local gradient change corner detection is implemented using the Harris corner detection algorithm. Its principle is to calculate the gradient change matrix for each pixel (after Gaussian smoothing with a standard deviation of 2.0) to obtain the corner response value. A gradient threshold of 0.1 is set; pixels with response values ​​greater than 0.05 are marked as potential corner points. For example, within a 100×100 pixel area, 15 points with response values ​​exceeding the threshold may be detected. These potential corner points constitute the initial candidate set, but may include false points caused by edge breaks.

[0028] If an edge segment is detected as broken, for example, an edge segment breaks at coordinates (50, 50) and is missing approximately 10 pixels, it is repaired through linear interpolation of adjacent edge segments. The equation of the line y = 0.4x + 30 for the two endpoints (45, 48) and (55, 52) is calculated, the coordinates of the missing point are generated by interpolation and connected, thus completing the edge. The point set matching accuracy requirement is set with a matching error threshold of 2.5 pixels. The nearest neighbor matching algorithm is used to filter the initially detected corner points, removing points with errors greater than the threshold. For example, 3 points with excessively large residuals are removed from 15 corner points, retaining 12 reliable key points. Furthermore, a clustering algorithm such as K-means (cluster number K = 3) is used to merge overly dense points (such as points with a spacing of less than 5 pixels), finally optimizing to obtain 10 evenly distributed corner points, forming a preliminary corner point candidate set.

[0029] Step S30: Perform perspective transformation estimation on the preliminary corner candidate set and remove outliers to generate a preliminary tag code instance set.

[0030] Perspective transformation estimation refers to finding a projection transformation matrix that maps quadrilateral regions in an image to a standard rectangular plane. This transformation requires four corner points as input parameters. Outlier removal involves identifying and removing points with significant deviations from the transformation model, which may be false corners caused by noise or false detections. Through perspective transformation estimation and outlier removal, the system can filter out a subset of corner points consistent with the geometry of the standard marker code from a noisy candidate set. Each such subset corresponds to a candidate marker code instance, and multiple instances together constitute a preliminary marker code instance set.

[0031] In the specific implementation, four corner points are randomly selected from the initial candidate set, and their corresponding perspective transformation matrices are calculated. To evaluate the reliability of this transformation matrix, the system maps the coordinates of all corner points using this matrix and calculates the residual error between the mapped coordinates and the original observed coordinates. A residual error threshold of 5 pixels is set; if the residual of a corner point is less than this threshold, it is considered an inlier; otherwise, it is considered an outlier. The system repeats the above random selection and calculation process multiple times (e.g., iterates 1000 times), retaining the transformation model with the most inliers. Assuming that in a certain iteration, the number of inliers reaches 6, and the average residual error of these 6 points is 3.2 pixels (below the 5-pixel threshold), then the transformation model is accepted, and those 6 inliers are considered the valid set of corner points, while the remaining outliers are discarded. Furthermore, based on the quadrilateral boundary formed by these valid corner points, and combined with boundary verification consistency (for example, comparing the quadrilateral side length with the preset marker code template side length of 100 pixels, and considering it valid if the overlap is higher than 85%), the system finally outputs a preliminary marker code instance set containing multiple valid corner point subsets.

[0032] Step S40: Extract the geometric center of each instance from the preliminary tag code instance set, determine the distance value and orientation angle value between adjacent geometric centers, and group instances whose distance value and orientation angle value meet the preset consistency condition into the same cluster to obtain a tag code cluster set.

[0033] The geometric center refers to the coordinates of the center point of the area covered by each tag code instance, calculated as the average of the coordinates of all boundary corner points of that instance. The distance value uses Euclidean distance to measure the straight-line length between two geometric centers. The orientation angle value refers to the angle between a directed line segment pointing from one geometric center to another and the horizontal axis. Preset consistency conditions include a distance threshold and an orientation angle deviation threshold. Only adjacent instances that simultaneously meet both conditions are considered to have spatial arrangement consistency and should belong to the same cluster. Through this clustering operation, the system can organize scattered tag code instances into multiple tag code clusters according to their layout rules. Each cluster represents a group of tags that are spatially compact and oriented in a regular manner.

[0034] In the specific implementation, the system iterates through each instance in the initial set of marked code instances and calculates its geometric center coordinates. For example, if the center of instance A is (50.5, 60.3) and the center of instance B is (55.2, 62.1), then the Euclidean distance between the two points is sqrt((55.2-50.5)²+(62.1-60.3)²)=5.0 units, and the orientation angle is arctan((62.1-60.3) / (55.2-50.5))≈20 degrees. Setting the distance threshold to 10 units and the orientation angle deviation threshold to 15 degrees, the distance of 5.0 is less than 10 and the orientation angle of 20 degrees is within the allowable deviation range. Therefore, instance A and instance B satisfy the consistency condition. The system uses a density clustering algorithm (e.g., DBSCAN, setting the neighborhood radius eps=10 units and the minimum number of samples min_samples=5) to group all instances, connecting instances that satisfy the consistency of distance and orientation angle into clusters. If 10 instances within a certain region all satisfy the above conditions, they are grouped into the same cluster. After clustering all instances, the system outputs multiple tag code clusters, such as three clusters containing 9, 12, and 15 instances respectively. The instances within each cluster have similar spatial distribution patterns, thus providing an inference basis based on intra-cluster consistency for subsequent missing corner completion.

[0035] Step S50: Detect the missing corner points of each marker code in the marker code cluster set, determine the coordinates of the missing corner points through affine transformation and complete the boundary to obtain a complete marker code boundary set.

[0036] It should be noted that missing corner points refer to marker vertices that were not successfully extracted in previous corner detection steps due to occlusion, uneven lighting, or image quality issues, but which should theoretically exist. Affine transformation is a linear transformation that preserves the parallelism of straight lines and includes rotation, scaling, translation, and shearing components. The transformation parameters can be uniquely determined by three non-collinear known points.

[0037] Within the same tag code cluster, the system uses known corner points of other complete tag codes in the cluster or existing corner points within the same tag code to estimate the affine transformation matrix, and then infers the coordinates of the missing corner points. Boundary completion involves connecting the inferred corner points with the existing corner points in sequence to form closed polygonal boundaries, ultimately obtaining a complete tag code boundary set where each tag code has a complete quadrilateral boundary.

[0038] In the specific implementation, each cluster in the set of marker codes is traversed. For each marker code instance within a cluster, the number of detected corner points is checked. If a quadrilateral marker code should have four corner points but only three are detected, then a missing corner point is detected. The system uses the three existing corner points within the marker code to calculate the affine transformation matrix: for example, if the corner point coordinates are known to be (100, 200), (200, 200), and (100, 300), the missing fourth corner point should be near (200, 300). The affine transformation parameters are solved using the least squares method to obtain the transformation matrix M. After mapping the three known corner points to the standard positions, the mapping coordinates of the missing point are inferred. The transformation error tolerance range is set to 5 pixels. The Euclidean distance between the inferred coordinates and the theoretical position is calculated. If the distance is 2.5 pixels, the tolerance condition is met. When multiple marker codes with complete corner points exist within the same cluster, the system can also use the geometric consistency of these complete instances to estimate more accurate affine transformation parameters, thereby improving the accuracy of missing corner point inference. After obtaining the coordinates of the missing corner points, the existing corner points and the inferred corner points are connected in quadrilateral order, and boundary smoothness correction is performed (e.g., ensuring that the included angle between adjacent sides is within a reasonable range). Finally, the complete boundary point set of each tag code is output. For example, the original point set is updated to {(100,200), (200,200), (200,300), (100,300)}, thus obtaining the complete tag code boundary set.

[0039] This application extracts continuous edges using multi-scale edge detection and local gradient enhancement techniques, completes broken edges using a geometric adaptation repair strategy, and infers missing corner points using linear interpolation and affine transformation. Simultaneously, it employs random sampling consistency and clustering algorithms to optimize boundary partitioning and instance grouping. Finally, noise suppression and illumination compensation mechanisms enhance image clarity, achieving accurate extraction of encoded information. The core innovation lies in the geometric constraint technology for edge repair and corner point completion, ensuring the robustness of the marker code in complex environments, significantly improving detection accuracy and stability, and providing reliable assurance for the application of visual marker codes in diverse scenarios.

[0040] It should be noted that the robust detection method based on visual marker codes described in this application is not limited to the specific image resolution, threshold parameters, or specific marker code encoding formats (such as QR codes) mentioned in the foregoing examples. The essence of this method lies in establishing a general geometric processing framework encompassing edge restoration, corner detection, perspective transformation estimation, clustering, and affine completion of missing corners. Its core logic relies on the geometric structural features of the marker codes in the image, rather than specific application environments or encoded content. Therefore, this method can be widely applied to various visual marker code detection scenarios, including but not limited to marker code arrays of different scales, marker patterns with non-standard geometric shapes, positioning in industrial production lines with complex lighting or partial occlusion, multi-target tracking in augmented reality, and landmark recognition in UAV visual navigation. Any technical scenario that requires utilizing the edge continuity and corner geometric constraints of marker codes in an image to improve detection robustness can adopt the processing flow proposed in this application. That is, the scope of protection of this application is not limited to the specific parameters or typical use cases set for ease of understanding in the embodiments; any visual marker code detection scheme based on the same geometric restoration and affine inference principles falls within the technical concept of this application.

[0041] Based on the first embodiment of this application, in the second embodiment of this application, the content that is the same as or similar to that in Embodiment 1 above can be referred to the above description, and will not be repeated hereafter. Based on this, please refer to... Figure 2 Step S50 may be followed by steps S01 to S07: Step S01: Determine the convex hull enclosing range of each tag code boundary within the cluster based on the complete tag code boundary set, identify the overlapping areas between different convex hull boundaries, perform geometric adaptation merging processing on the overlapping areas, and obtain the final target region boundary division result.

[0042] It should be understood that after obtaining the complete set of marked code boundaries, the convex hull enclosing range of each marked code boundary within the cluster is further calculated, and the overlapping areas between different convex hull boundaries are identified. Then, a geometric adaptation-based merging process is performed on these overlapping areas to obtain the final target region boundary partitioning result.

[0043] The convex hull refers to the smallest convex polygon that can encompass all boundary points of the marker codes within a cluster. This convex hull visually reflects the overall coverage area of ​​the entire marker code cluster. When multiple marker codes are closely arranged in an image, their respective convex hull boundaries often overlap. For example, the overlap area of ​​the convex hulls of two adjacent marker codes can reach 15.3 square units. In this case, the system calculates the geometric center of the overlapping region and analyzes the rate of change of boundary curvature (e.g., setting a curvature threshold of 0.05). A local region geometric adaptation method is used to smoothly merge the overlapping boundaries. After merging, the boundary length may be reduced to 85% of the original. Through this merging process, the originally scattered marker code boundaries are integrated into a coherent and non-overlapping complete region division.

[0044] Step S02: Based on the final target region boundary division result, the marker code region is separated from the input image, and the pixel distribution features within the marker code region are extracted.

[0045] It should be noted that image segmentation refers to the process of separating the foreground (marker code) from the background based on boundary division results. For example, using connected component analysis, a connected component area threshold of 1000 pixels can be set to filter out noise regions smaller than this threshold, thereby extracting candidate regions that meet the marker code size. Pixel distribution features include statistical information such as the proportion of black and white pixels within the region, grayscale histogram distribution, or local texture patterns. These features reflect the coding structure of the marker code. Assuming the marker code region size is 200×200 pixels, the statistics show that black pixels account for 45.5% and white pixels account for 54.5%. This distribution feature can be used as a basis for determining whether the preset coding pattern is met.

[0046] Step S03: When the pixel distribution features conform to the preset encoding mode, the encoding information is extracted by the decoding algorithm to obtain the mark code recognition result.

[0047] The preset encoding mode refers to the black and white module arrangement rules specified by standard marking codes (such as QR codes or custom visual markings), requiring the black and white pixel ratio to be close to 1:1 and the error range to be within a preset percentage (such as ±10%). If the aforementioned black and white pixel ratio of 45.5% deviates from 50% by 4.5%, which falls within the allowable range, then the distribution characteristics are deemed to meet the requirements.

[0048] Furthermore, a decoding algorithm, such as a QR code decoding method based on the ZBar library, is used to read the pixel matrix and parse out the encoded information. This encoded information can be a binary sequence or an encoded string of numbers. The system outputs the decoded information as the recognition result, which is itself valid data read from the tag code and can proceed to subsequent processing without comparison with an external database.

[0049] Step S04: Apply a noise suppression filter to the image corresponding to the mark code recognition result to obtain an enhanced image after removing interfering pixels.

[0050] For the image region corresponding to the above-mentioned marker code recognition results, a noise suppression filter is applied to remove potential interfering pixels, thereby obtaining an enhanced marker code image. The noise suppression filter is implemented as a median filter, using a 3×3 window to traverse the image. After sorting the gray values ​​of all pixels within the window, the median value is taken as the new value of the center pixel, which can effectively remove isolated interference points such as salt-and-pepper noise. For example, if a pixel in the original image has a gray value of 255 while the gray values ​​of surrounding pixels are mostly around 120, after filtering, the pixel value is adjusted to 122, and the image smoothness is significantly improved. The enhanced image obtained after filtering retains the main edge information of the marker code while suppressing background noise.

[0051] Step S05: When the sharpness of the enhanced image is higher than a preset sharpness threshold, the contrast of the enhanced image is adjusted in conjunction with the illumination compensation mechanism to generate an optimized marker code instance.

[0052] After obtaining the enhanced marker code image, the system evaluates the image's sharpness. When the sharpness exceeds a preset sharpness threshold, an illumination compensation mechanism is used to adjust the image contrast, ultimately generating an optimized marker code instance. Sharpness is quantified by calculating the mean edge intensity using the Laplacian operator. A preset threshold of 50 is set; a score of 60 indicates that the sharpness meets the standard. The illumination compensation mechanism uses histogram equalization to expand the image's grayscale range from, for example, [50, 200] to [0, 255], while simultaneously using gamma correction (with a gamma value set to 1.2) to enhance the brightness of dark areas, thereby improving the marker code's contrast and detail visibility. The optimized marker code instance obtained through this process outperforms the original image in edge sharpness and brightness levels. Its structural similarity index (SSIM) compared to the standard template reaches over 0.85, significantly improving the reliability of subsequent feature extraction.

[0053] Step S06: Extract multi-dimensional geometric attributes from the optimized tag code instances, and use a clustering algorithm to group the instances of the multi-dimensional geometric attributes to obtain the grouped instances.

[0054] It should be noted that multi-dimensional geometric attributes include features such as the area, perimeter, shape factor (i.e., the ratio of area to the square of perimeter), and the ratio of side lengths between corner points. For example, through edge detection and contour extraction, the area of ​​a certain marker is calculated to be 1200 pixels squared, the perimeter to be 150 pixels, and the shape factor to be 0.85. The K-means clustering algorithm (setting the number of cluster centers K=3 and the number of iterations to 10) is used to group these feature vectors. The first group of feature centers has (area 1150, perimeter 148, shape factor 0.83), the second group has (area 800, perimeter 120, shape factor 0.75), and the third group has (area 1500, perimeter 180, shape factor 0.90). Instances within each group are grouped together because of their similar geometric attributes.

[0055] Step S07: When the grouped instances meet the consistency threshold, the geometric attributes within the group are integrated through a fusion algorithm to obtain the final tag code detection output.

[0056] When the instances grouped above meet the preset consistency threshold, the geometric attributes within the group are integrated using a fusion algorithm, and the final tag code detection result is output. The consistency threshold measures the dispersion of the geometric attributes of each instance within the same group. For example, if the variance threshold for the shape factor is set to 0.05, and the calculated variance of the shape factor within a group is 0.03, which is less than the threshold, then the group is considered to meet the consistency requirements. The fusion algorithm uses a weighted average strategy, assigning weights according to the number of instances in each group. For example, the first group contains 10 instances (weight 0.5), the second group contains 5 instances (weight 0.3), and the third group contains 5 instances (weight 0.2). The weighted and integrated geometric attribute output is (area 1100, perimeter 145, shape factor 0.82). This fusion result serves as the final tag code detection output and can be used to characterize the overall geometric statistical features of the tag codes within the region. If the deviation of the fusion result from the preset standard value is less than 5%, it is output directly; otherwise, secondary verification logic is triggered to ensure detection accuracy.

[0057] In one implementation, step S01 may include: For the set of boundary points belonging to the same cluster in the complete set of marker code boundaries, the convex hull algorithm is used to determine the convex hull enclosing range of each marker code boundary, and the convex hull boundaries are obtained. Determine the intersection region between the boundaries of each convex hull; If the area of ​​the intersection region is greater than zero, then the intersection region is determined as an overlapping region, and the geometric center point of the overlapping region is extracted. Using the geometric center point as a reference, and combining the curvature of the two boundary lines of the overlapping area, a curve fitting method is used to smoothly merge the two boundary lines into a new boundary line, generating the merged boundary. Verify whether the merged boundary meets the preset requirements for the number and positional consistency of corner points, and obtain the verification results; Corner points are extracted from the two side boundary lines to generate an original set of boundary corner points, and the missing corner points of the merged boundary relative to the preset number of corner points are detected to generate a set of missing corner point identifiers. Based on the original set of boundary corner points, an affine transformation is used to determine the coordinate positions corresponding to the missing corner point identifier set. The corner points are then inserted into the merged boundary to obtain the target region boundary division result.

[0058] In the specific implementation, the convex hull algorithm is used to extract the convex hull bounding range of each marker code within a cluster from the complete set of marker code boundaries. Then, geometric adaptation and merging are performed by identifying overlapping areas between convex hulls. The convex hull bounding range refers to the smallest convex polygon that can contain all boundary points of a single marker code. The convex hull of each marker code is calculated using, for example, the Graham scan algorithm, resulting in a set of convex hull boundaries. Since marker codes within the same cluster may be closely arranged or even partially overlap in the image, intersection areas will occur between the convex hull boundaries of different marker codes. The system calculates the intersection of every two convex hull boundaries one by one. If the area of ​​the intersection area is greater than zero (for example, the overlap area of ​​two convex hulls is 15.3 square pixels), the area is determined to be an overlapping area, and its geometric center point is extracted. This geometric center point serves as the spatial reference for subsequent boundary merging, guiding the positional alignment of the merging operation.

[0059] After determining the geometric center point of the overlapping region, and considering the curvature of the boundary lines on both sides of the overlapping region, a curve fitting method is used to smoothly merge the two boundary lines into a new boundary line. Specifically, the system extracts the discrete point set of the original boundary lines on the left and right sides of the overlapping region, calculates the curvature at each point (for example, setting a curvature change rate threshold of 0.05), and uses the geometric center point as the anchor point. A Bézier curve or B-spline fitting algorithm is used to generate a continuous and smooth new boundary line to replace the original two intersecting or overlapping boundaries. The merged new boundary line should maintain geometric continuity with the adjacent non-overlapping boundaries, while eliminating redundant line segments in the original overlapping region. For example, the convex hull boundaries of two adjacent marker codes originally formed an irregular zigzag intersection in the overlapping region. After curvature-guided curve fitting, a smooth arc is generated, reducing the total length of the merged boundary by about 15% compared to before merging. After the merge is completed, the system verifies whether the merged boundary meets the preset requirements for the number and position consistency of corner points. For example, it checks whether the number of corner points on the merged boundary reaches the four corner points that a quadrilateral should have, and whether the ratio of the side lengths between each corner point is within a reasonable range (such as the ratio of the longest side to the shortest side being less than 1.5). After the verification is passed, the corner point completion stage begins.

[0060] It should be understood that, due to the potential loss or positional shift of original corner points during the merging process, the system needs to further extract the original corner points from both boundary lines to generate an original boundary corner point set, and detect any missing corner points on the merged boundary relative to a preset number, generating a missing corner point identifier set. The original boundary corner point set stores the coordinates of all detected corner points on both boundaries before merging. The system compares the number of corner points on the merged boundary with a preset standard (e.g., a marker code should contain four corner points). If the number is insufficient, the system marks the missing positions and generates a missing corner point identifier set.

[0061] Furthermore, based on the known complete corner points in the original boundary corner point set, an affine transformation is used to determine the coordinate positions corresponding to the missing corner point identifier set. The affine transformation parameters can be obtained by solving for three non-collinear known corner points in the original boundary corner point set using the least squares method, thereby inferring the coordinates of the missing corner points. For example, if only three corner points are detected after merging, and a fourth corner point is missing, the coordinates of the missing point are predicted after calculating the affine transformation matrix using the three known points, and the missing corner point is inserted into the corresponding position of the merged boundary. In the final target region boundary partitioning result, each region has a complete convex hull enclosed range, a smoothly merged boundary, and a set of corner points with consistent number and position.

[0062] In one implementation, step S40 may include steps S401 to S404: Step S401: Extract the geometric center of each instance from the preliminary set of marked code instances to obtain a set of geometric center points.

[0063] It should be noted that the geometric center, representing the spatial location of each marker instance, is calculated by the arithmetic mean of the coordinates of all boundary corner points of that instance. Since each instance in the initial set of marker instances may contain a different number of corner points (e.g., three or four), the system first iterates through the corner coordinates of each instance, summing the x and y coordinate values ​​and dividing by the number of corner points to obtain a unique center point coordinate. This calculation process does not depend on whether the instance is complete; even if an instance is missing some corner points due to occlusion, its geometric center can still be estimated using the existing corner points, thus ensuring the continuity of the point set. For example, the geometric center of a triangular marker instance with only three detected corner points is the average of the coordinates of the three vertices; while a complete quadrilateral instance uses the average of the four corner points. After traversing all instances, all center points constitute a two-dimensional spatial point set as the geometric center point set, which preserves the relative positional relationships between the original instances.

[0064] Obtaining the geometric center point set essentially involves spatial dimensionality reduction of discrete tokenized code instances, simplifying each instance from a polygonal outline to a single point. This simplification sacrifices shape details but preserves positional information, making the calculation of distances and directions between adjacent instances efficient and robust. Furthermore, the geometric center point set does not require instances to be equally spaced in physical space; any regular or irregularly laid-out tokenized code array can reflect its spatial distribution pattern through this point set. During extraction, the system does not require pre-filtering or smoothing of the point set to preserve any minor positional deviations that may exist in the original instances. These deviations will be used as the basis for removing outliers in subsequent clustering and filtering steps.

[0065] Step S402: By determining the Euclidean distance and the direction angle of the line connecting every two adjacent center points in the geometric center point set, the distance value and direction angle value of adjacent instances are obtained.

[0066] In the geometric center point set, "adjacent" refers to the point-to-point connection established according to a certain spatial traversal order (e.g., ordered by x-coordinate and connected sequentially, or determined based on Delaunay triangulation to identify natural neighbors). For each center point, the Euclidean distance between it and its nearest neighbor is calculated, which is the straight-line length of the line connecting the two points; simultaneously, the angle between this line and the positive direction of the horizontal axis is calculated as the orientation angle value. This angle is usually expressed in radians or degrees, ranging from 0 to 180 degrees (regardless of the orientation, i.e., pointing from point A to point B and pointing from point B to point A are considered the same orientation angle). For example, two points with coordinates (50.5, 60.3) and (55.2, 62.1) are 5.0 units apart, with an orientation angle of approximately 20 degrees. The distance and orientation angle values ​​of all adjacent point pairs are recorded in pairs, forming a spatial relationship description set between adjacent instances. This set reflects the local spacing and arrangement orientation of the marker code array.

[0067] Step S403: Instances whose distance value is less than a first preset threshold and whose difference in orientation angle value is less than a second preset threshold are marked as candidate instances of the same type, and density clustering algorithm is used to cluster the candidate instances of the same type to generate an initial cluster.

[0068] It should be understood that the first preset threshold is used to limit the maximum allowable distance between adjacent instances. Instances exceeding this threshold should not be classified into the same cluster even if their directions are completely consistent, because excessively large spatial intervals usually mean that they belong to different tag code groups. The second preset threshold is used to limit the range of differences in the direction angles of the lines connecting adjacent instances. Pairs of adjacent points with a difference in direction angles less than this threshold are determined to have a consistent spatial arrangement direction.

[0069] The system iterates through all adjacent point pairs. If a point pair simultaneously satisfies a distance less than a first preset threshold (e.g., 10 pixels) and an angular difference less than a second preset threshold (e.g., 15 degrees), then these two instances are marked as candidate instances of the same type. This marking process does not directly form clusters, but instead establishes a candidate class relationship table for each instance, which records other instances that satisfy the above dual conditions.

[0070] Furthermore, based on the labeling results of candidate similar instances, a density-based clustering algorithm (such as DBSCAN) is used to cluster all instances, generating initial clusters. The DBSCAN algorithm automatically identifies density-connected regions using two parameters: neighborhood radius eps and minimum sample size min_samples. eps can be set to 1.2 times a first preset threshold, and min_samples can be set to 2 or 3, ensuring that each cluster consists of at least two instances that mutually satisfy the candidate similarity condition. During clustering, the system starts from an unvisited instance and recursively adds all its density-reachable candidate similar instances to the same cluster, ultimately forming multiple initial clusters. Instances within the initial clusters meet the threshold requirements in terms of spatial distance and orientation consistency, but may still contain a few outliers due to noise or detection errors. These outliers will be further removed in the next step.

[0071] Step S404: If the spatial position offset of any instance within the initial cluster exceeds a preset offset threshold, the instance is removed from the current cluster, and the cluster boundary is adjusted to obtain a set of labeled code clusters.

[0072] It should be understood that although the initial clusters have passed the consistency screening of distance and orientation angle, there may still be individual instances whose spatial positions are abnormally offset relative to the overall cluster distribution. For example, image noise or false corner detection may cause the center point of a marker code to deviate from its intended position on the array grid. A preset offset threshold is used to quantify the degree of this tolerable deviation, defined as a multiple of the standard deviation of the geometric centers of all instances within the cluster (e.g., 2 times the standard deviation), or an absolute distance threshold (e.g., 5 pixels). The system calculates the spatial position statistics of the instances within each initial cluster, including the mean and standard deviation of the center point coordinates, and then checks the coordinate offset of each instance (i.e., the Euclidean distance from the instance center to the cluster mean). If the offset exceeds the preset offset threshold, the instance is determined to be an outlier and removed from the current cluster.

[0073] After removing outlier instances, the system recalculates the spatial distribution range of the clusters and adjusts the cluster boundaries. The cluster boundary refers to the spatial extent of the instance set covered by the cluster, defined by the minimum bounding rectangle or convex hull of the remaining instances. The adjusted cluster boundaries more closely match the distribution of valid instances, eliminating redundant regions expanded due to outliers. After traversing and correcting all initial clusters, the system outputs the final set of labeled clusters. Each cluster in this set has a compact geometric distribution in space, with consistent spacing and orientation between instances, and no significant positional anomalies. This provides a highly reliable grouping basis for subsequent missing corner completion and boundary merging.

[0074] Based on the first embodiment of this application, in the third embodiment of this application, the content that is the same as or similar to that in the first embodiment described above can be referred to the above description, and will not be repeated hereafter. Based on this, please refer to... Figure 3 Step S50 may include steps S501 to S506: Step S501: Determine the missing corner identifier by detecting the number of corner points of each marker in the marker code cluster set.

[0075] In its implementation, the system iterates through each marker instance in the marker cluster set, counts the number of corners successfully detected for that instance, and compares this count with a preset standard number of corners (e.g., a quadrilateral marker should have four corners). If the actual number of detected corners is less than the standard number, the system marks the missing corner positions. For example, for a quadrilateral marker that only detects three corners, the missing fourth corner is recorded as a missing corner marker. This marker can be represented using a corner index (e.g., "bottom right corner") or a relative position code (e.g., "fourth vertex relative to the known three points"). The missing corner marker does not contain coordinate information; it only indicates which corner position has not yet been detected.

[0076] Step S502: Extract the coordinates of known corner points from the cluster to which the missing corner point belongs as a reference point set.

[0077] Because the marker codes within the same cluster have geometrical consistency in their spatial arrangement (e.g., equidistant distribution or coplanar deformation), the system is no longer limited to the single marker code containing the missing corner point. Instead, it uses all other marker codes within the cluster as reference sources. Specifically, the system extracts the coordinates of all successfully detected corner points from the cluster. These corner points may originate from other corner points within the same marker code or from corresponding corner points of other complete marker codes within the cluster. For example, if the missing corner point is located at the lower right corner of the third marker code within the cluster, the system can extract the three existing corner points of that marker code itself, and also extract the four corner points of adjacent complete marker codes within the cluster as a supplementary reference point set. The quality of the reference point set directly affects the accuracy of subsequent affine transformations; therefore, the system typically requires the reference point set to contain at least three non-collinear points.

[0078] Step S503: Based on the reference point set, solve the affine transformation matrix using the least squares method to obtain the initial affine transformation matrix.

[0079] It should be noted that the affine transformation matrix is ​​a homogeneous matrix of 2×3 or 3×3, which can be uniquely determined by at least three non-collinear point pairs. When the reference point set contains more than three point pairs, the system uses the least squares method to solve the overdetermined equations to minimize the sum of squared errors between the transformed coordinates of all reference points and the observed coordinates. Specifically, the standard coordinates of each known corner point in the reference point set (i.e., the theoretical coordinates of the marker code in an ideal, undistorted state, such as the four vertices of a unit square) are paired with their actual image coordinates to establish a linear equation system. The initial affine transformation matrix is ​​obtained through least squares fitting. This matrix describes the mapping relationship from the ideal marker code space to the actual image space, and can map the standard position to the actual observation position.

[0080] Step S504: Determine the transformation error of the initial affine transformation matrix through geometric constraints.

[0081] Geometric constraints are used to evaluate the reliability of the initial affine transformation matrix, verifying the accuracy of the transformation using reference points not involved in the solution or through other geometric relationships. The system uses a subset of points in the reference point set as verification points, applies the initial affine transformation matrix to them for coordinate mapping, and calculates the Euclidean distance between the mapped coordinates and the actual observed coordinates of each point. The average distance between all verification points is the transformation error. Furthermore, geometric constraints may include checks on the properties of the transformation matrix, such as ensuring the determinant value is not too small or too large (to avoid irreversible transformations) and that the shear component does not exceed a reasonable range (to avoid excessive deformation). For example, the system requires the determinant value of the transformation matrix to be between 0.5 and 2.0, and the rotation angle to not exceed 45 degrees. These constraints collectively ensure that the transformation results conform to the rationality of physical imaging.

[0082] Step S505: If the transformation error is within a preset tolerance range, then the initial affine transformation matrix is ​​determined to be the target affine transformation matrix.

[0083] The preset tolerance range is the upper limit of the quantization of the transformation error, for example, setting the maximum allowable error to 5 pixels or 2% relative to the mark code size.

[0084] The system compares the calculated transformation error with the tolerance range. If the error is less than or equal to the upper limit, the current initial affine transformation matrix is ​​accepted as the valid target affine transformation matrix; otherwise, the matrix is ​​deemed unreliable, and a new reference point set or adjustment of the solution strategy is required. In practice, if a solution fails once, the system removes the pair of points with the largest error in the reference point set and re-executes least-squares fitting, or iteratively searches for the optimal transformation matrix using the RANSAC framework until the error meets the tolerance requirements. Only the transformation matrix that passes this verification can be used for subsequent missing corner coordinate inference.

[0085] Step S506: Substitute the standard coordinates corresponding to the missing corner markers into the target affine transformation matrix to determine the actual coordinates corresponding to the missing corner markers, and use Bézier curves to smooth and correct the boundaries to obtain a complete set of marker code boundaries.

[0086] Each missing corner identifier corresponds to a standard coordinate in the ideal tag code model (e.g., the theoretical position of the missing vertex among the four vertices of a quadrilateral). The system substitutes the standard coordinates into the validated target affine transformation matrix, performs a homogeneous coordinate transformation, and calculates the actual coordinates of the corner in the image. This process transfers the spatial mapping relationship between known corners within the cluster to the missing corner, achieving corner completion based on intra-cluster consistency.

[0087] However, the coordinates obtained by direct transformation may result in insufficiently smooth boundaries due to noise or minor deformations. Therefore, a Bézier curve is further used to smooth and correct the completed boundaries: using known corner points and completed corner points as control points, a smooth curve passing through all corner points is generated, and the optimized boundary point sequence is obtained by resampling. Finally, the boundaries of all marker codes are completed into closed and smooth polygons, forming a complete set of marker code boundaries, each boundary containing a standard number of corner points and possessing reasonable geometric continuity.

[0088] In one implementation, step S10 includes steps S101 to S105: Step S101: Perform multi-scale edge detection on the input image to generate the original edge distribution map.

[0089] In practical implementation, multi-scale edge detection refers to extracting edges separately at different image resolutions or smoothing scales, and then fusing the results to avoid detail loss or noise amplification at a single scale. Gaussian pyramid decomposition can be used to downsample the original image into multiple scale layers (e.g., scale factors of 0.5 and 0.25). Each layer uses the Canny algorithm: first, it smooths and denoises with a Gaussian kernel (standard deviation σ=1.5), calculates the gradient magnitude and direction, then filters strong and weak edges using dual thresholds (low threshold 30, high threshold 100), and finally connects the edges through hysteresis boundary tracking. After the edge maps at each scale are unified to the original image size through interpolation, the maximum value of each pixel is taken and superimposed to form an original edge distribution map covering different thicknesses. This map contains discontinuous edge segments that require further enhancement.

[0090] Step S102: Perform local gradient enhancement on the original edge distribution map to extract continuous edge segments.

[0091] Local gradient enhancement is applied to the original edge distribution map to improve the continuity of weak edges. This enhancement is adaptively adjusted based on the gradient magnitude statistics within each pixel's neighborhood. For example, a sliding window of 7×7 pixels is set, and the mean and standard deviation of the gradient magnitudes within the window are calculated. If the gradient magnitude of the center pixel is lower than the mean minus 0.5 times the standard deviation (i.e., a weak edge), its magnitude is multiplied by an enhancement factor of 1.3. Simultaneously, non-maximum suppression is used to maintain the edge width at a single pixel. This involves comparing adjacent pixels along the gradient direction and retaining only the point with the largest local magnitude. Edge segments that were originally discontinuous due to insufficient contrast are connected into longer, continuous line segments, thus extracting a set of continuous edge segments. However, breaks may still exist, typically manifested as a noticeable gap between two edge endpoints, with gap lengths potentially reaching 5 to 15 pixels.

[0092] Step S103: Detect the break in the continuous edge segment and extract the geometric shape information of the local area of ​​the break.

[0093] After detecting breaks in continuous edge segments, it is necessary to extract the geometric shape information of the local region at the break. This geometric shape information includes the tangent directions at the two ends of the break, the distance between the lines connecting the two ends, and the image grayscale distribution within the break area. Specifically, within a 15×15 pixel region around the break endpoints, the edge direction histogram is calculated, and the principal direction is taken as the tangent. Simultaneously, the Euclidean distance between the two ends and the angle between the connecting line and their respective tangents are calculated. Preset geometric adaptation conditions include, for example: the distance between the two ends does not exceed 20 pixels, the angle between the tangents at the two ends is less than 30 degrees, and the average deviation between the connecting line and the tangents at the two ends is less than 20 degrees.

[0094] Step S104: If the geometric shape information meets the preset geometric adaptation conditions, then the Bézier curve is used to fit and connect the fracture to obtain the repaired edge segment.

[0095] It should be understood that if the preset geometric fit conditions are met, the fracture is considered suitable for fitting and connecting using a Bézier curve. A Bézier curve is a smooth parametric curve defined by control points. A cubic Bézier curve requires four control points: two fracture endpoints are taken as the start and end control points, and then two intermediate control points are obtained by extending a distance d (e.g., d equals one-third of the distance between the two endpoints) along the tangential direction of each endpoint. The de Castelli algorithm is used to recursively calculate the point sequence on the curve, generating a curve that smoothly connects the two fracture edge segments.

[0096] Step S105: Perform morphological closing operation on the repaired edge segments to obtain a complete set of edge features.

[0097] After obtaining the repaired edge segments, morphological closing operations are performed to obtain a complete set of edge features. Morphological closing involves first dilating the binary edge image, followed by erosion. Dilation expands the edge lines outwards, filling small gaps no wider than the structuring element; erosion restores the approximate width of the edge lines while maintaining connectivity. A 3×3 pixel cross-shaped structuring element (i.e., the center pixel and its four neighbors) is typically used. For example, if an edge has a 2-pixel gap at the break point, after dilution, the gap is filled and closed by the surrounding edge pixels; after erosion, the edge line is restored to a single-pixel width. Closing operations can also remove isolated noise points (such as debris smaller than 5 pixels). Finally, a complete set of edge features is obtained, where each edge is a continuously closed or fully extended curve.

[0098] In one implementation, step S20 includes steps S201 to S204: Step S201: Based on each edge point in the edge feature set, determine the local gradient change matrix, and determine the corner response value according to the local gradient change matrix. Mark the points whose corner response values ​​are greater than a preset corner threshold as potential corner points, and generate a potential corner point set.

[0099] It should be understood that the system takes the complete set of edge features as input, traverses each edge point in the set, establishes a local window (e.g., 5×5 pixels or 7×7 pixels) around the point, calculates the gradient components of the image grayscale in the x and y directions within the window, and then constructs a 2×2 local gradient change matrix (also known as the second-order moment matrix or structure tensor). The eigenvalues ​​of this matrix reflect the intensity of gradient changes along different directions.

[0100] The corner response value is a scalar calculated from the trace and determinant of the matrix using a specific formula (such as the response function of the Harris operator). Its physical meaning is to quantify the intensity of gradient change at that point in two orthogonal directions. The system sets a preset corner threshold, such as 0.05 or 0.1, and marks edge points with corner response values ​​greater than this threshold as potential corner points. All marked points constitute a potential corner point set. For example, in a feature set containing 500 edge points, there may be 30 points with response values ​​exceeding 0.05. These points are initially considered to have corner features, even though some may be inaccurately located due to edge breaks or noise.

[0101] Step S202: Determine whether there is an interruption in the edge segment corresponding to the potential corner point set. If there is an interruption, extract the adjacent edge segments at both ends of the interruption, and use a linear interpolation method to generate connection points to repair the interruption, thereby obtaining the repaired corner point sequence.

[0102] The points in the potential corner point set were originally located on their respective edge segments, but because there may still be tiny breaks that are not fully connected after edge extraction and repair, these breaks will cause the spatial distribution of corner points to be discontinuous. For example, on an edge that should be continuous, there may be two isolated corner points with several intermediate pixels missing between them.

[0103] The system maps potential corner points back to their respective edge segments based on the edge connectivity relationships in the edge feature set, and checks the actual pixel distance between adjacent potential corner points on the same edge segment. If this distance exceeds a preset interruption threshold (e.g., 5 pixels), an interruption is determined to exist at that location. Further, the system extracts adjacent edge segments at both ends of the interruption, obtains the coordinates and gradient directions of the two endpoints, and generates connection points using a linear interpolation method: assuming the coordinates of one end of the interruption are (x1, y1) and the other end are (x2, y2), the system interpolates along a straight line between the two points with a step size of 1 pixel to generate the coordinates of the intermediate point, and inserts these connection points sequentially at the interruption location. After interpolation, the original edge segment is repaired into a continuous path, and all potential corner points on it, together with the newly added connection points, constitute the repaired corner point sequence. The spatial order of the corner points in this sequence is consistent with the edge orientation.

[0104] Step S203: Based on a preset point set matching accuracy threshold, perform a reliability assessment on each corner point in the repaired corner point sequence to remove corner points with matching errors greater than the accuracy threshold, thereby obtaining a filtered corner point set.

[0105] The repaired corner point sequence may contain false corner points introduced by linear interpolation or pre-existing false corner points. Although these points are continuous on the edge, their positions may deviate from the true corner points beyond acceptable limits. The system uses a point set matching accuracy evaluation method to score the reliability of each corner point: it checks the consistency of the geometric features (such as tangent direction and curvature) of the current corner point with its adjacent edge segments, or uses the known geometric constraints of the marker code (such as the side length ratio that should be satisfied between adjacent corner points) to estimate the expected position of each corner point, and then calculates the Euclidean distance between the actual position and the expected position as the matching error.

[0106] A preset point set matching accuracy threshold is set to, for example, 2.5 pixels. If the matching error of a corner point exceeds this threshold, it is determined to be an unreliable point and is removed. For example, if the repaired corner point sequence contains 12 corner points, after reliability evaluation, it is found that the matching errors of 3 of these corner points are 3.2, 4.1, and 2.8 pixels, respectively, all exceeding the 2.5-pixel threshold. Therefore, these 3 points are removed, and the remaining 9 corner points constitute the filtered corner point set. This process effectively removes points with large positioning deviations introduced by edge repair.

[0107] Step S204: Cluster the filtered corner point set, and merge corner points with a distance less than a preset distance threshold into a single corner point to obtain a preliminary corner point candidate set.

[0108] Even after reliability screening, the set of corner points may still have multiple corner points that are too densely packed together in space. This is usually due to minor jitter in edge detection or local extrema in the corner point response area. For example, three potential corner points with high response values ​​may be detected simultaneously within a 2×2 pixel range around a real corner point.

[0109] The system performs clustering on the filtered corner point set, using distance-based hierarchical clustering or the DBSCAN algorithm. A preset distance threshold, such as 5 pixels, is used to group any two corner points with a spatial distance less than this threshold into the same cluster. For multiple corner points within each cluster, the system calculates the arithmetic mean of their coordinates as the coordinates of the merged single corner point, thus replacing all dense points in that cluster with a representative point. For example, if the filtered corner point set contains 10 corner points, after clustering, 8 clusters are formed. Two clusters each contain 2 dense points (spaced 3 pixels and 4 pixels apart), and the remaining 6 clusters each contain 1 isolated point, resulting in 8 corner points after merging. This set of 8 corner points constitutes the initial candidate corner point set. The corner points in this set avoid missing points due to edge interruptions, eliminate unreliable false positives, and remove localized redundancy.

[0110] In one implementation, step S30 includes steps S301 to S304: Step S301: Select multiple pairs of corresponding points from the preliminary corner point candidate set to construct a perspective transformation matrix as a candidate transformation matrix.

[0111] In its implementation, the system treats the initial candidate set of corner points as a set of two-dimensional image coordinates to be estimated. Each point corresponds to the observation value of a marker at a standard position (e.g., the four vertices of a standard rectangular template) in an ideal, undeformed state. Therefore, it is necessary to establish a perspective mapping relationship from the ideal standard coordinates to the actual image coordinates. The perspective transformation matrix is ​​a 3×3 homogeneous matrix with 8 degrees of freedom, requiring at least four pairs of non-collinear corresponding points to uniquely determine the coordinates.

[0112] In this step, the system randomly selects four pairs of corresponding points (i.e., four actual image coordinates and their corresponding four standard template coordinates) from the initial candidate corner point set, and uses the direct linear transformation method to solve for the candidate transformation matrix. For example, the standard template is a square with a side length of 100 pixels, and its four vertex coordinates are (0,0), (100,0), (100,100), and (0,100), respectively. The four candidate corner point coordinates detected in the actual image are (105,150), (205,148), (203,252), and (102,250). The system uses these four pairs of points to calculate the candidate perspective transformation matrix H. Since the four pairs of points randomly selected each time may contain outliers, the matrix obtained in a single calculation may not be accurate and needs to be verified and filtered in subsequent steps.

[0113] Step S302: Project the remaining corner points in the preliminary corner point candidate set through the candidate transformation matrix, determine the residual between the projected point and the corresponding actual point, mark the corner points whose residual is less than the preset interior point threshold as interior points, and count the number of interior points.

[0114] After obtaining the candidate transformation matrix, the system applies this matrix to the remaining corner points in the initial candidate corner point set that were not used to construct the matrix. This involves projecting the standard coordinates of each corner point onto the image space using matrix H to obtain the projected coordinates. The Euclidean distance between the projected point and the actual observed coordinates of that corner point is then calculated; this distance is the residual.

[0115] A preset interior point threshold is used to determine whether a corner point is consistent with the transformation matrix. For example, if the threshold is set to 5 pixels, the corner point is marked as an interior point if the residual is less than 5 pixels; otherwise, it is marked as an exterior point. After traversing all remaining corner points, the system counts the total number of interior points corresponding to the candidate transformation matrix. For example, in a certain iteration, the initial candidate corner point set has 15 corner points, of which 4 are used to construct the matrix. Among the remaining 11 corner points, 8 have residuals less than 5 pixels, so the number of interior points in this iteration is 8 (plus the 4 points used for construction, which are usually also considered interior points, for a total of 12 interior points). The number of interior points reflects the degree of fit between the candidate transformation matrix and the overall corner point distribution; a higher number indicates a more reliable matrix.

[0116] Step S303: Iteratively execute the matrix construction step and the interior point screening and statistics step. After reaching the preset number of iterations, select the candidate transformation matrix with the most interior points as the target transformation matrix, and take the corresponding set of interior points as the effective corner point set.

[0117] It should be understood that since the four pairs of corresponding points randomly selected in a single iteration may contain outliers, the calculated candidate transformation matrix may be of poor quality. Therefore, the system needs to repeatedly execute the above two steps to find the optimal matrix. The preset number of iterations is usually set based on the size of the initial candidate corner matrix and the expected confidence level, for example, 1000 or 2000 iterations. Each iteration generates a candidate transformation matrix and counts its interior points. After the iteration, the system compares the number of interior points of all candidate matrices and selects the one with the most interior points as the target transformation matrix. At the same time, all interior points corresponding to this matrix (i.e., corner points whose residuals are less than the preset interior point threshold) are extracted to form the effective corner point set. For example, after 1000 iterations, the best candidate matrix has 12 interior points, while the second best has only 9. In this case, the system accepts this matrix as the target transformation matrix and retains these 12 interior points as the effective corner point set.

[0118] Step S304: Based on the set of valid corner points, perform boundary consistency verification in conjunction with the preset marker code template to eliminate abnormal corner points whose boundary overlap with the preset marker code template is lower than the preset overlap threshold, and obtain a preliminary marker code instance set.

[0119] Although the corner points in the effective corner point set are filtered through the interior points of perspective transformation, there may still be individual corner points that, although they meet the residual threshold, cause the quadrilateral boundary formed by these corner points to deviate significantly from the geometry of the preset mark code template.

[0120] Therefore, the system further introduces boundary consistency verification: a closed quadrilateral boundary is constructed based on the spatial order of corner points in the effective corner point set (e.g., clockwise or counterclockwise arrangement), and the boundary coincidence degree between this boundary and the preset marker code template (e.g., a standard square) is calculated. The boundary coincidence degree can be quantified by mapping the template boundary to the image space through the target transformation matrix and then calculating the overlap area ratio or average edge distance between the mapped boundary and the actual corner point boundary.

[0121] A preset overlap threshold is set to, for example, 85%. If the overlap is lower than this threshold, it is determined that there are abnormal corner points in the valid corner point set. The system needs to remove the abnormal corner points that cause the decrease in overlap (for example, by removing the corner points one by one and recalculating the overlap). After removing the abnormal corner points, the boundary enclosed by the remaining corner point set has a high consistency with the template boundary. Each quadrilateral formed by these corner points corresponds to a tag code instance, and all instances together form the initial tag code instance set. Each instance in this set has passed perspective transformation consistency verification and template boundary matching verification, and has a reliable geometric structure.

[0122] This application also provides a robust detection device based on visually marked codes, please refer to... Figure 4 The robust detection device based on visual marker codes includes: The edge extraction and break repair module 10 is used to extract edges from the input image and perform geometric adaptation repair on the broken edges to obtain a complete set of edge features. The local gradient corner detection module 20 is used to detect corners based on local gradient changes according to the edge feature set, and obtain a preliminary corner candidate set. The perspective transformation estimation and anomaly removal module 30 is used to perform perspective transformation estimation on the preliminary corner point candidate set and remove anomalies to generate a preliminary marker code instance set. The geometric consistency clustering module 40 is used to extract the geometric center of each instance from the preliminary tag code instance set, determine the distance value and orientation angle value between adjacent geometric centers, and group instances whose distance value and orientation angle value satisfy a preset consistency condition into the same cluster to obtain a tag code cluster set; The missing corner point completion and boundary repair module 50 is used to detect the missing corner points of each tag code in the tag code cluster set, determine the coordinates of the missing corner points through affine transformation and complete the boundary to obtain a complete tag code boundary set.

[0123] The robust detection device based on visual marker codes provided in this application, employing the robust detection method based on visual marker codes in the above embodiments, can solve the technical problem of corner point loss caused by visual marker codes in complex environments. Compared with the prior art, the beneficial effects of the robust detection device based on visual marker codes provided in this application are the same as those of the guide rail error compensation method provided in the above embodiments, and other technical features in the robust detection device based on visual marker codes are the same as those disclosed in the methods of the above embodiments, and will not be repeated here.

[0124] This application also provides a robust detection system based on visual marker codes, in which the system fully executes the robust detection method based on visual marker codes during actual operation.

[0125] It should be noted that, in this document, the terms "comprising," "including," and any other variations are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Specific examples have been used in this document to illustrate the principles and implementation methods of the present invention. These examples are only intended to aid in understanding the method and core ideas of the present invention. The above descriptions are merely preferred embodiments of the present invention. It should be pointed out that, due to the limitations of written expression and the objective existence of infinite specific structures, those skilled in the art can make various improvements, modifications, or variations without departing from the principles of the present invention, and can also combine the above technical features in an appropriate manner. These improvements, modifications, variations, or combinations, or the direct application of the concept and technical solution of the present invention to other situations without modification, should all be considered within the scope of protection of the present invention.

Claims

1. A robust detection method based on visually labeled codes, characterized in that, include: Edges are extracted from the input image, and geometric adaptation and repair are performed on the broken edges to obtain a complete set of edge features; Based on the edge feature set, corner points are detected based on local gradient changes to obtain a preliminary candidate set of corner points; Perspective transformation estimation is performed on the preliminary corner candidate set, and outliers are removed to generate a preliminary set of marked code instances; By extracting the geometric center of each instance from the preliminary set of marked code instances, determining the distance and orientation angle between adjacent geometric centers, and grouping instances whose distance and orientation angle satisfy a preset consistency condition into the same cluster, a set of marked code clusters is obtained. The missing corner points of each marker code within the marker code cluster set are detected, and the coordinates of the missing corner points are determined by affine transformation and the boundaries are filled in to obtain a complete marker code boundary set.

2. The robust detection method based on visual marker codes according to claim 1, characterized in that, After detecting the missing corner points of each tag code within the tag code cluster set, determining the coordinates of the missing corner points through affine transformation, and filling in the boundaries to obtain a complete tag code boundary set, the process further includes: The convex hull enclosing range of each tag code boundary within the cluster is determined based on the complete tag code boundary set, and the overlapping areas between different convex hull boundaries are identified. The overlapping areas are then geometrically adapted and merged to obtain the final target region boundary division result. Based on the final target region boundary division result, the marker code region is separated from the input image, and the pixel distribution features within the marker code region are extracted; When the pixel distribution features conform to the preset encoding mode, the encoding information is extracted by the decoding algorithm to obtain the mark code recognition result; A noise suppression filter is applied to the image corresponding to the mark code recognition result to obtain an enhanced image after removing interfering pixels; When the sharpness of the enhanced image is higher than a preset sharpness threshold, the contrast of the enhanced image is adjusted in conjunction with the illumination compensation mechanism to generate an optimized marker code instance; Multi-dimensional geometric attributes are extracted from the optimized tag code instances, and clustering algorithms are used to group the instances of the multi-dimensional geometric attributes to obtain the grouped instances; When the grouped instances meet the consistency threshold, the geometric attributes within the group are integrated through a fusion algorithm to obtain the final tag code detection output.

3. The robust detection method based on visual marker codes according to claim 2, characterized in that, The process of determining the convex hull bounding range of each tag code boundary within a cluster based on the complete tag code boundary set, identifying overlapping regions between different convex hull boundaries, and performing geometric adaptation merging processing on the overlapping regions to obtain the final target region boundary division result includes: For the set of boundary points belonging to the same cluster in the complete set of marker code boundaries, the convex hull algorithm is used to determine the convex hull enclosing range of each marker code boundary, and the convex hull boundaries are obtained. Determine the intersection region between the boundaries of each convex hull; If the area of ​​the intersection region is greater than zero, then the intersection region is determined as an overlapping region, and the geometric center point of the overlapping region is extracted. Using the geometric center point as a reference, and combining the curvature of the two boundary lines of the overlapping area, a curve fitting method is used to smoothly merge the two boundary lines into a new boundary line, generating the merged boundary. Verify whether the merged boundary meets the preset requirements for the number and positional consistency of corner points, and obtain the verification results; Corner points are extracted from the two side boundary lines to generate an original set of boundary corner points, and the missing corner points of the merged boundary relative to the preset number of corner points are detected to generate a set of missing corner point identifiers. Based on the original set of boundary corner points, an affine transformation is used to determine the coordinate positions corresponding to the missing corner point identifier set. The corner points are then inserted into the merged boundary to obtain the target region boundary division result.

4. The robust detection method based on visual marker codes according to claim 1, characterized in that, The step involves extracting the geometric center of each instance from the initial set of marked code instances, determining the distance and orientation angle between adjacent geometric centers, and grouping instances whose distance and orientation angle values ​​satisfy a preset consistency condition into the same cluster to obtain a set of marked code clusters, including: Extract the geometric center of each instance from the initial set of marked code instances to obtain the set of geometric center points; By determining the Euclidean distance and the direction angle of the line connecting each pair of adjacent center points in the geometric center point set, the distance and direction angle values ​​of adjacent instances are obtained. Instances whose distance value is less than a first preset threshold and whose difference in orientation angle value is less than a second preset threshold are marked as candidate instances of the same type, and density clustering algorithm is used to cluster the candidate instances of the same type to generate an initial cluster. If the spatial location offset of any instance within an initial cluster exceeds a preset offset threshold, the instance is removed from the current cluster, and the cluster boundary is adjusted to obtain a set of labeled code clusters.

5. The robust detection method based on visual marker codes according to claim 1, characterized in that, The process of detecting missing corner points of each tag code within the tag code cluster set, determining the coordinates of the missing corner points through affine transformation, and filling in the boundaries to obtain a complete tag code boundary set includes: The missing corner identifier is determined by detecting the number of corner points of each marker in the marker code cluster set; Extract the coordinates of known corner points from the cluster to which the missing corner point's tag code belongs, and use them as a reference point set; Based on the reference point set, the affine transformation matrix is ​​solved using the least squares method to obtain the initial affine transformation matrix; The transformation error of the initial affine transformation matrix is ​​determined by geometric constraints; If the transformation error is within a preset tolerance range, then the initial affine transformation matrix is ​​determined to be the target affine transformation matrix; Substitute the standard coordinates corresponding to the missing corner markers into the target affine transformation matrix to determine the actual coordinates corresponding to the missing corner markers, and use Bézier curves to smooth and correct the boundaries to obtain a complete set of marker code boundaries.

6. The robust detection method based on visual marker codes according to claim 1, characterized in that, The process of extracting edges from the input image and performing geometric adaptation and repair on broken edges to obtain a complete set of edge features includes: Perform multi-scale edge detection on the input image to generate the original edge distribution map; Local gradient enhancement is performed on the original edge distribution map to extract continuous edge segments; Detect the break points in the continuous edge segments and extract the geometric shape information of the local area at the break point; If the geometric shape information meets the preset geometric adaptation conditions, then the fracture is fitted and connected using a Bézier curve to obtain the repaired edge segment. A morphological closing operation is performed on the repaired edge segments to obtain a complete set of edge features.

7. The robust detection method based on visual marker codes according to claim 1, characterized in that, The step of detecting corner points based on local gradient changes according to the edge feature set to obtain a preliminary candidate set of corner points includes: Based on each edge point in the edge feature set, a local gradient change matrix is ​​determined, and a corner response value is determined according to the local gradient change matrix. Points with corner response values ​​greater than a preset corner threshold are marked as potential corner points, and a potential corner point set is generated. Determine whether there is an interruption in the edge segment corresponding to the potential corner point set. If there is an interruption, extract the adjacent edge segments at both ends of the interruption, and use a linear interpolation method to generate connection points to repair the interruption, thereby obtaining the repaired corner point sequence. Based on a preset point set matching accuracy threshold, a reliability assessment is performed on each corner point in the repaired corner point sequence to remove corner points with matching errors greater than the accuracy threshold, thereby obtaining a filtered corner point set. The filtered set of corner points is clustered to merge corner points with distances less than a preset distance threshold into a single corner point, thus obtaining a preliminary candidate set of corner points.

8. The robust detection method based on visual marker codes according to claim 1, characterized in that, The step of performing perspective transformation estimation on the preliminary corner candidate set and removing outliers to generate a preliminary tag code instance set includes: Matrix construction steps: By selecting multiple pairs of corresponding points from the preliminary corner point candidate set, a perspective transformation matrix is ​​constructed as a candidate transformation matrix; Inner point screening and statistics steps: Project the remaining corner points in the preliminary corner point candidate set through the candidate transformation matrix, determine the residual between the projected point and the corresponding actual point, mark the corner points whose residual is less than the preset inner point threshold as inner points, and count the number of inner points; The matrix construction step and the interior point screening and statistics step are executed iteratively. After reaching the preset number of iterations, the candidate transformation matrix with the most interior points is selected as the target transformation matrix, and the corresponding set of interior points is taken as the effective corner point set. Based on the set of valid corner points, a boundary consistency verification is performed using a preset marker code template to eliminate abnormal corner points whose boundary overlap with the preset marker code template is lower than a preset overlap threshold, thereby obtaining a preliminary set of marker code instances.

9. A robust detection device based on visual marker codes, characterized in that, include: The edge extraction and break repair module is used to extract edges from the input image and perform geometric adaptation repair on broken edges to obtain a complete set of edge features. The local gradient corner detection module is used to detect corners based on local gradient changes according to the edge feature set, and obtain a preliminary corner candidate set. The perspective transformation estimation and anomaly removal module is used to perform perspective transformation estimation on the preliminary corner point candidate set and remove anomalies to generate a preliminary set of marked code instances. The geometric consistency clustering module is used to extract the geometric center of each instance from the preliminary set of marked code instances, determine the distance value and orientation angle value between adjacent geometric centers, and group instances whose distance value and orientation angle value satisfy a preset consistency condition into the same cluster to obtain a set of marked code clusters; The missing corner point completion and boundary repair module is used to detect the missing corner points of each tag code in the tag code cluster set, determine the coordinates of the missing corner points through affine transformation and complete the boundary to obtain a complete tag code boundary set.

10. A robust detection system based on visually marked codes, characterized in that, The system includes: an edge extraction and break repair module, a local gradient corner detection module, a perspective transformation estimation and anomaly removal module, a geometric consistency clustering module, and a missing corner completion and boundary repair module. The edge extraction and break repair module is used to extract edges from the input image and perform geometric adaptation repair on broken edges to obtain a complete set of edge features. The local gradient corner detection module is used to detect corners based on local gradient changes according to the edge feature set, and obtain a preliminary corner candidate set. The perspective transformation estimation and anomaly removal module is used to perform perspective transformation estimation on the preliminary corner point candidate set and remove anomalies to generate a preliminary set of marked code instances. The geometric consistency clustering module is used to extract the geometric center of each instance from the preliminary tag code instance set, determine the distance value and orientation angle value between adjacent geometric centers, and group instances whose distance value and orientation angle value satisfy a preset consistency condition into the same cluster to obtain a tag code cluster set; The missing corner point completion and boundary repair module is used to detect the missing corner points of each tag code in the tag code cluster set, determine the coordinates of the missing corner points through affine transformation and complete the boundary to obtain a complete tag code boundary set.

Images