Near distance positioning and long distance detection conversion method and system based on camera and triangulation conversion fusion

Abstract

The application relates to a near-distance positioning and long-distance detection transformation method and system based on camera and triangle transformation fusion, and belongs to the technical field of computer vision and visual positioning. In the long-distance stage, stable detection is completed by relying on outer large-scale markers; in the detection transformation stage, a detection algorithm self-adaptive transformation is triggered based on pixel scale and a geometric model; and in the near-distance positioning stage, high-precision three-dimensional positioning is completed based on inner small-scale markers and a triangle transformation model. The application introduces a detection transformation mechanism facing distance changes, preferentially utilizes outer visual markers to complete stable detection in the long-distance stage, and then switches to inner visual markers for accurate positioning after the target gradually approaches, so that the effective detection distance range of the system is significantly expanded, and the continuity and stability of the long-distance and near-distance switching process are improved without reducing the near-distance positioning accuracy.

Description

Technical Field

[0001] This invention relates to a near-range positioning and far-range detection transformation method and system based on camera and triangulation transformation fusion, aiming to solve the problems of unreliable detection under long-range conditions and the disconnect between near-range and far-range positioning methods in the existing ArUco marker positioning method, belonging to the field of computer vision and visual positioning technology. Background Technology

[0002] Localization and ranging technology based on artificial visual tags is one of the important research directions in the field of computer vision. Among them, ArUco tags are widely used in application scenarios such as autonomous landing of UAVs, navigation of mobile robots, positioning and calibration of industrial equipment, and augmented reality due to their clear encoding rules, mature detection algorithms, and low requirements for computing resources.

[0003] Existing ArUco marker localization methods typically rely on image information acquired by monocular, binocular, or depth cameras, and combine this with known marker physical dimensions to calculate the marker's spatial position and orientation relative to the camera through perspective imaging models, PnP pose solving, or trigonometric transformations. Under close-range conditions, when the marker has sufficient pixel size in the image, these methods can achieve high localization accuracy and have the advantages of simple implementation and good real-time performance.

[0004] In practical engineering applications, especially in scenarios such as UAV landing guidance, large-scale outdoor positioning, and long-distance target guidance, the single ArUco marker localization method has significant shortcomings. First, the ArUco marker localization method based on camera and triangulation transformation fusion relies on the target marker having a sufficient pixel scale in the imaging plane for effective operation. When the target is far away, the projected size of the marker in the image decreases rapidly, and the reliability of corner point extraction and encoding / decoding decreases significantly, causing the localization method to fail to start or operate stably.

[0005] Existing technologies employ nested visual marker structures, using large outer visual markers and small inner visual markers on the same target. Under the condition that the markers can be detected and decoded, pose estimation is performed separately, and multiple pose results are weighted or fused to improve the stability and accuracy of the localization results. However, the improvement of this method mainly focuses on the pose estimation stage, still presupposing that the visual markers have sufficient pixel resolution in the image and can complete decoding and corner extraction.

[0006] However, when the target is at a distance, the pixel scale of the visual markers in the image is significantly reduced due to limitations in imaging resolution and environmental factors. This often makes it difficult to meet the conditions for complete decoding and accurate corner extraction. The aforementioned nested marking scheme still cannot reliably start the detection and localization process at long distances, and its effective working distance improvement is limited.

[0007] Secondly, existing technologies often treat long-range target detection and close-range precise positioning as two independent problems. For example, only coarse target detection is performed at long range, and then a high-precision positioning algorithm is switched at close range. Such solutions lack a unified label design and recognition transformation mechanism, which makes the system prone to problems such as recognition interruption, positioning jitter, or misjudgment during the switching between long and short ranges, making it difficult to meet the engineering requirements of continuous and stable positioning.

[0008] Furthermore, in depth camera applications, depth information is prone to distortion or loss at extremely long or short distances, limiting the applicability of positioning solutions that rely solely on depth data. Therefore, balancing long-range detection capabilities with high-precision short-range positioning within the same system has become a significant technical bottleneck restricting the further application of ArUco marker-based positioning technology.

[0009] In summary, there is an urgent need for a technical solution that can significantly expand the effective detection range of the system by maintaining the existing ArUco marker near-range positioning accuracy based on camera and triangulation transformation fusion, while through new marker design and detection transformation mechanism. Summary of the Invention

[0010] To address the shortcomings of existing technologies, this invention provides a near-range positioning and far-range detection transformation method and system based on camera and triangulation transformation fusion. It introduces a detection transformation mechanism oriented towards distance changes, prioritizing the use of outer visual markers for stable detection at long distances, and switching to inner visual markers for precise positioning as the target gradually approaches. This significantly expands the effective detection range of the system without reducing near-range positioning accuracy, and improves the continuity and stability of the near-range switching process.

[0011] The present invention adopts the following technical solution:

[0012] A near-range localization and far-range detection transformation method based on camera and triangulation transformation fusion includes the following steps:

[0013] S1: Construct multi-scale chimeric ArUco tags;

[0014] S2: Acquire image data containing the embedded ArUco markers through the camera, and detect the outer large-scale markers of the embedded ArUco markers in the long-distance detection stage, thereby completing the recognition and decoding of the outer large-scale markers;

[0015] S3: After the outer large-scale markers are detected, when the detection transformation criterion is met, the detection transformation stage is entered. In the detection transformation stage, a parallel detection strategy is adopted, and the detection algorithm for the inner small-scale markers is activated while the outer large-scale markers are continuously detected.

[0016] S4: Statistically analyze the detection confidence of the inner small-scale markers for several consecutive frames. When the detection result of the inner small-scale markers meets the stability criterion, the system completes the smooth switching of the detection strategy from the outer large-scale markers to the inner small-scale markers and enters the near-range positioning stage. Based on the relationship between the inner small-scale markers and the triangular transformation of the embedded ArUco markers, high-precision three-dimensional positioning is completed.

[0017] When the detection transformation criterion is not met, the continuous tracking of the inner small-scale markers is stopped and the detection of the outer large-scale markers is resumed.

[0018] Preferably, in step S1, the embedded ArUco marker consists of an outer large-scale marker and an inner small-scale marker. The inner small-scale marker is located in the central region of the outer large-scale marker, and the inner small-scale marker is surrounded by white unit regions to isolate the encoding structure of the inner and outer markers. The central module of the outer large-scale marker and its adjacent modules should meet the preset color distribution constraints to improve the detection stability of the nested visual markers and avoid the inner markers from interfering with the decoding process of the outer markers.

[0019] In this embodiment, two ArUco markers with different coding scales are selected as the basis for chimerism. The outer large-scale marker uses ArUco markers with a large number of coding units and a large physical size for long-distance detection, while the inner small-scale marker uses ArUco markers with a small number of coding units and a small physical size for close-range precise positioning. This invention uses a multi-scale chimerism ArUco marker as a unified visual guidance carrier. The coding size, number of units, and physical size of the outer large-scale marker and the inner small-scale marker are not fixed and can be adaptively adjusted according to the actual detection distance range, camera resolution, and application scenario.

[0020] When generating the chimeric ArUco marker, the inner small-scale marker is first generated and placed in the geometric center region of the outer large-scale marker. Subsequently, the encoding matrix of the outer large-scale marker is constrained. The inner small-scale marker is only allowed to be embedded in the outer large-scale marker if the surrounding coding units in the center region of the outer large-scale marker are all white regions, so as to avoid destroying the bounding box structure and encoding integrity of the outer ArUco.

[0021] Through the above method, a chimeric ArUco tag is constructed that still conforms to the ArUco encoding rules visually, but has multi-level detectable characteristics in spatial scale.

[0022] In the overall process of long-range detection and short-range positioning, the interlocking relationship between the outer large-scale markers and the inner small-scale markers is used not only for structural design but also for the continuous constraint of detection results. During detection transformation and short-range positioning, the system can utilize the relative positional relationship between the outer large-scale markers and the inner small-scale markers to perform consistency verification on the detection results of the inner small-scale markers, thereby improving the stability of the detection transformation process and avoiding recognition jumps or positioning jitters during distance changes.

[0023] The consistency verification process is as follows: by statistically analyzing the relationship between the number of black pixels and white pixels in the inner small-scale marker region, the perceived color of the inner small-scale marker is determined; and the perceived color is then compared with the expected color corresponding to the center encoding of the outer large-scale marker. When the two are consistent, the detection result of the inner small-scale marker is confirmed to be valid; when the two are inconsistent, the detection result of the inner small-scale marker is determined to be invalid, thereby improving the stability and reliability of the nested visual marker detection and detection transformation process.

[0024] Before performing positioning, the camera is automatically calibrated using a chessboard calibration board. This includes automatically acquiring calibration images and calculating camera intrinsic parameters after detecting stable chessboard corner points. Based on multiple frames of images, the correspondence between spatial three-dimensional points and two-dimensional pixels is established to obtain focal length parameters and principal point parameters.

[0025] Preferably, in step S2, during the long-distance detection stage, the acquired outer large-scale markers are first preprocessed by grayscale conversion, adaptive threshold segmentation, and contour extraction. Then, the existing ArUco marker detection algorithm is used to extract candidate quadrilateral contours from the image, perform perspective correction and dictionary matching, thereby completing the recognition and decoding of the outer markers. For each quadrilateral contour obtained, perspective correction is performed, mapping the region to a standard frontal plane, and the corrected image is encoded and decoded according to the preset ArUco dictionary rules. When the decoding result matches successfully in the preset dictionary and passes the verification, it is determined that an outer large-scale marker has been detected; if decoding fails or the verification fails, it is determined that the candidate region does not constitute a valid outer large-scale marker.

[0026] Because the outer marker has a larger physical size and coarser coding units, it can still maintain sufficient pixel coverage under long-distance conditions, ensuring detection stability.

[0027] In this context, "long distance" refers to the distance range where, under the current camera resolution, imaging conditions, and physical size of the visual markers, the projected pixel size of the inner small-scale markers in the image is lower than its minimum detectable pixel threshold, thus failing to meet the requirements for corner extraction and encoding / decoding. Within this distance range, even if the inner small-scale markers actually exist, their visual features in the imaging plane are insufficient for reliable identification. Therefore, this implementation only enables the detection of the outer large-scale markers at long distances.

[0028] The preprocessing steps of grayscale conversion, adaptive threshold segmentation, and contour extraction mainly serve to reduce the impact of illumination changes and noise on subsequent visual marker detection, and to enhance the contrast between the marker region and the background.

[0029] Preferably, in step S2, the distance between the target marker and the camera is estimated:

[0030] After detecting the large-scale outer marker, the pixel coordinates of the four corner points in the image coordinate system are extracted. The pixel side length of the outer large-scale marker is calculated based on the pixel distance between adjacent corner points, and the average pixel side length is taken as the equivalent pixel side length of the outer large-scale marker. :

[0031]

[0032] in, , , , These represent the coordinates of the four corner points; , , , These represent the side lengths of the four adjacent corner points, , , , ;

[0033] Equivalent pixel side length It is used to characterize the visual size of large-scale outer markers under the current imaging conditions, and serves as an important basis for subsequent distance estimation and detection transformation determination;

[0034] Let the actual physical side length of the outer large-scale marker be... The camera's equivalent focal length is Then, according to the pinhole imaging model, we can obtain:

[0035]

[0036] in, The distance between the target marker and the camera;

[0037] Preferably, the camera in step S2 is a depth camera. When the depth data is valid, the spatial three-dimensional coordinates of the target marker can be further calculated using the depth data. Obtain the center pixel of the outer large-scale marker. Corresponding depth value The camera intrinsic parameter matrix is ​​obtained through the checkerboard calibration method. :

[0038]

[0039] in, The focal length of the camera in the horizontal direction. The focal length of the camera in the vertical direction. Principal point coordinates;

[0040] The center pixel is back-projected onto the camera coordinate system to obtain the spatial coordinates of the target marker. :

[0041]

[0042]

[0043]

[0044] When depth data is invalid, distance perception is achieved solely based on the pixel scale and geometric relationships of the outer large-scale markers, i.e., the triangular transformation relationships are used to mark the three-dimensional coordinates in space.

[0045] The following conditions must be met to determine the validity of depth data:

[0046] The depth value is not zero or there is no invalid placeholder value;

[0047] The depth value did not exceed the effective measurement range of the depth camera;

[0048] There are no anomalous abrupt changes in depth values ​​between spatial neighborhoods or adjacent time frames.

[0049] Preferably, to achieve adaptive switching from long-range detection mode to short-range high-precision positioning mode, this invention does not use a fixed distance as the detection transformation condition. Instead, it constructs a calculable and dynamically adjustable detection transformation criterion based on the imaging geometric model. In step S3, the detection transformation criterion is not triggered by a single condition, but is based on the joint constraint of pixel scale and spatial distance. The detection transformation criterion is satisfied when any of the following conditions are met:

[0050] a. When the equivalent pixel side length of the outer large-scale marker Greater than or equal to the preset pixel threshold ;

[0051] b. The distance between the target marker and the camera obtained from the pinhole imaging model Less than the preset distance threshold .

[0052] Preferably, a preset pixel threshold Determined based on the smallest stable recognizable pixel scale of the inner layer small-scale markers under the current camera resolution and detection algorithm: ,in The minimum stable detection pixel side length for the inner small-scale marker under the current algorithm and imaging conditions; , The actual physical side length marked at the inner small scale. The side length of the outermost large-scale marker;

[0053] Preset distance threshold This is obtained by substituting the minimum detectable pixel size of the inner layer small-scale markers into the pinhole imaging model.

[0054] .

[0055] Distance threshold It is directly related to the camera focal length, the physical size of the marker, and the performance of the detection algorithm, and can be adaptively adjusted according to different hardware configurations and application scenarios.

[0056] The aforementioned detection transformation criteria are calculated in real time during image processing and marker detection, and dynamically updated based on the pixel scale changes of the outer ArUco markers in each frame. When any criterion condition is met, the system enters the detection transformation phase. During the detection transformation phase, the system uses the region containing the inner small-scale markers as the region of interest. While maintaining the detection of the outer large-scale markers, it enables the detection process of the inner small-scale markers, thereby achieving a smooth transition of the detection strategy from outer markers to inner markers and avoiding misjudgments or detection instability caused by fixed-distance switching.

[0057] Preferably, in step S3, after detecting the outer large-scale marker, the center position and inscribed region range of the outer large-scale marker are calculated based on the pixel positions of the four corner points in the image coordinate system. Based on the center position, a region of interest (ROI) is constructed inside the outer large-scale marker. The ROI is obtained by scaling the pixel scale of the outer large-scale marker proportionally and is used to cover the area where the inner small-scale marker may appear in the close-range stage.

[0058] During the detection transformation phase, the detection algorithm for inner-layer small-scale markers is only activated within the Region of Interest (ROI), without performing a global search across the entire image. By limiting the detection area, the influence of background interference and irrelevant textures on the detection results is effectively reduced, significantly lowering the false detection probability and improving the real-time performance and stability of inner-layer marker detection. Simultaneously, the system continues to detect outer-layer large-scale markers to provide spatial constraint information and maintain the continuity of overall detection during the detection transition phase.

[0059] Preferably, during the detection transformation phase, the system simultaneously maintains two parallel states: outer-layer large-scale marker detection and inner-layer small-scale marker detection. Outer-layer large-scale marker detection primarily provides a large-scale spatial reference for the target and serves as the basis for updating the Region of Interest (ROI), while inner-layer small-scale marker detection gradually undertakes the task of precise localization. When an inner-layer small-scale marker is successfully detected, the system evaluates the confidence level of its detection result. If an inner-layer small-scale marker is temporarily lost, the inner-layer visual marker can be re-detected by expanding the ROI range and performing multi-scale image magnification processing on the ROI.

[0060] In step S4, the confidence level can be calculated based on the following factors: whether the inner small-scale marker decoding is successful, whether the detected marker ID is consistent with the preset ID, whether the four corner points of the marker are complete and form a reasonable geometric shape, and whether the changes in the detected corner point positions in adjacent frames are continuous and smooth. In the implementation process, the stability criterion can be determined by counting the number of frames in consecutive image frames in which the inner small-scale marker is successfully detected: when the inner small-scale marker is successfully identified in images with no less than a preset number of consecutive frames, and the change in the corner point position in the pixel coordinate system is lower than the set threshold, the detection result of the inner small-scale marker is determined to meet the stability.

[0061] Once the aforementioned stability conditions are met, the system considers the inner small-scale markers to have sufficient reliability for dominant localization, completes the adaptive transformation of the detection algorithm, and the subsequent localization process is dominated by the inner small-scale markers, while the outer large-scale marker detection can maintain or gradually reduce its weight as needed. By combining parallel detection with Region of Interest (ROI) constraints, a continuous transition from long-range detection to short-range high-precision localization is achieved, avoiding the localization instability caused by abrupt changes in detection modes.

[0062] During the close-range positioning phase, the target distance is:

[0063]

[0064] in, The equivalent pixel side length of the inner small-scale marker in the image is ,

[0065] Combine the pixel coordinates of the inner small-scale marker center point in the image Calculate the target's three-dimensional coordinates in the camera coordinate system to obtain precise positioning during the close-range positioning phase:

[0066]

[0067]

[0068] in, This represents the spatial displacement of the target marker center in the camera coordinate system along the horizontal direction of the camera (i.e., corresponding to the horizontal direction of the image plane). This represents the spatial displacement of the target marker center in the camera coordinate system along the camera-vertical direction (i.e., the direction perpendicular to the image plane). This indicates the distance between the center of the target marker and the optical center of the camera along the camera's optical axis.

[0069] This invention, based on monocular camera conditions, achieves a complete process from ultra-long-range target detection and adaptive transformation of detection algorithms to close-range three-dimensional precise positioning without relying on a depth camera.

[0070] A near-range localization and far-range detection system based on camera and triangulation transformation fusion, used to implement the aforementioned near-range localization and far-range detection transformation method based on camera and triangulation transformation fusion, includes:

[0071] The image acquisition module is used to acquire images containing chimeric ArUco tags in real time;

[0072] The marker detection module is used to detect large-scale markers in the outer layer, and to complete the recognition and decoding of large-scale markers in the outer layer.

[0073] The depth calculation module is used to calculate the three-dimensional coordinates of the target marker when the depth data is valid;

[0074] The triangulation estimation module is used to estimate the 3D coordinates of the target marker based on triangulation when the depth data is invalid.

[0075] The switching control module is used to switch between the long-range detection stage and the short-range positioning stage based on the detection transformation criterion and the stability criterion of the detection results of the inner small-scale markers.

[0076] For any details not covered in this invention, please refer to the prior art.

[0077] The beneficial effects of this invention are as follows:

[0078] This invention does not rely on depth sensors and achieves the following continuous operating mechanism: In the long-range phase, stable detection is achieved using large-scale outer markers; in the detection transformation phase, adaptive transformation of the detection algorithm is triggered based on pixel scale and geometric model; in the short-range localization phase, high-precision 3D localization is achieved based on small-scale inner markers and triangulation model. Throughout the entire process, the system operates using the same camera, the same visual markers, and the same imaging model, requiring no additional sensors or external auxiliary information, thus achieving coordinated operation of stable long-range detection, adaptive transformation of detection strategies, and high-precision short-range 3D localization.

[0079] This invention introduces a detection transformation mechanism oriented towards distance changes. In the long-distance stage, it prioritizes the use of outer visual markers to complete stable detection. As the target gradually approaches, it switches to inner visual markers for precise positioning. This significantly expands the effective detection range of the system without reducing the accuracy of near-distance positioning, and improves the continuity and stability of the long-distance and near-distance switching process. It is especially suitable for application scenarios such as drones, mobile robots, and vision-guided positioning. Attached Figure Description

[0080] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments of this application and their descriptions are used to explain this application and do not constitute an undue limitation of this application.

[0081] Figure 1 This is a schematic diagram of the overall process of the present invention;

[0082] Figure 2 This is a schematic diagram of the multi-scale interlocking ArUco marker structure used in this invention. Detailed Implementation

[0083] To enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this invention will be clearly and completely described below with reference to the accompanying drawings. However, this is not the only description; all aspects not described in detail herein are based on conventional techniques in the field.

[0084] Example 1

[0085] A near-range localization and long-range detection transformation method based on camera and triangulation transformation fusion, the overall process is as follows: Figure 1 As shown, it includes the following steps:

[0086] S1: Construct multi-scale chimeric ArUco tags;

[0087] S2: Acquire image data containing the embedded ArUco markers through the camera, and detect the outer large-scale markers of the embedded ArUco markers in the long-distance detection stage, thereby completing the recognition and decoding of the outer large-scale markers;

[0088] S3: After the outer large-scale markers are detected, when the detection transformation criterion is met, the detection transformation stage is entered. In the detection transformation stage, a parallel detection strategy is adopted, and the detection algorithm for the inner small-scale markers is activated while the outer large-scale markers are continuously detected.

[0089] S4: Statistically analyze the detection confidence of the inner small-scale markers for several consecutive frames. When the detection result of the inner small-scale markers meets the stability criterion, the system completes the smooth switching of the detection strategy from the outer large-scale markers to the inner small-scale markers and enters the near-range positioning stage. Based on the relationship between the inner small-scale markers and the triangular transformation of the embedded ArUco markers, high-precision three-dimensional positioning is completed.

[0090] When the detection transformation criterion is not met, the continuous tracking of the inner small-scale markers is stopped and the detection of the outer large-scale markers is resumed.

[0091] Example 2

[0092] A near-range localization and long-range detection transformation method based on camera and triangulation transformation fusion, as shown in Example 1, differs in that, in step S1, the fused ArUco marker consists of an outer large-scale marker and an inner small-scale marker, as shown in Example 1. Figure 2 As shown, the inner small-scale marker is set in the central area of ​​the outer large-scale marker, and the inner small-scale marker is surrounded by white unit areas to isolate the encoding structure of the inner and outer markers. The central module of the outer large-scale marker and its adjacent modules should meet the preset color distribution constraints to improve the detection stability of nested visual markers and avoid the inner markers from interfering with the decoding process of the outer markers.

[0093] In this embodiment, two ArUco markers with different coding scales are selected as the basis for chimerism. The outer large-scale marker uses ArUco markers with a large number of coding units and a large physical size for long-distance detection, while the inner small-scale marker uses ArUco markers with a small number of coding units and a small physical size for close-range precise positioning. This invention uses a multi-scale chimerism ArUco marker as a unified visual guidance carrier. The coding size, number of units, and physical size of the outer large-scale marker and the inner small-scale marker are not fixed and can be adaptively adjusted according to the actual detection distance range, camera resolution, and application scenario.

[0094] When generating the chimeric ArUco marker, the inner small-scale marker is first generated and placed in the geometric center region of the outer large-scale marker. Subsequently, the encoding matrix of the outer large-scale marker is constrained. The inner small-scale marker is only allowed to be embedded in the outer large-scale marker if the surrounding coding units in the center region of the outer large-scale marker are all white regions, so as to avoid destroying the bounding box structure and encoding integrity of the outer ArUco.

[0095] Through the above method, a chimeric ArUco tag is constructed that still conforms to the ArUco encoding rules visually, but has multi-level detectable characteristics in spatial scale.

[0096] In the overall process of long-range detection and short-range positioning, the interlocking relationship between the outer large-scale markers and the inner small-scale markers is used not only for structural design but also for the continuous constraint of detection results. During detection transformation and short-range positioning, the system can utilize the relative positional relationship between the outer large-scale markers and the inner small-scale markers to perform consistency verification on the detection results of the inner small-scale markers, thereby improving the stability of the detection transformation process and avoiding recognition jumps or positioning jitters during distance changes.

[0097] The consistency verification process is as follows: by statistically analyzing the relationship between the number of black pixels and white pixels in the inner small-scale marker region, the perceived color of the inner small-scale marker is determined; and the perceived color is then compared with the expected color corresponding to the center encoding of the outer large-scale marker. When the two are consistent, the detection result of the inner small-scale marker is confirmed to be valid; when the two are inconsistent, the detection result of the inner small-scale marker is determined to be invalid, thereby improving the stability and reliability of the nested visual marker detection and detection transformation process.

[0098] Before performing positioning, the camera is automatically calibrated using a chessboard calibration board. This includes automatically acquiring calibration images and calculating camera intrinsic parameters after detecting stable chessboard corner points. Based on multiple frames of images, the correspondence between spatial three-dimensional points and two-dimensional pixels is established to obtain focal length parameters and principal point parameters.

[0099] Example 3

[0100] A near-range localization and far-range detection transformation method based on camera and triangulation transformation fusion, as shown in Example 2, differs in that, in step S2, during the far-range detection stage, the acquired outer large-scale markers are first preprocessed by grayscale conversion, adaptive threshold segmentation, and contour extraction. Then, the existing ArUco marker detection algorithm is used to extract candidate quadrilateral contours from the image, perform perspective correction and dictionary matching, thereby completing the recognition and decoding of the outer markers. For each selected quadrilateral contour, perspective correction is performed, mapping the region to a standard frontal plane, and the corrected image is encoded and decoded according to the preset ArUco dictionary rules. When the decoding result matches successfully in the preset dictionary and passes the verification, it is determined that an outer large-scale marker has been detected; if decoding fails or the verification fails, it is determined that the candidate region does not constitute a valid outer large-scale marker.

[0101] Because the outer marker has a larger physical size and coarser coding units, it can still maintain sufficient pixel coverage under long-distance conditions, ensuring detection stability.

[0102] In this context, "long distance" refers to the distance range where, under the current camera resolution, imaging conditions, and physical size of the visual markers, the projected pixel size of the inner small-scale markers in the image is lower than its minimum detectable pixel threshold, thus failing to meet the requirements for corner extraction and encoding / decoding. Within this distance range, even if the inner small-scale markers actually exist, their visual features in the imaging plane are insufficient for reliable identification. Therefore, this implementation only enables the detection of the outer large-scale markers at long distances.

[0103] The preprocessing steps of grayscale conversion, adaptive threshold segmentation, and contour extraction mainly serve to reduce the impact of illumination changes and noise on subsequent visual marker detection, and to enhance the contrast between the marker region and the background.

[0104] Estimate the distance between the target marker and the camera:

[0105] After detecting the large-scale outer marker, the pixel coordinates of the four corner points in the image coordinate system are extracted. The pixel side length of the outer large-scale marker is calculated based on the pixel distance between adjacent corner points, and the average pixel side length is taken as the equivalent pixel side length of the outer large-scale marker. :

[0106]

[0107] in, , , , These represent the coordinates of the four corner points; , , , These represent the side lengths of the four adjacent corner points, , , , ;

[0108] Equivalent pixel side length It is used to characterize the visual size of large-scale outer markers under the current imaging conditions, and serves as an important basis for subsequent distance estimation and detection transformation determination;

[0109] Let the actual physical side length of the outer large-scale marker be... The camera's equivalent focal length is Then, according to the pinhole imaging model, we can obtain:

[0110]

[0111] in, Mark the distance between the target and the camera.

[0112] Example 4

[0113] A near-range localization and far-range detection transformation method based on camera and triangulation transformation fusion is shown in Example 3. The difference is that the camera in step S2 is a depth camera. When the depth data is valid, the spatial three-dimensional coordinates of the target marker can be further calculated using the depth data. Obtain the center pixel of the outer large-scale marker. Corresponding depth value The camera intrinsic parameter matrix is ​​obtained through the checkerboard calibration method. :

[0114]

[0115] in, The focal length of the camera in the horizontal direction. The focal length of the camera in the vertical direction. Principal point coordinates;

[0116] The center pixel is back-projected onto the camera coordinate system to obtain the spatial coordinates of the target marker. :

[0117]

[0118]

[0119]

[0120] When depth data is invalid, distance perception is achieved solely based on the pixel scale and geometric relationships of the outer large-scale markers, i.e., the triangular transformation relationships are used to mark the three-dimensional coordinates in space.

[0121] The following conditions must be met to determine the validity of depth data:

[0122] The depth value is not zero or there is no invalid placeholder value;

[0123] The depth value did not exceed the effective measurement range of the depth camera;

[0124] There are no anomalous abrupt changes in depth values ​​between spatial neighborhoods or adjacent time frames.

[0125] Example 5

[0126] A near-range localization and far-range detection transformation method based on camera and triangulation transformation fusion, as shown in Example 4, differs in that, in order to achieve adaptive switching from far-range detection mode to near-range high-precision localization mode, this invention does not use a fixed distance as the detection transformation condition. Instead, it constructs a calculable and dynamically adjustable detection transformation criterion based on the imaging geometric model. In step S3, the detection transformation criterion is not triggered by a single condition, but is based on the joint constraint of pixel scale and spatial distance. The detection transformation criterion is satisfied when any of the following conditions are met:

[0127] a. When the equivalent pixel side length of the outer large-scale marker Greater than or equal to the preset pixel threshold ;

[0128] b. The distance between the target marker and the camera obtained from the pinhole imaging model Less than the preset distance threshold .

[0129] Preset pixel threshold Determined based on the smallest stable recognizable pixel scale of the inner layer small-scale markers under the current camera resolution and detection algorithm: ,in This represents the minimum stable detection pixel side length for the inner layer small-scale marker under the current algorithm and imaging conditions. , The actual physical side length marked at the inner small scale. The side length of the outermost large-scale marker;

[0130] Preset distance threshold This is obtained by substituting the minimum detectable pixel size of the inner layer small-scale markers into the pinhole imaging model.

[0131] .

[0132] Distance threshold It is directly related to the camera focal length, the physical size of the marker, and the performance of the detection algorithm, and can be adaptively adjusted according to different hardware configurations and application scenarios.

[0133] The aforementioned detection transformation criteria are calculated in real time during image processing and marker detection, and dynamically updated based on the pixel scale changes of the outer ArUco markers in each frame. When any criterion condition is met, the system enters the detection transformation phase. During the detection transformation phase, the system uses the region containing the inner small-scale markers as the region of interest. While maintaining the detection of the outer large-scale markers, it enables the detection process of the inner small-scale markers, thereby achieving a smooth transition of the detection strategy from outer markers to inner markers and avoiding misjudgments or detection instability caused by fixed-distance switching.

[0134] Example 6

[0135] A near-range localization and far-range detection transformation method based on camera and triangulation transformation fusion, as shown in Example 5, differs in that, in step S3, after detecting the outer large-scale marker, the center position and inscribed region range of the outer large-scale marker are calculated based on the pixel positions of the four corner points in the image coordinate system, and a region of interest (ROI) is constructed inside the outer large-scale marker based on the center position. The ROI is obtained by scaling the pixel scale of the outer large-scale marker proportionally, and is used to cover the area where the inner small-scale marker may appear in the near-range stage.

[0136] During the detection transformation phase, the detection algorithm for inner-layer small-scale markers is only activated within the Region of Interest (ROI), without performing a global search across the entire image. By limiting the detection area, the influence of background interference and irrelevant textures on the detection results is effectively reduced, significantly lowering the false detection probability and improving the real-time performance and stability of inner-layer marker detection. Simultaneously, the system continues to detect outer-layer large-scale markers to provide spatial constraint information and maintain the continuity of overall detection during the detection transition phase.

[0137] During the detection and transformation phase, the system simultaneously maintains two parallel states: outer-layer large-scale marker detection and inner-layer small-scale marker detection. Outer-layer large-scale marker detection primarily provides a large-scale spatial reference for the target and serves as the basis for updating the Region of Interest (ROI), while inner-layer small-scale marker detection gradually undertakes the task of precise localization. When an inner-layer small-scale marker is successfully detected, the system evaluates the confidence level of the detection result. If an inner-layer small-scale marker is temporarily lost, it can be re-detected by expanding the ROI range and performing multi-scale image magnification processing on the ROI.

[0138] Example 7

[0139] A near-range localization and far-range detection transformation method based on camera and triangulation transformation fusion, as shown in Example 6, differs in that, in step S4, the confidence level can be calculated based on the following factors: whether the inner small-scale marker decoding is successful, whether the detected marker ID is consistent with the preset ID, whether the four corner points of the marker are complete and form a reasonable geometric shape, and whether the changes in the detected corner point positions in adjacent frames are continuous and smooth. In the implementation process, the stability criterion can be determined by counting the number of frames in consecutive image frames in which the inner small-scale marker is successfully detected: when the inner small-scale marker is successfully identified in images with no less than a preset number of consecutive frames, and the change in the corner point position in the pixel coordinate system is lower than a set threshold, the detection result of the inner small-scale marker is determined to meet the stability.

[0140] Once the aforementioned stability conditions are met, the system considers the inner small-scale markers to have sufficient reliability for dominant localization, completes the adaptive transformation of the detection algorithm, and the subsequent localization process is dominated by the inner small-scale markers, while the outer large-scale marker detection can maintain or gradually reduce its weight as needed. By combining parallel detection with Region of Interest (ROI) constraints, a continuous transition from long-range detection to short-range high-precision localization is achieved, avoiding the localization instability caused by abrupt changes in detection modes.

[0141] During the close-range positioning phase, the target distance is:

[0142]

[0143] in, The equivalent pixel side length of the inner small-scale marker in the image is ,

[0144] Combine the pixel coordinates of the inner small-scale marker center point in the image Calculate the target's three-dimensional coordinates in the camera coordinate system to obtain precise positioning during the close-range positioning phase:

[0145]

[0146]

[0147] in, This represents the spatial displacement of the target marker center in the camera coordinate system along the horizontal direction of the camera (i.e., corresponding to the horizontal direction of the image plane). This represents the spatial displacement of the target marker center in the camera coordinate system along the camera-vertical direction (i.e., the direction perpendicular to the image plane). This indicates the distance between the center of the target marker and the optical center of the camera along the camera's optical axis.

[0148] This invention, based on monocular camera conditions, achieves a complete process from ultra-long-range target detection and adaptive transformation of detection algorithms to close-range three-dimensional precise positioning without relying on a depth camera.

[0149] Example 8

[0150] A near-range localization and far-range detection system based on camera and triangulation transformation fusion, used to implement any of the near-range localization and far-range detection transformation methods based on camera and triangulation transformation fusion in Examples 1-7, comprising:

[0151] The image acquisition module is used to acquire images containing chimeric ArUco tags in real time;

[0152] The marker detection module is used to detect large-scale markers in the outer layer, and to complete the recognition and decoding of large-scale markers in the outer layer.

[0153] The depth calculation module is used to calculate the three-dimensional coordinates of the target marker when the depth data is valid;

[0154] The triangulation estimation module is used to estimate the 3D coordinates of the target marker based on triangulation when the depth data is invalid.

[0155] The switching control module is used to switch between the long-range detection stage and the short-range positioning stage based on the detection transformation criterion and the stability criterion of the detection results of the inner small-scale markers.

[0156] The above description represents the preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A near-range localization and long-range detection transformation method based on camera and triangulation transformation fusion, characterized in that, Includes the following steps: S1: Construct multi-scale chimeric ArUco tags; S2: Acquire image data containing the embedded ArUco markers through the camera, and detect the outer large-scale markers of the embedded ArUco markers in the long-distance detection stage, thereby completing the recognition and decoding of the outer large-scale markers; S3: After the outer large-scale markers are detected, when the detection transformation criterion is met, the detection transformation stage is entered. In the detection transformation stage, a parallel detection strategy is adopted, and the detection algorithm for the inner small-scale markers is activated while the outer large-scale markers are continuously detected. S4: Statistically analyze the detection confidence of the inner small-scale markers for several consecutive frames. When the detection result of the inner small-scale markers meets the stability criterion, the system completes the smooth switching of the detection strategy from the outer large-scale markers to the inner small-scale markers and enters the near-range positioning stage. Based on the relationship between the inner small-scale markers and the triangular transformation of the embedded ArUco markers, high-precision three-dimensional positioning is completed. When the detection transformation criterion is not met, the continuous tracking of the inner small-scale markers is stopped and the detection of the outer large-scale markers is resumed.

2. The near-range positioning and long-range detection transformation method based on camera and triangulation transformation fusion according to claim 1, characterized in that, In step S1, the embedded ArUco marker consists of an outer large-scale marker and an inner small-scale marker. The inner small-scale marker is located in the central region of the outer large-scale marker, and the area around the inner small-scale marker is set as a white unit area to isolate the coding structure of the inner and outer markers.

3. The near-range positioning and long-range detection transformation method based on camera and triangulation transformation fusion according to claim 2, characterized in that, In step S2, during the long-distance detection stage, the acquired outer large-scale markers are first preprocessed by grayscale conversion, adaptive threshold segmentation, and contour extraction. Then, the existing ArUco marker detection algorithm is used to extract candidate quadrilateral contours from the image and perform perspective correction and dictionary matching to complete the recognition and decoding of the outer markers.

4. The near-range localization and long-range detection transformation method based on camera and triangulation transformation fusion according to claim 3, characterized in that, In step S2, the distance between the target marker and the camera is estimated: After detecting the large-scale outer marker, the pixel coordinates of the four corner points in the image coordinate system are extracted. The pixel side length of the outer large-scale marker is calculated based on the pixel distance between adjacent corner points, and the average pixel side length is taken as the equivalent pixel side length of the outer large-scale marker. : in, , , , These represent the coordinates of the four corner points; , , , These represent the side lengths of the four adjacent corner points; Let the actual physical side length of the outer large-scale marker be... The camera's equivalent focal length is Then, according to the pinhole imaging model, we can obtain: in, Mark the distance between the target and the camera.

5. The near-range localization and long-range detection transformation method based on camera and triangulation transformation fusion according to claim 4, characterized in that, In step S2, the camera is a depth camera. When the depth data is valid, the spatial three-dimensional coordinates of the target marker are calculated using the depth data. Obtain the center pixel of the outer large-scale marker. Corresponding depth value And combined with the camera intrinsic parameter matrix: in, The focal length of the camera in the horizontal direction. The focal length of the camera in the vertical direction. Principal point coordinates; The center pixel is back-projected onto the camera coordinate system to obtain the spatial coordinates of the target marker. : When depth data is invalid, distance perception is achieved solely based on the pixel scale and geometric relationships of the outer large-scale markers; The following conditions must be met to determine the validity of depth data: The depth value is not zero or there is no invalid placeholder value; The depth value did not exceed the effective measurement range of the depth camera; There are no anomalous abrupt changes in depth values ​​between spatial neighborhoods or adjacent time frames.

6. The near-range localization and long-range detection transformation method based on camera and triangulation transformation fusion according to claim 5, characterized in that, In step S3, the detection transformation criterion is satisfied when any of the following conditions are met: a. When the equivalent pixel side length of the outer large-scale marker Greater than or equal to the preset pixel threshold ; b. The distance between the target marker and the camera obtained from the pinhole imaging model Less than the preset distance threshold .

7. The near-range localization and long-range detection transformation method based on camera and triangulation transformation fusion according to claim 6, characterized in that, Preset pixel threshold Determined based on the smallest stable recognizable pixel scale of the inner layer small-scale markers under the current camera resolution and detection algorithm: ,in The minimum stable detection pixel side length for the inner small-scale marker under the current algorithm and imaging conditions; , The actual physical side length marked at the inner small scale. The side length of the outermost large-scale marker; Preset distance threshold This is obtained by substituting the minimum detectable pixel size of the inner layer small-scale markers into the pinhole imaging model. 。 8. The near-range localization and long-range detection transformation method based on camera and triangulation transformation fusion according to claim 7, characterized in that, In step S3, after detecting the outer large-scale marker, the center position and inscribed region range of the outer large-scale marker are calculated based on the pixel positions of the four corner points in the image coordinate system. Based on the center position, a region of interest (ROI) is constructed inside the outer large-scale marker. The ROI is obtained by scaling the pixel scale of the outer large-scale marker proportionally and is used to cover the area where the inner small-scale marker may appear in the close-range stage. The detection transformation phase enables the detection algorithm with inner-layer small-scale labeling only within the region of interest (ROI).

9. The near-range localization and long-range detection transformation method based on camera and triangulation transformation fusion according to claim 8, characterized in that, In step S4, when the inner small-scale markers are successfully identified in images with no less than a preset number of consecutive frames, and the change in the corner position in the pixel coordinate system is less than a set threshold, the detection result of the inner small-scale markers is determined to meet the stability requirement. During the close-range positioning phase, the target distance is: in, The equivalent pixel side length of the inner small-scale marker in the image is , Combine the pixel coordinates of the inner small-scale marker center point in the image Calculate the target's three-dimensional coordinates in the camera coordinate system to obtain precise positioning during the close-range positioning phase: in, This represents the spatial displacement of the target marker center along the horizontal direction of the camera in the camera coordinate system. This represents the spatial displacement of the target marker center in the camera coordinate system along the direction perpendicular to the camera. This indicates the distance between the center of the target marker and the optical center of the camera along the camera's optical axis.

10. A near-range positioning and long-range detection system based on camera and triangulation transformation fusion, characterized in that, The method for implementing the near-range localization and long-range detection transformation based on camera and triangulation transformation fusion as described in any one of claims 1-9 includes: The image acquisition module is used to acquire images containing chimeric ArUco tags in real time; The marker detection module is used to detect large-scale markers in the outer layer, and to complete the recognition and decoding of large-scale markers in the outer layer. The depth calculation module is used to calculate the three-dimensional coordinates of the target marker when the depth data is valid; The triangulation estimation module is used to estimate the 3D coordinates of the target marker based on triangulation when the depth data is invalid. The switching control module is used to switch between the long-range detection stage and the short-range positioning stage based on the detection transformation criterion and the stability criterion of the detection results of the inner small-scale markers.

Images