Composite AGV repositioning system and method with plate body as space reference

Abstract

The invention relates to a composite AGV repositioning system and method with a plate body as a space benchmark, and the system comprises a composite AGV platform which is used for guiding to a nominal position according to the information of a speedometer and a station map when a current process is completed and the plate body is transferred to a next station; the scheduling module is used for acquiring a plate surface number at the nominal position and a nominal pose set of a feature mark of the plate surface number; the point cloud acquisition module is used for triggering a point cloud camera to acquire local point cloud data in the marked theoretical area; the industrial control module is used for resolving the actual pose of each feature mark by using the local point cloud data, performing rigid body registration and point-to-point registration with the nominal pose set, generating a pose deviation between a board coordinate system and an AGV base coordinate system, further calculating a pose correction amount, and outputting the pose correction amount; and updating the poses of the AGV base and the robot base in the task coordinate system by using the pose correction amount. The positioning device aims to solve the problem of accurate positioning of large-size plates in a multi-station continuous grabbing scene.

Description

Technical Field

[0001] This invention relates to the field of AGV positioning technology, and in particular to a composite AGV repositioning system and method that uses the plate body as a spatial reference. Background Technology

[0002] Existing composite AGVs, when performing repositioning in continuous cross-station grasping operations, primarily rely on externally set landmarks or environmental features. The closest implementation to this technical solution typically involves placing artificial markers or geometric references on the ground, tooling, or workbench surface, such as QR codes, reflective targets, positioning pins, or reference blocks. When the AGV moves to the next station, its onboard robotic arm vision system searches for these landmarks within a pre-defined nominal area, calculating the vehicle's pose correction using homography transformation of 2D vision or finite 3D point cloud registration technology. This correction is then applied to the robot's base coordinate system to support subsequent grasping operations. In this type of solution, the markers are usually fixed to the site or tooling structure, lacking a direct geometric connection to the material being handled, and the marker generation process is independent of the material layout data, relying on separate calibration and teaching procedures before deployment for system maintenance. A typical example is KUKA's KMR mobile robot system, which achieves ±5mm positioning accuracy through pre-calibrated ground AprilTags.

[0003] Besides the aforementioned "landmark-based repositioning" method, several common alternative paths exist. One is the simultaneous localization and mapping (SLAM) repositioning technology that integrates odometer, inertial measurement unit (IMU), and lidar or camera: the automated guided vehicle (AGV) corrects its base posture near the workstation using loopback detection on the environmental map and sends this posture information to the robotic arm. Another approach is to use an external global positioning system, such as ultra-wideband (UWB) or optical motion capture systems, to provide the AGV with absolute posture information within its coverage area to correct for accumulated drift errors in the odometer. A third approach relies on mechanical positioning and docking pins to achieve repeatable positioning: after the AGV is in position, it forms a rigid mechanical coupling with the tooling, at which point the robotic arm's initial grasping posture is assumed to be repeatable. A fourth approach involves performing a large-scale 3D point cloud scan of the entire board and globally registering the 3D computer-aided design (CAD) model of the tooling or workbench with the measured point cloud, thereby indirectly deriving the correction amount for the AGV's base posture. While the aforementioned solutions can achieve a certain degree of repositioning accuracy and gripping action coordination under different working conditions, none of them incorporate the "identifiable features of the board surface" into the positioning feedback loop, and most lack a direct system-level connection with the material layout file. In other words, existing technical solutions achieve repositioning through ground or tooling markings, offering the advantage of ease of implementation but exhibiting significant limitations in engineering applications. First, the decoupling of the marking location from the board surface results in a lack of direct constraint between the AGV base coordinate system and the board surface coordinate system. When the board's placement posture is slightly disturbed or the tooling undergoes thermal deformation, repositioning based on site landmarks cannot accurately reflect the board's true posture, causing cumulative system deviations in the robotic arm's initial gripping posture at the new workstation. Second, these solutions heavily rely on modifications to the work site and continuous maintenance. The cleanliness, wear, obstruction status of the markings, and changes in ambient lighting significantly affect the stability of the detection system; when changing production lines or products, markings need to be rearranged and calibrated, resulting in insufficient flexibility and high maintenance costs. Third, 2D vision-based marker recognition technology struggles to provide reliable height and tilt information and is highly sensitive to surface reflections, shadows, and localized occlusions. Its pose calculation process often relies on additional assumptions or simplified models, making it difficult to meet the high-precision requirements of cross-station grasping operations for large-size sheet metal. Fourth, the marker system and the nesting file are independent, lacking a unified data source for the nesting stage. After deployment, inconsistencies arise between the nominal pose of the markers and the theoretical layout of the nested parts, causing a break in the data chain and significantly increasing the complexity of deployment configuration and troubleshooting abnormal operating conditions.

[0004] SLAM-based alternatives also have inherent limitations, mainly in three aspects. First, the system is highly dependent on map construction and sensitive to dynamic environmental changes. Temporary material stacking at the cutting site, displacement of equipment around the workstation, dust interference, and metal reflections can easily lead to degradation of environmental features, significantly reducing the stability of SLAM repositioning. Second, the vehicle's pose relative to the environment output by SLAM is difficult to directly map to the board coordinate system. When relative displacement or deflection occurs between the board and the tooling fixture, the initial coordinates of the robotic arm's grasping operation still need to rely on additional sensor data for pose compensation. Third, there is a time delay in the loop closure detection and repositioning process. Cross-workstation repositioning operations will interfere with the stability of the production line cycle time, making it difficult to meet the strict timing requirements of "grab as soon as it arrives in position".

[0005] Existing technologies also disclose external positioning systems that can provide absolute pose information, but these are costly to deploy, complex to maintain, and sensitive to interference factors such as occlusion, reflection, and multipath effects. In metal processing workshop environments, both UWB signals and optical marker spheres are susceptible to equipment occlusion and metal surface reflection, requiring additional infrastructure and regular calibration to build high-reliability coverage areas. More critically, such systems are decoupled from the board itself, and the robotic arm still needs to incorporate board geometry perception and pose alignment processes, resulting in a longer engineering implementation chain and more potential sources of error.

[0006] Mechanical positioning and pin-connection methods are feasible for repetitive production; however, they lack flexibility and are highly dependent on tooling accuracy. Large, thin plates are susceptible to warping due to temperature changes and residual stress. Even with repeatable mechanical coordination between the AGV and tooling, the actual position of the plate may still deviate from its nominal coordinates. To ensure safety during the gripping process, additional height and tilt angle detection and compensation mechanisms are required, leading to increased system complexity and debugging costs.

[0007] Existing technical solutions also disclose a method combining full-board scanning with global point cloud registration to provide board surface pose information, but data acquisition and calculation are time-consuming. High-density scanning of large boards will significantly impact production cycle time. Furthermore, factors such as repeated processing of geometric features, residual material, broken bridges, and surface burrs can interfere with the stability of boundary segmentation and point cloud registration. These methods often require multiple initializations and manual intervention, making them unsuitable for the needs of continuous multi-station automated grasping.

[0008] The objective drawbacks of the existing technical solutions are shown in Table 1.

[0009] Table 1 Summary of Objective Disadvantages of Various Technologies

[0010] Technology Category Core limitations Quantitative indicators Landmark Positioning Markings are decoupled from the board material, making them sensitive to environmental interference. Positioning error >3mm under heat deformation conditions (when the sheet length is 5m) SLAM relocation Strong map dependency, poor robustness in dynamic scenes The failure rate of loopback detection in dusty environments is >40%. External positioning system Obstruction is a sensitive issue, and infrastructure costs are high. UWB positioning error in metal workshops increases exponentially with the duration of occlusion (half-life 8s). Full-board scanning registration It takes too long and cannot meet the rhythm requirements. Scanning a 2m×4m sheet material takes more than 15 seconds (at a point cloud density of 100 points / mm²).

[0011] In summary, existing technologies generally suffer from three common defects in cross-station continuous grasping and repositioning tasks. First, the data link lacks closed-loop integrity. The repositioning data source and the material layout file are isolated, and the material itself is not involved in positioning constraints, leading to a systematic deviation between the AGV base correction and the actual pose of the material. Second, robustness and cycle time performance are difficult to simultaneously satisfy. Two-dimensional vision is susceptible to glare interference, SLAM and external positioning are sensitive to environmental changes, and whole-board point cloud processing is time-consuming, all making stable operation difficult under strong interference and high-cycle production conditions. Third, maintenance and replacement costs are high. Reliance on site modifications, peripheral maintenance, complex calibration, and manual secondary confirmation makes it difficult to support the flexible production needs of multiple varieties and small batches. These objective limitations collectively restrict the widespread application of existing technologies in large-size material and multi-station continuous grasping scenarios, and also provide a clear technical improvement direction for the technical route proposed in this study: "embedding material feature markers in the material layout stage and directly aligning the material with point cloud detection." Summary of the Invention

[0012] In order to solve the problems existing in the prior art, the purpose of this invention is to provide a composite AGV repositioning system and method with the plate body as the spatial reference, which aims to solve the problem of accurate positioning in the scenario of continuous grasping of large-size plates at multiple stations.

[0013] To achieve the above objectives, the present invention provides the following solution:

[0014] A composite AGV repositioning system using the sheet metal itself as a spatial reference includes:

[0015] The composite AGV platform is used to guide the AGV to its nominal position based on odometer information and workstation map when the current process is completed and the AGV is transferred to the next workstation.

[0016] The scheduling module is used to obtain the nominal pose set of the board number and its feature markers at the nominal position;

[0017] The point cloud acquisition module is used to trigger the point cloud camera to acquire local point cloud data within the marked theoretical area;

[0018] The industrial control module is used to calculate the actual pose of each feature marker using the local point cloud data, and to perform rigid body registration and point-to-point registration with the nominal pose set to generate the pose deviation between the board coordinate system and the AGV base coordinate system. It further calculates the pose correction amount and uses the pose correction amount to update the pose of the AGV base and robot base in the task coordinate system.

[0019] Optionally, the system also includes:

[0020] The coordinate calibration module is used to calibrate the external parameters from the robot flange coordinate system to the camera coordinate system and the external parameters from the robot base coordinate system to the AGV base coordinate system before the composite AGV platform starts working.

[0021] Optionally, the scheduling module includes:

[0022] The panel number acquisition submodule is used to obtain the panel number at the nominal location;

[0023] The nominal pose acquisition submodule is used to mark the center-to-center distance and side length of the L-shaped corner mark and the concentric circle three-point matrix combination, as well as the effective part contour boundary. It introduces slight asymmetric features on the L-shaped corner mark or the three-point matrix as directional anchor points, and each mark is assigned a unique board surface number. The nominal pose information corresponding to the board surface number is retained in the GNC file. The nominal pose information includes: two-dimensional coordinates of the board plane, in-plane orientation, and mark size parameters.

[0024] Optionally, the industrial control module includes:

[0025] The point cloud detection submodule is used to perform planar model fitting on the local point cloud data in the camera coordinate system, calculate the normal of the plate surface and the plane equation, and remove significant out-of-plane points and specular reflection pseudo-points.

[0026] The pose calculation module is used to perform sub-pixel level geometric fitting of corner points and center points in the planar projection space, and then map the feature markers to the three-dimensional space reconstructed coordinates. That is, for L-shaped corner markers, the least squares fitting of two intersecting straight lines is used to solve the corner point coordinates and direction vectors. The direction vectors and the normal of the board surface jointly define the in-plane pose of the markers. For three-point lattice markers, a local coordinate system is constructed through the three centroids of the circles and the two baseline directions.

[0027] The shape distance and topological consistency score of the feature markers in the reconstructed 3D spatial coordinates are calculated. Candidate markers that deviate from the nominal size threshold are removed. A constraint set is constructed by using a point-to-surface iterative nearest point algorithm or a normal distribution transformation algorithm to mark the geometric center and feature boundary points. The maximum number of iterations and the convergence threshold are preset to obtain the actual pose.

[0028] Optionally, the industrial control module further includes:

[0029] The positioning correction submodule is used to perform rigid body registration between the actual pose and the nominal pose set, obtain the transformation matrix from the plate coordinate system to the AGV base coordinate system, and solve the transformation matrix using the least squares method or a point-to-point registration algorithm based on singular value decomposition when the number of valid markers is not less than three, obtain the translation vector and rotation matrix, and obtain the pose correction amount using the translation vector and the rotation matrix. When the number of valid markers is less than three, the theoretical region of the markers is expanded, and the shooting height or incident angle of the point cloud camera is adjusted for data re-acquisition to ensure that the number of valid markers is not less than three.

[0030] The task coordinate update submodule is used to update the pose of the AGV base and robot base in the task coordinate system using the pose correction amount.

[0031] Optionally, the system also includes:

[0032] The safety and interlock module is used to ensure that the movement speed and acceleration of the robotic arm are constrained by a preset upper limit during the transition from shooting posture to correction posture, and to activate collision detection and real-time monitoring of joint torque.

[0033] If the pose correction result does not meet the preset quality criteria, the safety limit threshold for the gripping operation shall be prohibited from being lifted. If the end effector detects that the clamp has not reached the safety threshold in vacuum pressure or the electromagnetic adsorption device has not fed back a closing signal, the lifting process shall be prohibited.

[0034] To achieve the above objectives, the present invention also provides a method for repositioning a composite AGV using the sheet metal body as a spatial reference, comprising:

[0035] When the current process is completed and the work is transferred to the next workstation, the system guides the user to the nominal position based on the odometer information and the workstation map, and obtains the nominal pose set of the board number and its feature markers at the nominal position.

[0036] The point cloud camera is triggered to collect local point cloud data within the marked theoretical area. The actual pose of each feature marker is calculated using the local point cloud data, and rigid body registration and point-to-point registration are performed with the nominal pose set to generate the pose deviation between the board coordinate system and the AGV base coordinate system. The pose correction amount is further calculated, and the pose of the AGV base and robot base in the task coordinate system is updated using the pose correction amount.

[0037] Optionally, obtaining the nominal pose set includes:

[0038] The center-to-center distance and side length of the L-shaped corner mark and the concentric circle lattice are marked, as well as the effective part outline boundary. Slight asymmetric features are introduced on the L-shaped corner mark or the lattice as directional anchor points. Each mark is assigned a unique board surface number. The nominal pose information corresponding to the board surface number is retained in the GNC file. The nominal pose information includes: two-dimensional coordinates of the board plane, in-plane orientation, and mark size parameters.

[0039] Optionally, obtaining the actual pose includes:

[0040] Subpixel-level geometric fitting of corner points and center points is performed in the planar projection space. Then, the feature markers are mapped to the three-dimensional space reconstructed coordinates. That is, for L-shaped corner markers, the least squares fitting of two intersecting straight lines is used to solve the corner point coordinates and direction vectors. The direction vectors and the normal of the board surface jointly define the in-plane attitude of the markers. For three-point lattice markers, a local coordinate system is constructed through the three centroids of the circles and the two baseline directions.

[0041] The shape distance and topological consistency score of the feature markers in the reconstructed 3D spatial coordinates are calculated. Candidate markers that deviate from the nominal size threshold are removed. A constraint set is constructed by using a point-to-surface iterative nearest point algorithm or a normal distribution transformation algorithm to mark the geometric center and feature boundary points. The maximum number of iterations and the convergence threshold are preset to obtain the actual pose.

[0042] Optionally, obtaining the pose correction amount includes:

[0043] Rigid body registration is performed between the actual pose and the nominal pose set to obtain the transformation matrix from the plate coordinate system to the AGV base coordinate system. When the number of effective markers is not less than three, the transformation matrix is ​​solved using the least squares method or a point-to-point registration algorithm based on singular value decomposition to obtain the translation vector and rotation matrix. The pose correction amount is obtained using the translation vector and the rotation matrix. When the number of effective markers is less than three, the theoretical region of the markers is expanded, and the shooting height or incident angle of the point cloud camera is adjusted for data re-acquisition to ensure that the number of effective markers is not less than three.

[0044] The beneficial effects of this invention are as follows:

[0045] The key technical point of this invention lies in integrating the "identifiable panel markings" with the guidance, navigation, and control (GNC) system during the material arrangement stage, and efficiently calculating the pose correction amount from the panel to the vehicle body based on point cloud data. Specifically:

[0046] The design incorporates L-shaped and concentric circle three-point matrix combination markings with directional determination capabilities, limiting line width, spacing, and minimum safety distance. A slightly asymmetrical anchor point structure is also introduced to ensure robust geometric properties even under conditions of high reflectivity and edge burrs after cutting.

[0047] A process for local point cloud acquisition and template geometry fitting based on nominal pose is proposed: after sub-pixel-level fitting of corner points and circle centers in the projection domain of the board surface, the coordinates of feature points are recovered by back-projection to three-dimensional space, thereby achieving fast and low-noise marker detection.

[0048] By utilizing multi-label over-constraint conditions and employing the singular value decomposition (SVD) least squares registration method, the rigid body transformation matrix from the plate surface to the AGV base is directly solved. The extrinsic parameters of the hand-eye system and the feature fitting noise are then propagated to the covariance matrix of the pose correction through the Jacobian matrix, forming a quantifiable confidence criterion.

[0049] The rigid body transformation matrix of the Automated Guided Vehicle (AGV) base is synchronously applied to the coordinate system of the AGV and the robot task to establish a "grab upon arrival" repositioning link. When the number of effective markers is insufficient or the quality score does not reach the threshold, the system automatically expands the region of interest (ROI), adjusts the laser incident angle, and performs secondary data acquisition. After retrying a maximum of two times, it switches to the safety degradation mode to ensure production cycle and operational safety.

[0050] A minimally modified engineering integration framework is constructed, which combines external triggers and direct Ethernet communication between AGVs, robots and cameras. All pose corrections, quality indicators and anomaly cause codes are stored in the database, supporting long-term accuracy consistency for continuous grasping operations across workstations. Attached Figure Description

[0051] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0052] Figure 1 This is a schematic diagram of a composite AGV repositioning method using the plate body as a spatial reference according to an embodiment of the present invention;

[0053] Figure 2 This is a schematic diagram of the point cloud camera setup according to an embodiment of the present invention;

[0054] Figure 3 This is a schematic diagram of the template matching results in an embodiment of the present invention. Detailed Implementation

[0055] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0056] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0057] like Figure 1 As shown, this embodiment discloses a composite AGV repositioning method using the sheet metal itself as a spatial reference. Its core lies in: integrating feature markers with orientation determination capabilities and geometric stability into the GNC cutting process file during the material feeding stage; after the current process is completed and the material is transferred to the next workstation, the robotic arm mounted on the AGV drives a 3D point cloud camera to perform rapid 3D scanning within the marked theoretical area; the pose deviation between the sheet metal surface coordinate system and the AGV body coordinate system is directly calculated using the point cloud data, and this real-time correction is synchronously applied to the task coordinate systems of the AGV base and the robotic arm base, thereby achieving high-precision "grabbing upon arrival" operation. This method uses the sheet metal surface as the sole spatial reference, avoiding reliance on ground markings, external positioning systems, and global scanning of the entire sheet metal, while simultaneously considering repositioning accuracy, system robustness, and production cycle requirements, significantly improving the efficiency of continuous multi-station operations.

[0058] This embodiment discloses a composite AGV repositioning system with the sheet metal body as the spatial reference. The system consists of a composite AGV platform, a six-degree-of-freedom industrial robot, and an eye-on-hand point cloud camera. Figure 2 The composite AGV consists of an end effector, an industrial control computer, and a scheduling and safety interconnection system. The composite AGV is equipped with an odometer, IMU unit, and chassis control interface; the industrial robot provides flange pose data and motion control interface; the point cloud camera supports external triggering mode and Ethernet data stream transmission; the end effector can output adsorption status or vacuum feedback signals. The industrial control computer integrates three core software modules: a feature marking and GNC integrated output module, a point cloud detection and pose calculation module, and a repositioning correction and task coordinate update module. The scheduling system issues the workstation sequence and target panel number to the composite AGV. The system performs a single repositioning closed-loop operation at each workstation and writes the results back to the database for cycle time statistics and quality traceability analysis.

[0059] Feature marking is achieved by cutting geometric patterns in non-functional areas or scrap areas of the sheet metal plane. A combination of L-shaped corner markers containing directional information and concentric three-point lattices is preferred. The center-to-center distance and side length of the markers are in whole millimeters, with a minimum line width of 1 mm and a safe distance of at least 10 mm between the marker and the boundary of the effective part contour. At least three non-collinear markers are placed on each sheet metal plane, preferably four to six, to enhance the constraint stability of attitude calculation. Each marker is assigned a unique number in the nesting software, and its nominal pose information, including the two-dimensional coordinates of the sheet metal plane, in-plane orientation, and marker size parameters, is retained in the GNC file. To avoid directional ambiguity caused by pattern rotational symmetry, slight asymmetric features are introduced on the L-shaped corner markers or three-point lattices as directional anchor points. The marker cutting process should be arranged after the machining of other contours of the sheet metal to avoid warping of the heat-affected zone of the cut affecting the detection area.

[0060] The coordinate system adopts a unified symbol system. The world coordinate system (W) serves as the reference benchmark for log recording and offline analysis. The AGV base coordinate system (A) is defined as the vehicle body coordinate system. The robot base coordinate system (B) and coordinate system (A) are connected by fixed external parameters determined through assembly calibration. The pose of the robot flange coordinate system (F) is determined in real time by joint measurement data. The camera coordinate system (C) establishes a rigid body transformation relationship with coordinate system (F) through hand-eye calibration. The normal direction and in-plane direction of the plate coordinate system (P) are calculated during the detection stage. The feature marker coordinate system (M) has a nominal pose in coordinate system (P). Two types of calibration need to be completed before production: the first type is hand-eye calibration, which aims to obtain the external parameters from coordinate system (F) to coordinate system (C). This calibration process is achieved by collecting multiple sets of posture data and applying the least squares algorithm. The calibration residuals and covariance matrices are stored in the parameter library. The second category involves calibrating the external parameters from the robot base coordinate system (B) to the AGV base coordinate system (A). This is accomplished using fixed markers or high-precision measurement methods, ultimately determining the static external parameters from coordinate system (B) to coordinate system (A). The above calibration process is equipped with a rapid re-inspection mechanism on-site to ensure the consistency of the system's long-term operation.

[0061] like Figure 3As shown, the repositioning workflow revolves around the panel markings. After the composite AGV is guided to its nominal position based on odometer information and the workstation map, the scheduling system issues the nominal pose set of the current panel number and its feature markings. Subsequently, the robotic arm moves above the nominal area of ​​the markings, triggering the point cloud camera to acquire local point cloud data. The vision system performs point cloud denoising and normal estimation in the camera coordinate system (C), and transforms the point cloud to the AGV base coordinate system (A) using hand-eye extrinsic parameters and the real-time flange pose. Next, template-based point cloud matching and geometric consistency verification are performed in this coordinate system. Using known geometric constraints such as marking size, corner points, and center points, the actual pose of each marking is calculated. Then, these actual pose sets are rigidly registered with the nominal pose set provided by the GNC system to solve for the transformation matrix from the panel coordinate system (P) to the AGV base coordinate system (A). When there are at least three valid markers, the translation vector and rotation matrix are solved using the least squares method or a point-to-point registration algorithm based on singular value decomposition (SVD). For point-to-point registration, the key geometric points of the feature markers are used as fixed registration points, not randomly selected, but based on preset key geometric points of the feature markers, and the correspondence between registration pairs is fixed. The root mean square residual (RMSE) and maximum angular error of the pose correction are calculated, and their uncertainty is evaluated. The obtained pose correction is directly used to update the pose representation of the AGV base in the task coordinate system in real time. At the same time, the same transformation is applied to update the pose of the robot base in the task coordinate system, ensuring that both are accurately aligned with the board coordinate system (P) at the new workstation.

[0062] The algorithm strictly adheres to the principles of prior guidance and precise local registration. The system first generates a predefined nominal region of interest boundary to suppress interfering points and improve matching efficiency. For the local point cloud, a planar model is fitted, calculating the board surface normal and plane equations. This is used for point cloud denoising and data filtering, providing a spatial reference for the 3D pose calculation of feature markers, and establishing a connection between the "local point cloud-board surface coordinate system." This provides a precise projection reference for the planar projection space and eliminates significantly out-of-plane points and specular reflection pseudopoints. Sub-pixel-level corner point and center point geometric fitting is performed in the planar projection space, and then the features are mapped to reconstructed coordinates in 3D space. For L-shaped corner markers, the least-squares fitting of two intersecting lines is used to solve for the corner point coordinates and direction vectors. This direction vector, together with the board surface normal, defines the in-plane pose of the marker. For three-point lattice markers, a local coordinate system is constructed using the three centroids and two baseline directions to eliminate scale errors. In the template matching stage, shape distance and topological consistency scores are calculated to filter valid feature markers and eliminate candidate markers that deviate from the nominal size threshold. The registration stage employs either a point-to-surface iterative nearest-point algorithm or a normal distribution transformation algorithm. A constraint set is constructed using the marked geometric center and feature boundary points. A maximum number of iterations and a convergence threshold are preset to ultimately obtain a stable marked actual pose. The observation noise and hand-eye extrinsic parameter covariance matrix of all feature points are transmitted to the final pose correction covariance matrix via the Jacobian propagation method. This covariance is used for subsequent evaluation of the correction accuracy, ensuring command reliability. Based on this, the system outputs confidence intervals and qualification criteria.

[0063] This embodiment also discloses a workflow for a composite AGV repositioning system using the sheet metal body as a spatial reference: First, the "coordinate calibration" step before the process completes hand-eye calibration, acquiring the external parameters and calibration residuals from the robot flange coordinate system to the camera coordinate system, and then generating and storing the hand-eye external parameter covariance matrix, which is the initial error input for covariance matrix calculation; Next, the "point cloud acquisition and feature calculation" step in the process performs planar model fitting on the local point cloud, removes out-of-plane points and pseudo-points, and then performs sub-pixel level corner point and center geometric fitting. The shape distance deviation and topology consistency score residual generated during the process are analyzed. Differential quantization yields the feature point observation noise, which forms the real-time error input for covariance matrix calculation. Subsequently, in the "pose registration and correction generation" stage, while solving the transformation matrix (pose correction) from the board coordinate system to the AGV base coordinate system, a Jacobian matrix is ​​constructed to map the error propagation relationship. Combining the hand-eye extrinsic parameter covariance matrix and the feature point observation noise, the final pose correction covariance matrix is ​​calculated using the Jacobian propagation method. This matrix is ​​also used in the "correction quality criterion judgment" stage to help determine whether the correction meets the accuracy requirements. The whole process runs through the entire "preparation-execution-evaluation" link of the repositioning workflow.

[0064] Recommended value ranges for key thresholds and parameters are provided based on conventional industrial hardware configurations. For point cloud cameras operating at distances between 300mm and 600mm, a spatial resolution better than 0.4mm is recommended, and single-frame acquisition time should be constrained to the range of 200ms to 400ms. The recommended threshold for the root mean square error (RMSE) of geometric fitting for a single marker point should not exceed 0.8mm, and the angular error threshold should not exceed 0.5°. The recommended value for the overall residual RMSE threshold of board registration is not higher than 1.0mm, and the lower limit of the interior point proportion threshold is recommended to be set at 70%. To ensure continuous production cycle time requirements, the detection and fitting time for a single marker point should be controlled within the range of 0.2s to 0.4s, and the complete repositioning calculation time for four marker points should be constrained to within 1.0s. Under the working condition of a standard 2m×4m board surface, the time delay from the completion of repositioning to the start of the gripping action should not exceed 2.0s to meet the "grab upon arrival" cycle time target. The above parameters can be adjusted on-site according to the camera model, the reflectivity of the plate surface, and the target accuracy requirements. The system supports online dynamic adjustment and fully records all parameter changes.

[0065] Robust design is implemented at both the hardware and software levels. On the hardware side, a polarizer combined with an oblique incidence lighting scheme effectively suppresses the interference of metal mirror reflections on geometric fitting; when necessary, a dual-view short baseline structure is employed to significantly improve the geometric stability of corner and center features. On the software side, normal consistency filtering and median filtering suppress burr and slag noise; a model-based outlier removal algorithm and consistency verification mechanism are used to avoid incorrect target locking. When the number of effective marker points is insufficient or the quality score is lower than a preset threshold, the system automatically expands the region of interest and adjusts the shooting height or incident angle for data reacquisition, performing a maximum of two retries. Tasks that fail to retry are transferred to a manual review process, while the system maintains a conservative pose correction strategy to ensure the safety of subsequent workpiece grasping operations. All task failures and retry events are fully recorded in the system log, including timestamps, camera frame numbers, relevant threshold configurations, and cause codes, supporting offline reproduction and quality auditing.

[0066] To ensure the reliability of online operations, the system implements the following safety and interlocking strategies: During the transition from shooting posture to correction posture, the robotic arm's speed and acceleration are constrained by preset upper limits, and collision detection and real-time joint torque monitoring functions are activated simultaneously; when the pose correction result does not meet the preset quality criteria, the safety limit threshold for the gripping operation is prohibited from being lifted; the lifting process is prohibited before the vacuum pressure reaches the safety threshold or the electromagnetic adsorption device returns a closing signal; the scheduling system can only switch to the gripping task after the panel repositioning is successful; if two consecutive panel repositionings fail, a maintenance alarm mechanism is triggered and the automated process is suspended.

[0067] The detection of two consecutive board repositioning failures includes: firstly, using the root mean square residual of pose correction amount ≤1.0mm and the number of effective markers ≥3 as the criteria for judging single board repositioning failure. If any criterion is not met, it is judged as a failure and recorded; the results are stored in the failure database of the industrial control module, and a consecutive failure counter is set. The counter is incremented by 1 when there is a failure and reset to zero when there is a success. When the counter value reaches 2, two consecutive board repositioning failures are detected, triggering a maintenance alarm and pausing the automated process.

[0068] This embodiment also constructs an engineering integration framework with minimal modifications. By combining external triggers and direct Ethernet communication between AGV, robot and camera, all pose corrections, quality indicators and abnormal cause codes are stored in the database, supporting long-term accuracy consistency of continuous grasping operations across workstations.

[0069] This embodiment discloses a repositioning method for a composite AGV using the sheet metal body as a spatial reference. The method includes: On a production line with three gripping stations, the composite AGV moves from the first station to the second station after completing a gripping operation. Accumulated odometer error results in a nominal positioning error of approximately 3-5 mm. At the second station, the system detects four reference markers. The root mean square error (RMSE) of the geometric fit for each marker is between 0.4 and 0.6 mm, and the in-plane angle error is less than 0.3 degrees. The root mean square error of the registration residual for the four markers is 0.8 mm, and the repositioning calculation takes 0.9 seconds. After applying the obtained pose correction to the AGV base and the robot base, the robotic arm performs a gripping operation in the new task coordinate system. The alignment error of its end effector, verified offline, is less than 1 mm, meeting the set safety margin requirements. The process is repeated at the third station. The system has not experienced any repositioning quality issues in 30 consecutive operating cycles, and the log records are complete and traceable.

[0070] This embodiment can form a complete closed-loop implementation path. At the method level, it includes the following complete process: tag embedding and numbering, local point cloud acquisition and template matching, board surface registration and pose correction, task coordinate system update and quality criterion determination. At the system level, it includes a hardware and software combination consisting of a composite AGV platform, industrial robot, point cloud camera, industrial control computer, and scheduling safety system, and achieves system interconnection through a standard industrial Ethernet interface with the controller. At the media level, it provides executable programs, parameter libraries, and log structures on the industrial control computer, supporting rapid deployment and long-term maintenance. This technical solution uses the workpiece board surface as a reference benchmark, directly establishing a repositioning closed loop between the board surface and the moving carrier (vehicle body), eliminating external benchmark dependence, and possessing engineering practicality and promotional value in actual industrial scenarios with strong reflection, high dust levels, and tight schedules.

[0071] The embodiments described above are merely preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Various modifications and improvements made to the technical solutions of the present invention by those skilled in the art without departing from the spirit of the present invention should fall within the protection scope defined by the claims of the present invention.

Claims

1. A composite AGV repositioning system using the sheet metal body as a spatial reference, characterized in that, include: The composite AGV platform is used to guide the AGV to its nominal position based on odometer information and workstation map when the current process is completed and the AGV is transferred to the next workstation. The scheduling module is used to obtain the nominal pose set of the board number and its feature markers at the nominal position; The point cloud acquisition module is used to trigger the point cloud camera to acquire local point cloud data within the marked theoretical area; The industrial control module is used to calculate the actual pose of each feature marker using the local point cloud data, and to perform rigid body registration and point-to-point registration with the nominal pose set to generate the pose deviation between the board coordinate system and the AGV base coordinate system. It further calculates the pose correction amount and uses the pose correction amount to update the pose of the AGV base and robot base in the task coordinate system.

2. The composite AGV repositioning system with the plate body as a spatial reference according to claim 1, characterized in that, The system also includes: The coordinate calibration module is used to calibrate the external parameters from the robot flange coordinate system to the camera coordinate system and the external parameters from the robot base coordinate system to the AGV base coordinate system before the composite AGV platform starts working.

3. The composite AGV repositioning system with the plate body as a spatial reference according to claim 1, characterized in that, The scheduling module includes: The panel number acquisition submodule is used to obtain the panel number at the nominal location; The nominal pose acquisition submodule is used to mark the center-to-center distance and side length of the L-shaped corner mark and the concentric circle three-point matrix combination, as well as the effective part contour boundary. It introduces slight asymmetric features on the L-shaped corner mark or the three-point matrix as directional anchor points, and each mark is assigned a unique board surface number. The nominal pose information corresponding to the board surface number is retained in the GNC file. The nominal pose information includes: two-dimensional coordinates of the board plane, in-plane orientation, and mark size parameters.

4. The composite AGV repositioning system with the plate body as a spatial reference according to claim 1, characterized in that, The industrial control module includes: The point cloud detection submodule is used to perform planar model fitting on the local point cloud data in the camera coordinate system, calculate the normal of the plate surface and the plane equation, and remove significant out-of-plane points and specular reflection pseudo-points. The pose calculation module is used to perform sub-pixel level geometric fitting of corner points and center points in the planar projection space, and then map the feature markers to the three-dimensional space reconstructed coordinates. That is, for L-shaped corner markers, the least squares fitting of two intersecting straight lines is used to solve the corner point coordinates and direction vectors. The direction vectors and the normal of the board surface jointly define the in-plane pose of the markers. For three-point lattice markers, a local coordinate system is constructed through the three centroids of the circles and the two baseline directions. The shape distance and topological consistency score of the feature markers in the reconstructed 3D spatial coordinates are calculated. Candidate markers that deviate from the nominal size threshold are removed. A constraint set is constructed by using a point-to-surface iterative nearest point algorithm or a normal distribution transformation algorithm to mark the geometric center and feature boundary points. The maximum number of iterations and the convergence threshold are preset to obtain the actual pose.

5. The composite AGV repositioning system with the plate body as a spatial reference according to claim 4, characterized in that, The industrial control module also includes: The positioning correction submodule is used to perform rigid body registration between the actual pose and the nominal pose set, obtain the transformation matrix from the plate coordinate system to the AGV base coordinate system, and solve the transformation matrix using the least squares method or a point-to-point registration algorithm based on singular value decomposition when the number of valid markers is not less than three, obtain the translation vector and rotation matrix, and obtain the pose correction amount using the translation vector and the rotation matrix. When the number of valid markers is less than three, the theoretical region of the markers is expanded, and the shooting height or incident angle of the point cloud camera is adjusted for data re-acquisition to ensure that the number of valid markers is not less than three. The task coordinate update submodule is used to update the pose of the AGV base and robot base in the task coordinate system using the pose correction amount.

6. The composite AGV repositioning system with the plate body as a spatial reference according to claim 1, characterized in that, The system also includes: The safety and interlock module is used to ensure that the movement speed and acceleration of the robotic arm are constrained by a preset upper limit during the transition from shooting posture to correction posture, and to activate collision detection and real-time monitoring of joint torque. If the pose correction result does not meet the preset quality criteria, the safety limit threshold for the gripping operation shall be prohibited from being lifted. If the end effector detects that the clamp has not reached the safety threshold in vacuum pressure or the electromagnetic adsorption device has not fed back a closing signal, the lifting process shall be prohibited.

7. A method for repositioning a composite AGV using the sheet metal body as a spatial reference, implemented according to any one of claims 1-6, characterized in that, include: When the current process is completed and the work is transferred to the next workstation, the system guides the user to the nominal position based on the odometer information and the workstation map, and obtains the nominal pose set of the board number and its feature markers at the nominal position. The point cloud camera is triggered to collect local point cloud data within the marked theoretical area. The actual pose of each feature marker is calculated using the local point cloud data, and rigid body registration and point-to-point registration are performed with the nominal pose set to generate the pose deviation between the board coordinate system and the AGV base coordinate system. The pose correction amount is further calculated, and the pose of the AGV base and robot base in the task coordinate system is updated using the pose correction amount.

8. The composite AGV repositioning method using the plate body as a spatial reference according to claim 7, characterized in that, Obtaining the nominal pose set includes: The center-to-center distance and side length of the L-shaped corner mark and the concentric circle lattice are marked, as well as the effective part outline boundary. Slight asymmetric features are introduced on the L-shaped corner mark or the lattice as directional anchor points. Each mark is assigned a unique board surface number. The nominal pose information corresponding to the board surface number is retained in the GNC file. The nominal pose information includes: two-dimensional coordinates of the board plane, in-plane orientation, and mark size parameters.

9. The composite AGV repositioning method using the sheet metal body as a spatial reference according to claim 7, characterized in that, Obtaining the actual pose includes: Subpixel-level geometric fitting of corner points and center points is performed in the planar projection space. Then, the feature markers are mapped to the three-dimensional space reconstructed coordinates. That is, for L-shaped corner markers, the least squares fitting of two intersecting straight lines is used to solve the corner point coordinates and direction vectors. The direction vectors and the normal of the board surface jointly define the in-plane attitude of the markers. For three-point lattice markers, a local coordinate system is constructed through the three centroids of the circles and the two baseline directions. The shape distance and topological consistency score of the feature markers in the reconstructed 3D spatial coordinates are calculated. Candidate markers that deviate from the nominal size threshold are removed. A constraint set is constructed by using a point-to-surface iterative nearest point algorithm or a normal distribution transformation algorithm to mark the geometric center and feature boundary points. The maximum number of iterations and the convergence threshold are preset to obtain the actual pose.

10. The composite AGV repositioning method using the sheet metal body as a spatial reference according to claim 9, characterized in that, Obtaining the pose correction amount includes: Rigid body registration is performed between the actual pose and the nominal pose set to obtain the transformation matrix from the plate coordinate system to the AGV base coordinate system. When the number of effective markers is not less than three, the transformation matrix is ​​solved using the least squares method or a point-to-point registration algorithm based on singular value decomposition to obtain the translation vector and rotation matrix. The pose correction amount is obtained using the translation vector and the rotation matrix. When the number of effective markers is less than three, the theoretical region of the markers is expanded, and the shooting height or incident angle of the point cloud camera is adjusted for data re-acquisition to ensure that the number of effective markers is not less than three.

Images