Mass production calibration method and system for external parameters of visual sensor of wheeled robot

Abstract

The invention discloses a mass production calibration method and system for external parameters of a visual sensor of a wheeled robot, and aims to solve the problems of high calibration cost, low efficiency and poor consistency in the prior art. The method comprises the steps that in a calibration room provided with known three-dimensional coordinate visual targets, a robot is accurately positioned to a preset static calibration position aligned with a world coordinate system of the calibration room through a mechanical tool; a visual sensor of the robot collects an image containing the visual target at the static calibration position; and finally, based on the detected two-dimensional pixel coordinates of the visual target in the image and the known three-dimensional coordinates of the world coordinate system, solving the external parameters of the camera through a SlovePnP algorithm. According to the invention, through mechanical fine positioning and an automatic process, the calibration efficiency and consistency are significantly improved, and low-cost and traceable unmanned quantity-production calibration is realized.

Description

Technical Field

[0001] This invention relates to the field of mass production calibration of robot sensors and intelligent manufacturing automation, specifically to a system and method for automating the external parameter calibration of vision cameras in the mass production stage of wheeled mobile robots. Background Technology

[0002] In wheeled mobile robots, such as service robots, AGVs (Automated Guided Vehicles), and AMRs (Autonomous Mobile Robots), multi-sensor fusion is a key technology for achieving high-precision positioning, navigation, and environmental perception. Among these, visual sensors (such as cameras) are the core sensing units, and their extrinsic parameters relative to the robot's body coordinate system—a six-degree-of-freedom pose transformation matrix—must be accurately calibrated. The accuracy of these extrinsic parameters directly affects the performance of the robot's core functions, such as SLAM (Simultaneous Localization and Mapping), obstacle avoidance, and path planning. During the robot's R&D phase, extrinsic parameter calibration is typically performed manually in a laboratory. However, this method has revealed numerous problems in mass production, necessitating an efficient, consistent, and repeatable automated calibration solution to ensure the quality of every product leaving the factory.

[0003] Currently, the camera extrinsic parameter calibration techniques related to this invention mainly include:

[0004] Manual calibration method: This method involves an operator holding a calibration board and moving it at multiple different angles and positions in front of the robot. The robot's camera captures images, and then software toolboxes such as OpenCV or MATLAB are used for calibration calculations. The main drawback of this approach is its extremely low calibration efficiency, typically exceeding 10 minutes per robot, severely impacting production line cycle time. Furthermore, the uncertainty introduced by manual operation leads to poor consistency in calibration results across different robots, and the high labor costs make it unsuitable for mass production.

[0005] The SLAM-based online self-calibration method utilizes the robot's perception data during actual operation and the SLAM algorithm to estimate the relative pose of the camera and the robot chassis online. Its problem lies in the uncontrollable calibration process and insufficient reliability of the calibration results, making it unsuitable as a rigorous basis for factory quality inspection.

[0006] High-precision motion capture system-assisted calibration method: This approach uses professional optical motion capture systems such as Vicon and OptiTrack to accurately acquire the robot's true pose, and then combines this with images captured by cameras for calibration. Although this method is highly accurate, its equipment costs are extremely high, and the system maintenance is complex, limiting its application to the R&D and verification stage and making it uneconomical to deploy on production lines.

[0007] In summary, when applied to mass production scenarios, existing technologies generally suffer from one or more technical problems, such as low calibration efficiency, poor result consistency, high equipment or labor costs, low automation, and difficulty in quality traceability due to the lack of systematic recording of calibration data. Summary of the Invention

[0008] This invention aims to solve the technical problems of high cost, low efficiency, poor consistency, low degree of automation, and difficulty in quality traceability in the existing technology for calibrating the external parameters of vision sensors during the mass production stage of wheeled robots.

[0009] To address the aforementioned problems, this invention provides a mass production calibration method for the extrinsic parameters of a wheeled robot's vision sensor. The method includes: within a pre-defined calibration room containing multiple visual targets with fixed spatial positions, the three-dimensional coordinates of the visual targets in a pre-defined world coordinate system are known; using mechanical fixtures, moving and positioning the wheeled robot to be calibrated to a pre-defined static calibration position, such that the robot's body coordinate system coincides with the world coordinate system or has a pre-defined fixed transformation relationship; while the robot remains in the static calibration position, controlling one or more of its vision sensors to acquire images containing the one or more visual targets; detecting the acquired images to obtain the two-dimensional pixel coordinates of the visual targets in the images; and based on the two-dimensional pixel coordinates and the corresponding known three-dimensional coordinates of the visual targets, solving for the extrinsic parameters of the vision sensor relative to the world coordinate system using the SlovePnP algorithm, thereby determining its extrinsic parameters relative to the robot's body coordinate system.

[0010] As a preferred option, in order to ensure the accuracy and reliability of positioning, the mechanical tooling uses a limit switch to confirm that the robot has accurately reached the static calibration position.

[0011] As a preferred solution, in order to achieve automated management and quality traceability of calibration data, after the extrinsic parameters are solved, the calibrated extrinsic parameters are automatically written into the robot system and uploaded to the background database management system.

[0012] As a preferred option, in order to quantitatively evaluate the accuracy of the calibration results, the calibration results are verified by calculating the reprojection error. The reprojection error is the error between the two-dimensional projected coordinates obtained by projecting the three-dimensional coordinates back to the image based on the solved extrinsic parameters and the two-dimensional pixel coordinates obtained by actual detection.

[0013] As a preferred option, in order to ensure the stability and robustness of the automated process, the robot is also subjected to a state self-check before image acquisition to determine whether it has been successfully positioned, whether it is in a stationary state, and whether the vision sensor can successfully capture data.

[0014] As a preferred option, in order to improve the accuracy and efficiency of target recognition, the visual target is preferably an AprilTag QR code pattern.

[0015] As a preferred option, in order to accommodate robots with complex camera layouts, the visual targets in the calibration room are designed to be layered and distributed in a circular array around the static calibration position and on the ground.

[0016] As a preferred embodiment, the present invention also provides a mass production calibration system for the extrinsic parameters of a wheeled robot's vision sensor. This system includes: a calibration chamber containing multiple visual targets with fixed spatial positions, the three-dimensional coordinates of which are known in a preset world coordinate system; a set of mechanical fixtures for moving and positioning the wheeled robot to be calibrated to a preset static calibration position within the calibration chamber; and a control terminal for controlling the robot's vision sensor to acquire images at the static calibration position and run calibration software; the calibration software is configured to perform the extrinsic parameter solving steps in the above method.

[0017] Compared with the prior art, the present invention has the following beneficial effects:

[0018] 1. Significantly reduce calibration costs: Compared to solutions using high-precision motion capture systems, it can save more than 90% of equipment investment.

[0019] 2. Significantly improve calibration efficiency: The fully automated process reduces the calibration time of a single robot to less than 3 minutes, meeting the needs of high-speed mass production.

[0020] 3. Ensure calibration consistency: Mechanical precision positioning eliminates errors introduced by human operation, ensuring that the external parameter deviation of the products leaving the factory is controlled within the precise range of ±3mm / ±0.5°.

[0021] 4. Achieve full-process automation: Supports unmanned calibration, reducing reliance on manpower on the production line.

[0022] 5. Supports quality traceability: Calibration data is automatically uploaded to the backend database, facilitating subsequent product quality analysis and recall management. Attached Figure Description

[0023] Figure 1 This is a block diagram of the overall composition of a calibration system according to an embodiment of the present invention.

[0024] Figure 2 This is a schematic diagram showing the coordinate system position distribution of the robot body and the camera according to an embodiment of the present invention.

[0025] Figure 3 This is a schematic diagram of the target space layout in a calibration room according to an embodiment of the present invention.

[0026] Figure 4 This is a diagram showing the shape and dimensions of a ground layer calibration pattern according to an embodiment of the present invention.

[0027] Figure 5 This is a shape and size diagram of a spatial layer calibration pattern according to an embodiment of the present invention.

[0028] Figure 6a The image is a target image captured in a simulated environment by camera number 3 according to an embodiment of the present invention.

[0029] Figure 6b The image is a target image captured in a simulated environment by camera number 5 according to an embodiment of the present invention.

[0030] Figure 6c The image is a target image captured in a simulated environment by camera number 4 according to an embodiment of the present invention.

[0031] Figure 6d The image is a target image captured in a simulated environment by camera number 2 according to an embodiment of the present invention.

[0032] Figure 7 This is a flowchart of robot production line calibration according to an embodiment of the present invention.

[0033] Figure 8 This is a flowchart of a calibration software algorithm according to an embodiment of the present invention. Detailed Implementation

[0034] To make the objectives, technical solutions, and advantages of the present invention clearer, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0035] System construction and working principle: Refer to Figure 1The calibration system in this embodiment mainly includes a human-machine interface (HMI), a calibration room main controller, a robot controller, a background database management system, and a mechanical drive device within the calibration room. The operator sends a calibration start signal via the HMI. Upon receiving the signal, the calibration room main controller controls the mechanical drive device (i.e., mechanical fixtures) to move the robot and the movable calibration pattern to their respective preset designated positions. This process is precisely controlled by limit switches. When the controller receives a limit switch trigger signal, it confirms that the robot has been precisely delivered to the static calibration position, and its body coordinate system is aligned with the world coordinate system of the calibration room. Subsequently, the calibration room main controller establishes communication with the robot controller (in this embodiment, this is done via Wi-Fi, and data publishing and subscription are based on the ROS2 framework), initiating the calibration software process. After the calibration software completes its calculations, it synchronizes the results to the background database management system and feeds back the status to the main controller. The main controller then controls the drive device to reset the robot and the calibration pattern.

[0036] Robot and camera layout: Reference Figure 2 In this embodiment, the wheeled robot 1 is equipped with four cameras. Cameras 2 and 4 are RGB cameras, and cameras 3 and 5 are RGBD cameras. Cameras 3, 4, and 5 face forward of the robot, while camera 2 faces backward. Cameras 2 and 5 have a large pitch angle. The robot body also has a horizontal axis of rotation, allowing it to rotate freely in the horizontal direction while carrying the sensors.

[0037] Calibration room layout and target design: To ensure that all cameras of the robot are covered by the calibration pattern when in a static calibration position, the calibration room was specially designed. (Refer to...) Figure 3 The calibration room is set up with three layers of calibration patterns based on the robot's orientation 11: the ground layer 12, the middle layer 13, and the top layer 14. Each layer has eight calibration patterns evenly distributed, with adjacent patterns at an angle of 45 degrees.

[0038] Ground layer calibration pattern 12a: as shown Figure 4 As shown, it consists of nine AprilTag QR codes (type 36h11) with sides of 140mm each, spaced 20mm apart, and an overall size of [missing information]. .

[0039] Spatial layer calibration patterns 13a and 14a: as shown Figure 5 As shown, it consists of 20 AprilTag QR codes, each with a side length of 100mm and a spacing of 20mm. The overall size is [size missing]. .

[0040] The three-dimensional coordinates of all QR code feature points on the calibration pattern were accurately measured using specialized measuring equipment during the calibration room construction phase, with an accuracy superior to [previous measurement]. The example spatial locations of the center of each layer's calibration pattern are shown in the table below. .

[0041] Automated calibration process: such as Figure 7 As shown, the entire calibration process is executed automatically without human intervention.

[0042] Preparation and Self-Check: The main controller reads the robot ID and controls the robot to enter the static calibration position. The robot controller performs a status self-check to determine whether the positioning was successful, whether the robot is stationary, and whether the camera can successfully capture data.

[0043] Start the calibration process: After the self-test passes, start the calibration process and control the camera to acquire still images.

[0044] Result Acquisition and Judgment: The calibration software performs calculations and returns results within a preset time (e.g., 20 seconds). If the timeout occurs or the process fails, a fault code is reported, and the software will retry after manual intervention.

[0045] Completion and Data Upload: If calibration is successful, the external parameter results are read, written into the robot system, and uploaded to the backend database management system.

[0046] Extrinsic parameter solution algorithm flow:

[0047] like Figure 8 As shown, the core algorithm flow of the calibration software is as follows: Parameter loading: Load the camera's intrinsic parameters, distortion coefficients, and the three-dimensional coordinates of the pre-measured calibration interval QR code feature points. .

[0048] Image processing and feature detection: After acquiring still images from the camera and removing distortion, a QR code detection algorithm is run to accurately identify the two-dimensional pixel coordinates of each QR code corner. .

[0049] Solving extrinsic parameters: based on the constructed 3D-2D point correspondence. The extrinsic parameters [RT] of the camera relative to the world coordinate system are obtained using the SlovePnp algorithm. Since the robot's body coordinate system and the world coordinate system have been aligned using mechanical tooling, these extrinsic parameters are either the same as or can be obtained through a simple transformation.

[0050] Accuracy Verification: To measure the calibration accuracy, the 3D point Pw is reprojected back onto the pixel plane according to formula (1) and the obtained extrinsic parameters to obtain the projected point (u, v). Then, according to formula (2), the difference between the projected point and the actual detection point is calculated. The root mean square error (i.e., reprojection error) between the two values ​​is determined, and it is determined whether the error is less than a preset threshold.

[0051] Formula (1):

[0052] Formula (2):

[0053] Simulation Verification: To verify the feasibility of the above calibration room design scheme, simulation tests were conducted in the software. For example... Figures 6a to 6d As shown, the simulation results indicate that all cameras can capture sufficiently clear and high-quality QR code feature points, proving that the current calibration room design scheme is reasonable.

[0054] Through the above-described system and method, this invention successfully transforms a complex sensor calibration task into a standardized production process that is low-cost, high-efficiency, highly consistent, fully automated, and with traceable results, greatly solving the application bottleneck of existing technologies in mass production scenarios.

[0055] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for mass production calibration of extrinsic parameters of a vision sensor for a wheeled robot, characterized in that, Includes the following steps: a) Environmental preparation and precise positioning: In a pre-set calibration room containing multiple visual targets with fixed spatial positions, the three-dimensional coordinates of the visual targets in a pre-set world coordinate system are known; Using mechanical tooling, the wheeled robot to be calibrated is moved and positioned to a preset static calibration position, so that the robot's body coordinate system coincides with the world coordinate system or has a preset fixed transformation relationship. b) Static image acquisition: While the robot is held in the static calibration position, one or more of its vision sensors are controlled to acquire images containing the one or more visual targets; c) Extrinsic parameter solution: Detect the acquired image to obtain the two-dimensional pixel coordinates of the visual target in the image; based on the two-dimensional pixel coordinates and the corresponding known three-dimensional coordinates of the visual target, solve the extrinsic parameters of the visual sensor relative to the world coordinate system using the SlovePnP algorithm, and thereby determine its extrinsic parameters relative to the robot body coordinate system.

2. The method according to claim 1, characterized in that, In step a), the mechanical tooling confirms that the robot has reached the static calibration position by using a limit switch.

3. The method according to claim 1, characterized in that, The method further includes the following steps: d) Result uploading and traceability: After the extrinsic parameter solution is completed, the calibrated extrinsic parameters are automatically written into the robot system and uploaded to the background database management system for quality archiving and traceability.

4. The method according to claim 1, characterized in that, In step c), the accuracy of the calibration result is evaluated by calculating the reprojection error, which is the error between the two-dimensional projected coordinates obtained by projecting the three-dimensional coordinates back to the image based on the solved extrinsic parameters and the two-dimensional pixel coordinates obtained by detection.

5. The method according to claim 1, characterized in that, Before step b), the method further includes: performing a state self-check on the robot to determine whether the robot has been successfully positioned, whether it is in a stationary state, and whether the vision sensor can successfully capture data. Subsequent steps are only executed after the self-check passes.

6. The method according to claim 1, characterized in that, The visual target is the AprilTag QR code pattern.

7. The method according to claim 1, characterized in that, The visual targets within the calibration chamber are layered and arranged in a circular array around the static calibration position and on the ground.

8. A mass production calibration system for the extrinsic parameters of a vision sensor for a wheeled robot, characterized in that, include: A calibration room contains multiple visual targets with fixed spatial positions, and the three-dimensional coordinates of the visual targets in a preset world coordinate system are known. A set of mechanical tooling is used to move and position the wheeled robot to be calibrated to a preset static calibration position within the calibration room; A control terminal is used to control the robot's vision sensors to acquire images at the static calibration position and to run calibration software; The calibration software is configured to: detect the acquired image to obtain the two-dimensional pixel coordinates of the visual target, and solve the extrinsic parameters of the visual sensor based on the two-dimensional pixel coordinates and their corresponding known three-dimensional coordinates using the SlovePnP algorithm.

Images