LiDAR-Camera Extrinsic Calibration Using Random Marker Boards

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Current methods for extrinsic calibration of LiDAR and camera sensors in autonomous machines are cumbersome, requiring precise placement of calibration targets and relying on intensity information from LiDAR, which is not always accurate, and are prone to errors in detecting geometric shapes.

Innovation Solution

The method involves using simple symmetric geometric shapes like Aruco or Charuco boards placed randomly within the sensors' field of view, allowing for detection of markers by cameras and geometric fitting in LiDAR point clouds to compute a six-degree-of-freedom transformation without relying on intensity information or precise target placement.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If a standard checkerboard is used for calibration, then the coordinate systems can be aligned, but the setup becomes complicated requiring precise placement of the checkerboard

Engineering Contradiction:

Improvecoordinate system alignment accuracyVSAvoidcalibration setup complexity

Core Design Contradiction:

Measurement precisionVSEase of operation

Solution Approach 1:

The patent uses a disposable calibration board with printed Aruco markers instead of a reusable checkerboard. The calibration board can be easily manufactured and discarded after use, eliminating the need for precise placement and complex setup procedures while maintaining calibration accuracy

Inventive Principle:

Principle #27Cheap short-living objects (Disposable)

Solution Approach 2:

The calibration board uses high-contrast black and white Aruco markers that are easily detectable by the camera system. The distinct color patterns enable reliable marker detection and positioning without requiring precise manual placement, simplifying the calibration setup

Inventive Principle:

Principle #32Color changes

2Measurement precision

If LiDAR intensity information is used for checkerboard detection, then the intersection points can be extracted, but the method fails when LiDAR does not provide accurate intensity information

Engineering Contradiction:

Improveintersection point extraction accuracyVSAvoidcalibration robustness

Core Design Contradiction:

Measurement precisionVSReliability

Solution Approach 1:

The patent introduces Aruco markers as an intermediary element that bridges LiDAR and camera data. The markers provide geometric features that can be detected by both sensors independently, eliminating reliance on LiDAR intensity information while enabling accurate coordinate system alignment

Inventive Principle:

Principle #24Intermediary (Mediator)

Solution Approach 2:

The calibration board is segmented into multiple Aruco markers with distinct geometric patterns. Each marker can be independently detected and processed, allowing the system to extract position information from geometric shape rather than intensity, improving reliability when intensity data is unavailable

Inventive Principle:

Principle #1Segmentation

3Ease of operation

If RANSAC algorithm is used to extract rectangle from LiDAR point cloud, then the rectangular board can be detected, but the boundary detection is prone to errors

Engineering Contradiction:

Improveautomatic boundary extractionVSAvoidboundary detection accuracy

Core Design Contradiction:

Ease of operationVSMeasurement precision

Solution Approach 1:

The calibration board is pre-designed with Aruco markers that have known geometric properties and dimensions. This preliminary design allows the system to use the known marker geometry to guide the detection process, improving boundary extraction accuracy compared to generic RANSAC methods

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The patent replaces the mechanical/geometric RANSAC approach with a marker-based detection system. The Aruco markers provide explicit geometric cues that can be detected more accurately than generic shape extraction, reducing boundary detection errors while maintaining automation

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

4Ease of operation

If a single Aruco marker is used, then the center position can be estimated, but the 3D position estimation and surface normal are inaccurate

Engineering Contradiction:

Improvesimple target structureVSAvoid3D position estimation accuracy

Core Design Contradiction:

Ease of operationVSMeasurement precision

Solution Approach 1:

The patent merges multiple Aruco markers onto a single calibration board to form a composite target structure. This combination allows the system to estimate both the center position and surface normal of the entire board by analyzing the spatial relationship between multiple markers, improving 3D position estimation accuracy while maintaining a simple overall target structure

Inventive Principle:

Principle #5Merging (Combining)

Data Source

PatentUS11520024B2Automatic autonomous vehicle and robot LiDAR-camera extrinsic calibration

Publication Date: 2022.12.06 NIO TECH ANHUI CO LTD

AI summary

Extrinsic calibration of a Light Detection and Ranging (LiDAR) sensor and a camera can comprise constructing a first plurality of reconstructed calibration targets in a three-dimensional space based on physical calibration targets detected from input from the LiDAR and a second plurality of reconstructed calibration targets in the three-dimensional space based on physical calibration targets detected from input from the camera. Reconstructed calibration targets in the first and second plurality of reconstructed calibration targets can be matched and a six-degree of freedom rigid body transformation of the LiDAR and camera can be computed based on the matched reconstructed calibration targets. A projection of the LiDAR to the camera can be computed based on the computed six-degree of freedom rigid body transformation.