58 results about "Extrinsic calibration" patented technology

In various examples, head pose prediction for automotive occupant sensing systems and applications is presented. The systems and methods described herein provide for a machine learning model trained using a dataset that comprises ground truth head pose data computed using a registered head model of a training subject. While operating a vehicle, one or more cameras and a depth sensor capture synchronized images of the training subject. To compute a ground truth 3D head pose, angular deviations between a 3D point cloud and the registered head model may be computed to obtain a 3D ground truth head pose measurement. Using an extrinsic calibration transform, the head pose measurement may be mapped into the sensor coordinate frame. Training samples may be produced for training the machine learning model that comprise an optical image frame and the head pose measurement transposed into the frame of reference for that optical image frame.

The invention provides a camera external parameter calibration method and device based on image and point cloud matching, and the technical scheme of the invention is that a point cloud rendering view corresponding to an original point cloud is rotated, so that the point cloud rendering view and a camera image are superposed visually, and initial space association is established; and directly learning and solving accurate camera external parameters from the local relevance between the original point cloud intensity information and the image RGB information in an end-to-end manner by using a deep learning model. According to the scheme, the strict camera-radar orientation consistency requirement in a traditional method is not needed through rough matching, manual participation in the whole process of traditional calibration is replaced, transition from visual alignment to geometric alignment is achieved, and the calibration efficiency and scene adaptability are remarkably improved. Meanwhile, the scheme of the invention initiates a process of visual field cone cutting-perspective projection rasterization-intensity-RGB end-to-end matching, and compared with a traditional feature point method, the calculation complexity is greatly reduced, and the memory occupation is reduced by 80%.

The invention relates to a laser radar camera calibration method and device, and a medium. The method comprises the steps: collecting multi-frame laser radar point cloud data and synchronous corresponding image data in a construction scene; carrying out multi-frame point cloud fusion and dense reconstruction to generate a laser dense point cloud; visual sparse point cloud reconstruction is carried out, and cross-modal scale unification and space alignment are carried out on an initial visual point cloud obtained through reconstruction and the laser dense point cloud; generating a visual dense point cloud through three-dimensional Gaussian splashing; and performing registration on the laser dense point cloud and the visual dense point cloud by adopting a point-to-line iterative nearest point algorithm, and performing calculation to obtain an external parameter calibration matrix of the camera and the laser radar. Compared with the prior art, the method has the advantages of high precision, low cost, high stability and the like.

A system is disclosed for providing extrinsic calibration of a camera to a relative working environment of a programmable motion device that includes an end-effector. The system includes a fiducial located at or near the end-effector, at least one camera system for viewing the fiducial as the programmable motion device moves in at least three degrees of freedom, and for capturing a plurality of images containing the fiducial, and a calibration system for analyzing the plurality of images to determine a fiducial location with respect to the camera to permit calibration of the camera with the programmable motion device.

Examples of the present disclosure relate to a computer-implemented method, an apparatus, and a computer program for extrinsic calibration of one or more cameras with respect to a reference coordinate system, and to various methods, apparatuses, and computer programs for using information on a pose of a reference plane being determined as part of the extrinsic calibration. The computer-implemented method for extrinsic calibration of one or more cameras with respect to a reference coordinate system comprises obtaining one or more images from one or more cameras, with each camera having a field of view, wherein each image shows an observation of at least one marker of a plurality of markers, wherein the plurality of markers are co-planar with respect to a reference plane, estimating, for the one or more cameras, a transformation between a respective camera coordinate system and the reference coordinate system based on an estimated pose of the at least one marker observed in the field of view of the camera relative to the camera coordinate system, estimating a pose of the reference plane with respect to the respective one or more camera coordinate systems based on a pre-defined or estimated pose of the reference plane with respect to the reference coordinate system and based on the estimated transformations between the one or more camera coordinate systems and the reference coordinate system, estimating planar poses of the plurality of markers with respect to the reference plane, and simultaneously adjusting the planar poses of the plurality of markers and the pose of the reference plane with respect to the respective one or more camera coordinate systems by iteratively reducing an error between a reprojection of the plurality of markers into the respective one or more camera coordinate systems and the position of the respective markers in the images.

The invention discloses a radar camera external parameter calibration method and system based on 3D Gaussian sputtering, and the method comprises the steps: enabling a laser radar and a camera system to stand, and accumulating laser radar point clouds of a collection scene; secondly, carrying out voxel segmentation on the point cloud density according to the point cloud density, constructing a covariance and a mean value of an initial Gaussian ellipsoid, expressing a three-dimensional scene by using a 3D Gaussian ellipsoid, and obtaining a rendering result; then, constructing a covariance change loss function and a total loss function, and performing coarse optimization and fine optimization on external parameters of the camera and the radar through back propagation of the loss functions; and finally, programming the calibration method, and finishing external parameter adjustment of the laser radar and the camera by using a computer processor. According to the invention, efficient and accurate external parameter calibration is carried out on the laser radar and the camera, the calibration precision of the camera and the radar in a non-structural environment can be effectively improved, and the problem of external parameter calibration in multi-sensor data fusion can be solved.

System, methods, and computer-readable media for extrinsically calibrating sensors on an autonomous vehicle (AV) by leveraging map data. By leveraging offline data, such as the map data on the AV, the extrinsic calibration of the sensors may be performed on road to update or check previous calibrations. Static objects are determined in the scene by leveraging a map with a semantic map layer and sensors are extrinsically calibrating using a portion of a set of static points associated with the determined static objects by determining relative positions and angles the sensors.

A method for calibrating a vehicle to infrastructure system using an infrared camera overlooking a road, comprises: obtaining from the camera a first image, each image of the camera having points identifiable in an image coordinate system and corresponding to objects in a real world scene; identifying in the first image a plurality of points each of which corresponds to a respective object that is touching the road's surface; obtaining a measurement that corresponds to a distance from a location of the camera to each respective one of the points in a camera coordinate system (CCS); determining a model of the road in the CCS based on the points and their obtained distances; and calculating a transformation between the CCS and a world coordinate system (WCS) using the road model, the transformation being usable to determine a distance in the WCS to an object in a second image.

The invention relates to a millimeter wave radar and laser radar external parameter calibration method, device, equipment and medium. The method comprises the following steps: performing data alignment on millimeter wave radar data and laser radar data to construct a data set; constructing an external parameter calibration matching model, performing key point extraction on the data set based on a preset loss function and the external parameter calibration matching model, and performing feature matching based on key points to obtain key point matching data; based on the key point matching data, distributing the key point matching data to adjacent areas by adopting a point-to-node strategy to obtain dense point matching data; and obtaining joint calibration parameters of the millimeter wave radar and the laser radar based on the dense point matching data. The method provided by the invention can maximize the effective information of the millimeter-wave radar and laser radar point clouds, solves the problems of sparse point clouds and multi-noise point clouds of the millimeter-wave radar, and improves the calibration precision and robustness.

The invention relates to the technical field of automatic control, and provides an external parameter calibration method of a sensor, a cleaning robot, a cleaning system, a computer readable storage medium and electronic device.The method comprises the steps that in the movement process of the cleaning robot, reference local sensing data of a reference sensor for each movement interval are obtained, and the reference local sensing data of the reference sensor for each movement interval are obtained; acquiring auxiliary local sensing data of each to-be-calibrated sensor for each motion interval; determining a first conversion matrix from each sensor to be calibrated to the reference sensor according to the reference local sensing data and the auxiliary local sensing data; obtaining a second conversion matrix from the reference sensor to the odometer coordinate system, and determining a candidate conversion matrix from each to-be-calibrated sensor to the odometer coordinate system according to the first conversion matrix and the second conversion matrix; and optimizing each candidate conversion matrix to obtain a final conversion matrix from each sensor to be calibrated to the odometer coordinate system. The external parameters of the sensor can be accurately calibrated at low cost.

The invention provides a vehicle-mounted camera external parameter calibration method, electronic equipment and a program product, and relates to the technical field of vehicles. The method comprises the following steps: when a camera external parameter dynamic calibration starting condition is satisfied, acquiring a road image sequence corresponding to each camera of a vehicle; each group of road image sequence comprises at least one image of a structured road and a corresponding acquisition moment; determining vanishing point positions and horizon line positions corresponding to the cameras by utilizing an end-to-end vanishing point / horizon line detection model based on each group of road image sequences; wherein the end-to-end vanishing point / horizon line detection model is used for outputting the vanishing point position in the road image sequence, or outputting the vanishing point position and the horizon line position in the road image sequence; and according to the vanishing point position and the horizon line position corresponding to each camera, external parameters of each camera are determined. According to the invention, an efficient and simple solution is provided for external parameter calibration of the vehicle-mounted camera.

The present disclosure provides a camera calibration method, device, electronic device and readable storage medium, which relate to technical fields such as autonomous driving and intelligent transportation. The camera calibration method includes: obtaining the camera intrinsic parameters of the camera to be calibrated; obtaining a detection image sequence based on at least one driving image taken by the camera to be calibrated; performing feature extraction on at least one driving image in the detection image sequence to obtain a target feature set; performing extrinsic calibration on the camera to be calibrated based on the camera height of the camera to be calibrated, the camera intrinsic parameters and the target feature set to obtain the camera extrinsic parameters of the camera to be calibrated. The present disclosure can simplify the steps of camera extrinsic calibration, expand the scope of application of camera extrinsic calibration, and improve the efficiency and accuracy of camera extrinsic calibration.

A tracking system and method are provided. A movable element of the tracking system performs a motion process on a movable path. A tracking device of the tracking system calculates a signal change at each of time points corresponding to the motion process based on first sensing signals and second sensing signals. The tracking device generates a variable extrinsic calibration parameter of a target device and the movable element corresponding to each of the time points based on the signal changes and the fixed motion relationship. The tracking device performs a tracking operation corresponding to the target device based on the variable extrinsic calibration parameters.

The invention provides a camera and laser radar combined external parameter calibration method based on a single calibration board. The method employs a novel algorithm to jointly estimate extrinsic parameters of both at least one camera and at least one lidar through a single calibration board. More specifically, the proposed novel calibration board is compact and has different calibration targets for both the camera and the lidar sensor. The proposed algorithm can efficiently and accurately detect the calibration target from the camera and the laser radar. Thus, extrinsic parameters of the at least one camera and the at least one lidar may be jointly estimated based on the detected calibration target. Due to the novel design of the calibration board and algorithm, the calibration setting is simple and fast, and the calibration capability is stable and accurate.

The application discloses a kind of vehicle camera extrinsic calibration methods based on control point and positioning information, comprising steps S1: with the center of rear axle of vehicle as origin establishes vehicle three-dimensional coordinate system, records the coordinate information of all vehicle cameras in vehicle three-dimensional coordinate system in vehicle;Step S2: select calibration area on ground, layout control point with different number in calibration area, record the position information of each control point on ground;Step S3: vehicle advances to calibration area, records the vehicle position information in vehicle advancing process by positioning system;Step S4: screen out all the images containing control point, based on number, obtain the position information of control point on ground in image, based on the position of vehicle when shooting image, convert the position information of control point on ground into coordinate information under vehicle three-dimensional coordinate system, according to the coordinate information of vehicle camera under vehicle three-dimensional coordinate system and the coordinate information of control point, calculate the extrinsic calibration of vehicle camera.

In various examples, head pose prediction for automotive occupant sensing systems and applications is disclosed. The systems and methods described herein provide a machine learning model trained using a dataset comprising ground-truth head pose data computed using a recorded head model of a training subject. While driving a vehicle, one or more cameras and a depth sensor capture synchronized images of the training subject. To compute a ground-truth 3D head pose, angular deviations between a 3D point cloud and the recorded head model may be calculated to obtain a 3D ground-truth head pose measurement. Using an extrinsic calibration transform, the head pose measurement may be mapped into the sensor coordinate frame.Training samples can be generated to train the machine learning model, which include an optical image frame and the head pose measurement transposed into the reference frame for that optical image frame.

A system performs pose calibration between a LiDAR sensor and a camera. The system may receive an image frame captured by the camera mounted on a device. The system may receive a point cloud captured by the LiDAR sensor mounted on the device, the LiDAR sensor having an overlapping field of view with the camera. The system may identify markers of a calibration target captured in the image frame by applying a feature identification model to the image frame. The system may identify the markers of the calibration target captured in the point cloud by: clustering points in the point cloud into one or more planes, selecting one of the planes based on sizes of the planes; and identifying holes in the selected plane as the markers of the calibration target. The system may determine a pose transformation between the camera and the LiDAR sensor based on information identifying the markers from the image frame and information identifying the markers from the point cloud.

This application provides a functional calibration plate and an external parameter calibration system, which can improve the calibration accuracy of external parameters while reducing the complexity of calibration operations and improving the requirements for the calibration environment. The functional calibration plate includes: a calibration fixture; and one or more reflective spheres, wherein the reflective spheres are fixed to the calibration fixture and protrude from the surface of the calibration fixture; the reflectivity of the reflective spheres is greater than or equal to 60%.

The present invention discloses a camera extrinsic parameter calibration method for hybrid display, which comprises the following steps: S11, obtaining an intrinsic parameter matrix of a camera by an intrinsic parameter calibration method; S12, starting an external locator; S13, controlling LED lamp beads on a screen, treating bright spots as feature markers, lighting up M of them, capturing images N times at different positions, and recording the output of the locator at that moment; S14, listing a reprojection residual equation of the extrinsic parameter coordinate system deviation based on the captured feature markers and combining them with the spatial posture of the locator; S15, iteratively obtaining the optimal extrinsic parameter coordinate system deviation by minimizing the reprojection residual; compared with the traditional calibration method, the present invention does not require the use of a rangefinder for measurement, while taking into account the offset between the locator origin and the screen coordinate system, and sampling multiple sets of positioning data synchronized with the camera and the locator to simultaneously solve the camera and locator extrinsic parameters and the locator origin and screen offset.

In accordance with one aspect of the inventive concepts, provided is an autonomous mobile robot, comprising: at least one processor in communication with at least one computer memory device; at least one vehicle-mounted sensor configured to acquire measurement data of at least one feature of the robot; and an extrinsic calibration system comprising computer program code executable by the at least one processor to register the measurement data to a common coordinate frame to perform extrinsic calibration of the at least one sensor. The measurement data can comprise point cloud data.

A motion observability quantification method for LiDAR-Inertial Navigation (LIDAR) calibration relates to the field of LiDAR-IMU data fusion technology. Data is collected and the poses of the IMU and LiDAR are calculated. Within the pose sequence, pose pairs at the same acquisition time are searched for to construct an extrinsic calibration problem. The rotation matrix between the LiDAR and IMU coordinate systems is solved using an optimized approach by considering multiple sets of measurements. All calibration data is divided into segments of equal length. An information matrix is ​​constructed for each segment based on the calibration problem. Singular value decomposition is performed on the information matrix, and the information index η of each segment is calculated. The minimum singular value is used to represent the information metric of a single segment. A threshold η0 is set, and segments with η ≥ η0 are selected for extrinsic calibration. This method reduces the interference of invalid and erroneous information, improving the convergence and robustness of the calibration algorithm while reducing computational cost.

The present invention discloses a method for calibrating external parameters of a visual sensor based on a stereo target. The method comprises the following steps: a mounting base is provided with at least two cylindrical protrusions, and a visual sensor is provided with at least two reference holes; the two protrusions are plugged together during installation; a housing coordinate system of the visual sensor is established using the reference holes or the cylindrical protrusions; the visual sensor is mounted on the calibration fixture in the laboratory, a two-dimensional image of the stereo target in the field of view is collected, and the conversion relationship between the camera coordinate system and the housing coordinate system is obtained by combining the two-dimensional image and the standard coordinates of the standard holes / spheres on the stereo target in the housing coordinate system; at a detection station, the visual sensor is mounted on the mounting base; a conversion relationship between the coordinate system of the object to be measured and the housing coordinate system is obtained using a standard instrument; and the conversion relationship between the coordinate system of the object to be measured and the camera coordinate system is obtained by combining the conversion relationship between the camera coordinate system and the housing coordinate system, thereby completing the external parameter calibration. The method can accurately obtain the desired value, is simple to operate, and shortens the time of the external parameter calibration process.

The invention relates to a method for the extrinsic calibration of an interior camera in a vehicle, wherein the interior camera is first calibrated with respect to a reference coordinate system and the position of an interior light in the vehicle relative to the reference coordinate system is known. According to the invention, it is provided that: - a three-dimensional environmental model is created with respect to the vehicle based on signals acquired from an environmental sensor; - the interior light in a section is activated corresponding to a position of a three-dimensional object acquired in the environmental model; - a projection vector between a light point of the activated section of the interior light and the position of the acquired object is determined; - a corneal reflection image is determined based on acquired image data from the interior camera to be calibrated.- the determined projection vector is projected onto the corneal reflection image and a reference vector is determined, - a reflected light point of the ambient light activated relative to the object is detected in the corneal reflection image, - a deviation between the reflected light point and the reference vector is determined, - the deviation is evaluated with respect to a threshold value, and - if it is determined that the deviation exceeds the threshold value, extrinsic calibration parameters of the indoor camera are recalibrated.

Systems and methods for external calibration of an in-cabin camera of a vehicle are disclosed. In one example, a method for performing external calibration of a vehicle camera includes: receiving a plurality of images from a camera of a vehicle; detecting and describing a plurality of two-dimensional (2D) feature points of the image using the trained neural network; mapping the detected 2D feature points to three-dimensional (3D) feature points stored in a feature point library generated by using the trained neural network; estimating a set of external parameters of the camera using the mapped 3D feature points; and performing an external calibration based on the set of external parameters.

This invention relates to a multi-IMU extrinsic parameter calibration method, comprising: constructing an IMU output model; calibrating the random walks and noise of each IMU in the IMU array; recording IMU array data; arbitrarily selecting one IMU from the IMU array as a reference IMU, using the coordinate system of the reference IMU as the reference coordinate system, establishing and solving a nonlinear least squares problem about rotation quaternions between the reference IMU and the other IMUs to be calibrated; generating virtual angular velocity observations for the other IMUs to be calibrated based on the calibrated relative rotation extrinsic parameters; and establishing and solving a nonlinear least squares problem about translation between the reference IMU and the other IMUs to be calibrated based on the virtual angular velocity observations of the other IMUs. This invention does not rely on the environment or external sensors and can complete calibration quickly.

Several examples provide navigation-based extrinsic calibration for multi-camera surveillance systems and applications. Image data is acquired by capturing images from a mobile calibration platform as the platform moves through an environment. Simultaneously, the mobile platform acquires navigation aid data (based on images of reference markers and / or received wireless navigation signals) representing its position in 3D space. A multi-camera calibration technology performs optimization based on a composite fusion of different sets of navigation aid data, which are used as optimization constraints to compute an extrinsic calibration transformation for individual sensors among a multitude of optical sensors distributed throughout an environment.