84 results about "Extrinsic calibration" patented technology

Systems and techniques are provided for calibrating cameras in a real space for tracking puts and takes of items by subjects. The method includes first processing one or more selected images captured by an extrinsic calibration tool to extract from the images, a three-dimensional (3D) point cloud of points corresponding to displays or structures that remain substantially immobile in the real space, and a second processing of a second set of images captured by a camera installation in the real space to match points in the second set of images to the point cloud. Differences in position of matching points can be used to determine transformation information therebetween, which can be used to calibrate the installed cameras.

In various examples, head pose prediction for automotive occupant sensing systems and applications is presented. The systems and methods described herein provide for a machine learning model trained using a dataset that comprises ground truth head pose data computed using a registered head model of a training subject. While operating a vehicle, one or more cameras and a depth sensor capture synchronized images of the training subject. To compute a ground truth 3D head pose, angular deviations between a 3D point cloud and the registered head model may be computed to obtain a 3D ground truth head pose measurement. Using an extrinsic calibration transform, the head pose measurement may be mapped into the sensor coordinate frame. Training samples may be produced for training the machine learning model that comprise an optical image frame and the head pose measurement transposed into the frame of reference for that optical image frame.

The invention provides a camera external parameter calibration method and device based on image and point cloud matching, and the technical scheme of the invention is that a point cloud rendering view corresponding to an original point cloud is rotated, so that the point cloud rendering view and a camera image are superposed visually, and initial space association is established; and directly learning and solving accurate camera external parameters from the local relevance between the original point cloud intensity information and the image RGB information in an end-to-end manner by using a deep learning model. According to the scheme, the strict camera-radar orientation consistency requirement in a traditional method is not needed through rough matching, manual participation in the whole process of traditional calibration is replaced, transition from visual alignment to geometric alignment is achieved, and the calibration efficiency and scene adaptability are remarkably improved. Meanwhile, the scheme of the invention initiates a process of visual field cone cutting-perspective projection rasterization-intensity-RGB end-to-end matching, and compared with a traditional feature point method, the calculation complexity is greatly reduced, and the memory occupation is reduced by 80%.

Proposed is a multi-sensor fusion device for performing extrinsic calibration in real time. The multi-sensor fusion device may include a camera unit including at least one camera and configured to produce image information of a surrounding environment by using the camera. The multi-sensor fusion device may also include a light detection and ranging (LiDAR) unit including at least one LiDAR and configured to produce a point cloud of the surrounding environment by using the LiDAR; a data combination unit configured to produce combined data in which the image information is combined with the point cloud. The multi-sensor fusion device may further include a data computation unit configured to derive an extrinsic calibration parameter by inputting the combined data into an extrinsic calibration model and perform extrinsic calibration of the camera and the LiDAR based on the derived extrinsic calibration parameter.

The invention relates to a laser radar camera calibration method and device, and a medium. The method comprises the steps: collecting multi-frame laser radar point cloud data and synchronous corresponding image data in a construction scene; carrying out multi-frame point cloud fusion and dense reconstruction to generate a laser dense point cloud; visual sparse point cloud reconstruction is carried out, and cross-modal scale unification and space alignment are carried out on an initial visual point cloud obtained through reconstruction and the laser dense point cloud; generating a visual dense point cloud through three-dimensional Gaussian splashing; and performing registration on the laser dense point cloud and the visual dense point cloud by adopting a point-to-line iterative nearest point algorithm, and performing calculation to obtain an external parameter calibration matrix of the camera and the laser radar. Compared with the prior art, the method has the advantages of high precision, low cost, high stability and the like.

A system is disclosed for providing extrinsic calibration of a camera to a relative working environment of a programmable motion device that includes an end-effector. The system includes a fiducial located at or near the end-effector, at least one camera system for viewing the fiducial as the programmable motion device moves in at least three degrees of freedom, and for capturing a plurality of images containing the fiducial, and a calibration system for analyzing the plurality of images to determine a fiducial location with respect to the camera to permit calibration of the camera with the programmable motion device.

The invention provides a camera external parameter calibration method, device and system, equipment and a storage medium. In one example, the method comprises the following steps: for any calibration site, performing reference frame image acquisition on the calibration site through a vehicle-mounted around-view camera of a calibration vehicle, and generating reference frame information corresponding to the reference frame image according to a detected marking instruction; and for any to-be-calibrated vehicle, acquiring reference frame information of the same calibration site according to the current calibration site, and performing external parameter calibration on the to-be-calibrated vehicle-mounted around-view camera according to the acquired reference frame information. According to the method, the universality of the calibration scheme can be improved under the condition that the calibration accuracy is ensured.

Examples of the present disclosure relate to a computer-implemented method, an apparatus, and a computer program for extrinsic calibration of one or more cameras with respect to a reference coordinate system, and to various methods, apparatuses, and computer programs for using information on a pose of a reference plane being determined as part of the extrinsic calibration. The computer-implemented method for extrinsic calibration of one or more cameras with respect to a reference coordinate system comprises obtaining one or more images from one or more cameras, with each camera having a field of view, wherein each image shows an observation of at least one marker of a plurality of markers, wherein the plurality of markers are co-planar with respect to a reference plane, estimating, for the one or more cameras, a transformation between a respective camera coordinate system and the reference coordinate system based on an estimated pose of the at least one marker observed in the field of view of the camera relative to the camera coordinate system, estimating a pose of the reference plane with respect to the respective one or more camera coordinate systems based on a pre-defined or estimated pose of the reference plane with respect to the reference coordinate system and based on the estimated transformations between the one or more camera coordinate systems and the reference coordinate system, estimating planar poses of the plurality of markers with respect to the reference plane, and simultaneously adjusting the planar poses of the plurality of markers and the pose of the reference plane with respect to the respective one or more camera coordinate systems by iteratively reducing an error between a reprojection of the plurality of markers into the respective one or more camera coordinate systems and the position of the respective markers in the images.

Systems and techniques are provided for calibrating cameras in a real space for tracking puts and takes of items by subjects. The method includes first processing one or more selected images captured by a first plurality of cameras to extract from the images, a three-dimensional (3D) point cloud of points, and a second processing of a second set of images captured by a second plurality of cameras in the real space to match points in the second set of images to the point cloud. Differences in position of matching points can be used to determine transformation information therebetween, which can be used to calibrate the second plurality of cameras.

The invention discloses a radar camera external parameter calibration method and system based on 3D Gaussian sputtering, and the method comprises the steps: enabling a laser radar and a camera system to stand, and accumulating laser radar point clouds of a collection scene; secondly, carrying out voxel segmentation on the point cloud density according to the point cloud density, constructing a covariance and a mean value of an initial Gaussian ellipsoid, expressing a three-dimensional scene by using a 3D Gaussian ellipsoid, and obtaining a rendering result; then, constructing a covariance change loss function and a total loss function, and performing coarse optimization and fine optimization on external parameters of the camera and the radar through back propagation of the loss functions; and finally, programming the calibration method, and finishing external parameter adjustment of the laser radar and the camera by using a computer processor. According to the invention, efficient and accurate external parameter calibration is carried out on the laser radar and the camera, the calibration precision of the camera and the radar in a non-structural environment can be effectively improved, and the problem of external parameter calibration in multi-sensor data fusion can be solved.

A system includes a camera capturing images in a first field of view of a sports ball bouncing and rolling, a storage arrangement, and a processing arrangement. The storage arrangement includes a three-dimensional 3D model of a part of a sports play area and the processing arrangement is configured to: detect a pixel location of a ball in an image; determine, based on intrinsic and extrinsic calibration parameters, a camera-ball line comprising a straight line passing through the camera in the direction of the sports ball and to determine, based on the 3D model, an intersection point of the camera-ball line with the 3D model. The processor outputs the intersection point as a 3D position of the ball in the image.

System, methods, and computer-readable media for extrinsically calibrating sensors on an autonomous vehicle (AV) by leveraging map data. By leveraging offline data, such as the map data on the AV, the extrinsic calibration of the sensors may be performed on road to update or check previous calibrations. Static objects are determined in the scene by leveraging a map with a semantic map layer and sensors are extrinsically calibrating using a portion of a set of static points associated with the determined static objects by determining relative positions and angles the sensors.

An extrinsic calibration parameter generating device, method, and non-transitory computer readable storage medium thereof are provided. The device calculates a base image capturing device pose based on the light-emitting points of a plurality of image frames. The device generates a first extrinsic calibration parameter corresponding to at least two image capturing devices based on the image frames, and the first extrinsic calibration parameter is a five-degree-of-freedom information. The device generates a second extrinsic calibration parameter corresponding to the at least two image capturing devices based on the base image capturing device pose and the first extrinsic calibration parameter, and the second extrinsic calibration parameter is six-degree-of-freedom information.

In accordance with one aspect of the inventive concepts, provided is an autonomous mobile robot, comprising: at least one processor in communication with at least one computer memory device: at least one vehicle-mounted sensor configured to acquire measurement data of at least one feature of the robot; and an extrinsic calibration system comprising computer program code executable by the at least one processor to register the measurement data to a common coordinate frame to perform extrinsic calibration of the at least one sensor. The measurement data can comprise point cloud data.

A method for calibrating a vehicle to infrastructure system using an infrared camera overlooking a road, comprises: obtaining from the camera a first image, each image of the camera having points identifiable in an image coordinate system and corresponding to objects in a real world scene; identifying in the first image a plurality of points each of which corresponds to a respective object that is touching the road's surface; obtaining a measurement that corresponds to a distance from a location of the camera to each respective one of the points in a camera coordinate system (CCS); determining a model of the road in the CCS based on the points and their obtained distances; and calculating a transformation between the CCS and a world coordinate system (WCS) using the road model, the transformation being usable to determine a distance in the WCS to an object in a second image.

Systems and methods for automatic multi-modality sensor calibration with near-infrared images (NIR). Image keypoints from collected images and NIR keypoints from NIR can be detected (110). A deep-learning-based neural network that learns relation graphs between the image keypoints and the NIR keypoints can match (120) the image keypoints and the NIR keypoints. Three dimensional (3D) points from 3D point cloud data can be filtered (130) based on corresponding 3D points from the NIR keypoints (NIR-to-3D points) to obtain filtered NIR-to-3D points. An extrinsic calibration can be optimized (140) based on a reprojection error computed from the filtered NIR-to-3D points to obtain an optimized extrinsic calibration for an autonomous entity control system. An entity can be controlled (150) by employing the optimized extrinsic calibration for the autonomous entity control system.

The invention relates to a millimeter wave radar and laser radar external parameter calibration method, device, equipment and medium. The method comprises the following steps: performing data alignment on millimeter wave radar data and laser radar data to construct a data set; constructing an external parameter calibration matching model, performing key point extraction on the data set based on a preset loss function and the external parameter calibration matching model, and performing feature matching based on key points to obtain key point matching data; based on the key point matching data, distributing the key point matching data to adjacent areas by adopting a point-to-node strategy to obtain dense point matching data; and obtaining joint calibration parameters of the millimeter wave radar and the laser radar based on the dense point matching data. The method provided by the invention can maximize the effective information of the millimeter-wave radar and laser radar point clouds, solves the problems of sparse point clouds and multi-noise point clouds of the millimeter-wave radar, and improves the calibration precision and robustness.

Method for validating a plurality of environment detection sensors rigidly connected to a vehicle (1), which are configured to detect a relative velocity (V1 to V7, Vn) of at least one object in the vicinity of the vehicle (1) relative to a sensor coordinate system (S1 to S7, Sn), and - in an extrinsic calibration for each sensor coordinate system (S1 to S7, Sn) a homogeneous coordinate transformation is determined to convert coordinates of the sensor coordinate system (S1 to S7, Sn) into coordinates of a vehicle coordinate system (V) rigidly connected to the vehicle (1), - based on homogeneous coordinate transformations ◯ For each relative velocity (V1 to V7, Vn), an object velocity (X1 to X7, Xn) related to the vehicle coordinate system (V) is determined and / or ◯ from the majority of the relative velocities (V1 to V7, Vn) at least one parameter of a motion model of the vehicle (1) is determined, characterized in that - a decalibrated state (C0) is assigned to the majority of environmental sensing sensors when the object velocities (X1 to X7, Xn) deviate from each other and / or from the motion model of the vehicle (1) by more than a predetermined amount and - Deviations of the object velocities (X1 to X7, Xn) are determined as pairwise vector differences and the maximum pairwise vector difference in magnitude and / or direction is compared with a predetermined magnitude difference or a predetermined angular difference.

The invention relates to the technical field of automatic control, and provides an external parameter calibration method of a sensor, a cleaning robot, a cleaning system, a computer readable storage medium and electronic device.The method comprises the steps that in the movement process of the cleaning robot, reference local sensing data of a reference sensor for each movement interval are obtained, and the reference local sensing data of the reference sensor for each movement interval are obtained; acquiring auxiliary local sensing data of each to-be-calibrated sensor for each motion interval; determining a first conversion matrix from each sensor to be calibrated to the reference sensor according to the reference local sensing data and the auxiliary local sensing data; obtaining a second conversion matrix from the reference sensor to the odometer coordinate system, and determining a candidate conversion matrix from each to-be-calibrated sensor to the odometer coordinate system according to the first conversion matrix and the second conversion matrix; and optimizing each candidate conversion matrix to obtain a final conversion matrix from each sensor to be calibrated to the odometer coordinate system. The external parameters of the sensor can be accurately calibrated at low cost.

The invention provides a vehicle-mounted camera external parameter calibration method, electronic equipment and a program product, and relates to the technical field of vehicles. The method comprises the following steps: when a camera external parameter dynamic calibration starting condition is satisfied, acquiring a road image sequence corresponding to each camera of a vehicle; each group of road image sequence comprises at least one image of a structured road and a corresponding acquisition moment; determining vanishing point positions and horizon line positions corresponding to the cameras by utilizing an end-to-end vanishing point / horizon line detection model based on each group of road image sequences; wherein the end-to-end vanishing point / horizon line detection model is used for outputting the vanishing point position in the road image sequence, or outputting the vanishing point position and the horizon line position in the road image sequence; and according to the vanishing point position and the horizon line position corresponding to each camera, external parameters of each camera are determined. According to the invention, an efficient and simple solution is provided for external parameter calibration of the vehicle-mounted camera.

Systems and methods for automatic multi-modality sensor calibration with near-infrared images (NIR). Image keypoints from collected images and NIR keypoints from NIR can be detected. A deep-learning-based neural network that learns relation graphs between the image keypoints and the NIR keypoints can match the image keypoints and the NIR keypoints. Three dimensional (3D) points from 3D point cloud data can be filtered based on corresponding 3D points from the NIR keypoints (NIR-to-3D points) to obtain filtered NIR-to-3D points. An extrinsic calibration can be optimized based on a reprojection error computed from the filtered NIR-to-3D points to obtain an optimized extrinsic calibration for an autonomous entity control system. An entity can be controlled by employing the optimized extrinsic calibration for the autonomous entity control system.

The present disclosure provides a camera calibration method, device, electronic device and readable storage medium, which relate to technical fields such as autonomous driving and intelligent transportation. The camera calibration method includes: obtaining the camera intrinsic parameters of the camera to be calibrated; obtaining a detection image sequence based on at least one driving image taken by the camera to be calibrated; performing feature extraction on at least one driving image in the detection image sequence to obtain a target feature set; performing extrinsic calibration on the camera to be calibrated based on the camera height of the camera to be calibrated, the camera intrinsic parameters and the target feature set to obtain the camera extrinsic parameters of the camera to be calibrated. The present disclosure can simplify the steps of camera extrinsic calibration, expand the scope of application of camera extrinsic calibration, and improve the efficiency and accuracy of camera extrinsic calibration.

A tracking system and method are provided. A movable element of the tracking system performs a motion process on a movable path. A tracking device of the tracking system calculates a signal change at each of time points corresponding to the motion process based on first sensing signals and second sensing signals. The tracking device generates a variable extrinsic calibration parameter of a target device and the movable element corresponding to each of the time points based on the signal changes and the fixed motion relationship. The tracking device performs a tracking operation corresponding to the target device based on the variable extrinsic calibration parameters.

The invention provides a camera and laser radar combined external parameter calibration method based on a single calibration board. The method employs a novel algorithm to jointly estimate extrinsic parameters of both at least one camera and at least one lidar through a single calibration board. More specifically, the proposed novel calibration board is compact and has different calibration targets for both the camera and the lidar sensor. The proposed algorithm can efficiently and accurately detect the calibration target from the camera and the laser radar. Thus, extrinsic parameters of the at least one camera and the at least one lidar may be jointly estimated based on the detected calibration target. Due to the novel design of the calibration board and algorithm, the calibration setting is simple and fast, and the calibration capability is stable and accurate.

The present invention provides a method and system (600) to calibrate a plurality of sensors with overlapping and non-overlapping field of views. More specifically, the present invention provides a reference sensor (610), configured to perform a step of generating a three-dimensional point cloud data of its surrounding environment with a first field of view and a second field of view, a plurality of source sensors (620a – 620d), configured to perform the step of capturing at least one depth image of its surrounding environment, and an extrinsic calibration module (640), configured perform a step calibrating the acquired three-dimensional point cloud data and the depth image data.

The application discloses a kind of vehicle camera extrinsic calibration methods based on control point and positioning information, comprising steps S1: with the center of rear axle of vehicle as origin establishes vehicle three-dimensional coordinate system, records the coordinate information of all vehicle cameras in vehicle three-dimensional coordinate system in vehicle;Step S2: select calibration area on ground, layout control point with different number in calibration area, record the position information of each control point on ground;Step S3: vehicle advances to calibration area, records the vehicle position information in vehicle advancing process by positioning system;Step S4: screen out all the images containing control point, based on number, obtain the position information of control point on ground in image, based on the position of vehicle when shooting image, convert the position information of control point on ground into coordinate information under vehicle three-dimensional coordinate system, according to the coordinate information of vehicle camera under vehicle three-dimensional coordinate system and the coordinate information of control point, calculate the extrinsic calibration of vehicle camera.