Calibration method for external parameters of sensor and electronic equipment

Abstract

The invention discloses a sensor external parameter calibration method and electronic equipment, and the method comprises the steps: determining a plurality of sensors which need to calibrate external parameters in a preset scene; calibrating translation external parameters of the plurality of sensors to obtain a first calibration result of each sensor relative to a vehicle body coordinate system; calibrating the rotation external parameters of the plurality of sensors to obtain a second calibration result of each sensor relative to the vehicle body coordinate system; based on the first calibration result and the second calibration result, target external parameters among the sensors are obtained through matrix transformation. The calibration method for each sensor is simple, convenient and convenient to implement, the precision of the obtained external parameters of the sensors is high, the vehicle calibration efficiency can be greatly improved, and the manual intervention degree is reduced.

Description

Technical Field

[0001] This application relates to the field of sensor technology, and in particular to a method for calibrating sensor extrinsic parameters and an electronic device. Background Technology

[0002] In intelligent driving, multiple intelligent sensors on the vehicle are used to collect data from all directions, enabling onboard equipment to perform positioning, planning, and control based on the sensor data, thus guiding the vehicle to travel along the planned path. Therefore, the accuracy of sensor extrinsic parameters is a prerequisite for intelligent driving.

[0003] Multi-sensor extrinsic parameter calibration aims to accurately determine the position and orientation relationships of each sensor in a unified coordinate system. For autonomous vehicles and intelligent robots, due to the diversity of sensor types and complex installation, the accuracy of extrinsic parameter calibration directly affects the collaborative effect of multiple sensors. How to complete the calibration efficiently and accurately is an urgent problem to be solved during the assembly process. Summary of the Invention

[0004] This disclosure provides a method, apparatus, device, and storage medium for calibrating sensor extrinsic parameters, in order to at least solve the above-mentioned technical problems existing in the prior art.

[0005] According to a first aspect of this application, a method for calibrating sensor extrinsic parameters is provided, comprising: In a predefined scenario, identify multiple sensors whose extrinsic parameters need to be calibrated; The translational extrinsic parameters of the multiple sensors are calibrated to obtain the first calibration result of each sensor relative to the vehicle coordinate system; The rotational extrinsic parameters of the multiple sensors are calibrated to obtain the second calibration results of each sensor relative to the vehicle coordinate system; Based on the first calibration result and the second calibration result, the target extrinsic parameters between each sensor are obtained through matrix transformation.

[0006] In one possible implementation, the plurality of sensors includes at least one of the following sensors: Inertial measurement sensors, lidar sensors, image acquisition sensors, or millimeter-wave radar sensors.

[0007] In one possible implementation, calibrating the translational extrinsic parameters of the plurality of sensors to obtain a first calibration result for each sensor relative to the vehicle coordinate system includes: Mark the vehicle coordinate system; The geometric feature points of each sensor are selected as the measurement points of the sensor; the geometric feature points are points that can be measured repeatedly. The distance between the origin of the vehicle coordinate system and the measurement point of the sensor is measured; the distance includes the distance between the origin of the vehicle coordinate system and the measurement point of the sensor in the horizontal axis direction, the vertical axis direction, and the longitudinal axis direction. The translation vector corresponding to the distance is used as the first calibration result.

[0008] In one possible implementation, the sensor whose extrinsic parameters need to be calibrated is an inertial measurement sensor. The calibration of the rotational extrinsic parameters of multiple sensors to obtain a second calibration result for each sensor relative to the vehicle coordinate system includes: Under the preset scenario, acquire vehicle trajectory data and the first heading angle collected by the inertial measurement sensor; Based on the vehicle trajectory data, multiple positioning points of the vehicle are obtained and straight line fitting is performed to obtain a straight line trajectory. Based on the straight trajectory, the vehicle's second heading angle is obtained; Calculate the heading deviation angle based on the first heading angle and the second heading angle; The heading deviation angle is used as the second calibration result of the inertial measurement sensor.

[0009] In one possible implementation, the sensor whose extrinsic parameters need to be calibrated is a lidar sensor. The calibration of the rotational extrinsic parameters of multiple sensors to obtain a second calibration result for each sensor relative to the vehicle coordinate system includes: In a preset scenario, laser point cloud data is collected using a lidar sensor while the vehicle is in motion. By filtering and fitting the ground point data in the laser point cloud data, the ground plane equation is obtained; Based on the ground plane equation, the plane normal vector is obtained; The angle between the plane normal vector and the theoretical normal vector is calculated to obtain the roll angle error of the lidar sensor; Filter target feature points in adjacent frames of the laser point cloud data; Based on the motion transformation of the target feature points, the driving trajectory of the lidar sensor relative to the vehicle coordinate system is determined; Based on the driving trajectory, the third heading angle of the vehicle is obtained, and based on the tangent direction of the driving trajectory, the fourth heading angle of the vehicle is obtained; wherein, the third heading angle is the heading angle output by the lidar sensor, and the fourth heading angle is the actual heading angle of the vehicle body; Based on the third and fourth heading angles, the heading angle error is obtained; Based on the roll angle error and heading angle error, the second calibration result of the lidar sensor relative to the vehicle coordinate system is determined.

[0010] In one possible implementation, the sensor whose extrinsic parameters need to be calibrated is an image acquisition sensor. The calibration of the rotational extrinsic parameters of multiple sensors to obtain a second calibration result for each sensor relative to the vehicle coordinate system includes: Under a preset scenario, acquire video images of the vehicle in motion; Lane lines are obtained for each frame of the video image based on a pre-built perception model. Determine the vanishing point and horizontal line of the target based on the intersection of the extensions of multiple lane lines; Based on the vanishing point of the target, calculate the first pitch angle and the fifth heading angle of the image acquisition sensor relative to the road direction, and calculate the roll angle of the image acquisition sensor relative to the ground plane based on the horizontal line; Based on the first pitch angle, the fifth heading angle, and the roll angle, the second calibration result of the image acquisition sensor relative to the vehicle coordinate system is determined.

[0011] In one possible implementation, the sensor whose extrinsic parameters need to be calibrated is a millimeter-wave radar sensor. The calibration of the rotational extrinsic parameters of multiple sensors to obtain a second calibration result for each sensor relative to the vehicle coordinate system includes: Under a preset scenario, acquire the radar data frame sequence collected by the millimeter-wave radar sensor; The speed of each data frame in the radar data frame sequence is compared with the vehicle speed to determine the initial stationary object; Identify the azimuth distribution of the initial stationary object, and determine the direction of vehicle movement as the direction in which the azimuth distribution is most concentrated; The first yaw angle of the millimeter-wave radar sensor is determined based on the vehicle's direction of motion. Based on the first yaw angle, the radar data frame is converted to the vehicle coordinate system; Based on preset screening conditions, target stationary objects are selected from initial stationary objects; the screening conditions are that the relative velocity direction of the initial stationary objects is opposite to the vehicle's motion direction and the relative velocity magnitude is within a preset range from the vehicle's velocity magnitude. Based on the cosine function relationship between the radial velocity and azimuth angle of the stationary target object, a cosine function curve is fitted. Based on the cosine function curve, the second yaw angle is obtained; Calculate the difference between the second yaw angle and the first yaw angle; When the difference is less than the second preset value, the second yaw angle is taken as the target yaw angle; otherwise, the new yaw angle is calculated iteratively until the difference between the new yaw angle and the previous yaw angle is less than the second preset value. Then, the new yaw angle is output as the target yaw angle. Based on the target yaw angle, a second calibration result of the millimeter-wave radar sensor relative to the vehicle coordinate system is obtained.

[0012] In one possible implementation, the preset scenario is:

[0013] The road on which the vehicle travels is a straight, level road.

[0014] In one possible implementation, it further includes: The data collected by each sensor is fused based on the target extrinsic parameters between the sensors.

[0015] According to a second aspect of this application, a calibration apparatus for sensor extrinsic parameters is provided, comprising: The determination module is used to determine multiple sensors whose extrinsic parameters need to be calibrated under a preset scenario; The first calibration module is used to calibrate the translational extrinsic parameters of the multiple sensors to obtain the first calibration result of each sensor relative to the vehicle coordinate system; The second calibration module is used to calibrate the rotational extrinsic parameters of the multiple sensors to obtain the second calibration results of each sensor relative to the vehicle coordinate system. The transformation module is used to obtain the target extrinsic parameters between each sensor through matrix transformation based on the first calibration result and the second calibration result.

[0016] According to a third aspect of this application, an electronic device is provided, comprising: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method described in this application.

[0017] According to a fourth aspect of this application, a non-transitory computer-readable storage medium is provided storing computer instructions for causing the computer to perform the methods described in this application.

[0018] According to a fifth aspect of this application, a computer program product is provided, comprising a computer program or instructions that, when executed by a processor, implement the method described in this application.

[0019] Using the technical solution of this application, sensor extrinsic parameters can be obtained with high accuracy through simple implementation. This application can significantly improve vehicle calibration efficiency and reduce the degree of manual intervention.

[0020] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this application, nor is it intended to limit the scope of this application. Other features of this application will become readily apparent from the following description. Attached Figure Description

[0021] The above and other objects, features, and advantages of exemplary embodiments of this application will become readily apparent from the following detailed description taken in conjunction with the accompanying drawings. Several embodiments of this application are illustrated in the drawings by way of example and not limitation, wherein: In the accompanying drawings, the same or corresponding reference numerals indicate the same or corresponding parts.

[0022] Figure 1 A schematic diagram illustrating the implementation flow of the sensor extrinsic parameter calibration method in an embodiment of this application is shown; Figure 2 A schematic diagram of a preset scenario in an embodiment of this application is shown; Figure 3 This illustration shows a schematic diagram of lane line extraction by the perception model in an embodiment of this application; Figure 4 The vanishing point extracted in the image is shown in an embodiment of this application; Figure 5 A structural block diagram of the sensor extrinsic parameter calibration device in an embodiment of this application is shown; Figure 6 A schematic diagram of the composition structure of an electronic device according to an embodiment of this application is shown. Detailed Implementation

[0023] To make the objectives, features, and advantages of this application more apparent and understandable, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0024] In the following description, the terms "first" and "second" are used merely to distinguish similar objects and do not represent a specific ordering of objects. It is understood that "first" and "second" may be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0025] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0026] The following description, in conjunction with the accompanying drawings, introduces a method for calibrating sensor extrinsic parameters and an electronic device provided in this application.

[0027] like Figure 1 As shown, this application provides a method for calibrating sensor extrinsic parameters, including: S101, under a preset scenario, identify multiple sensors whose extrinsic parameters need to be calibrated; The sensor extrinsic parameter calibration method provided in this application can be applied to autonomous vehicles. The scenario is the physical environment in which the autonomous vehicle is driving. Figure 2 As shown, the preset scenario in this application is a straight, planar road where vehicles travel. It should be noted that the preset scenario in this application also includes straight roads with clearly defined lane lines and areas with abundant stationary objects.

[0028] The multiple sensors in this application may be a combination of sensors mounted on a vehicle that require extrinsic parameter calibration. The sensor combination includes at least two types of sensors.

[0029] S102, calibrate the translation extrinsic parameters of the multiple sensors to obtain the first calibration result of each sensor relative to the vehicle coordinate system; In this application, the translational extrinsic parameters of multiple sensors can be calibrated by manual measurement. Before calibration, this application needs to establish a vehicle coordinate system. For example, the vehicle coordinate system can be designed as follows: the origin is located at the center of the rear axle or the center of mass of the vehicle, the X-axis points in the direction of vehicle movement, the Y-axis is perpendicular to the direction of vehicle movement and points to the left, and the Z-axis is vertically upward.

[0030] Simultaneously, physical reference points need to be set for each sensor, i.e., the origin of the sensor's own coordinate system, referred to as measurement points in this application. For example, the measurement point for a GNSS antenna is the phase center of the antenna. The measurement point for an inertial measurement sensor is its geometric center or a designated reference point. The measurement point for a lidar sensor is its rotation center or a reference mark on its casing. The measurement point for an image acquisition sensor is the optical center of the lens. The measurement point for a millimeter-wave radar sensor is the phase center of the antenna array or a reference mark on its casing.

[0031] In this application, a measuring tool can be used to measure the offset in the X, Y, and Z directions from the origin of the vehicle coordinate system to the physical reference point of the sensor. The offset vector is the translational extrinsic parameter of the sensor. The measuring tool can be a total station, laser tracker, measuring tape, laser rangefinder, etc.

[0032] S103, calibrate the rotational extrinsic parameters of the multiple sensors to obtain the second calibration result of each sensor relative to the vehicle coordinate system; This application calibrates the rotational extrinsic parameters of multiple sensors, specifically determining the three-dimensional attitude relationship (i.e., roll angle, pitch angle, and yaw angle) between each sensor and the vehicle coordinate system. This application can calibrate the extrinsic parameters of each sensor after determining the specific sensors for which extrinsic parameters need to be calibrated, thereby obtaining a second calibration result for each sensor relative to the vehicle coordinate system.

[0033] S104, Based on the first calibration result and the second calibration result, the target extrinsic parameters between each sensor are obtained through matrix transformation.

[0034] In this application, matrix transformations are performed on the first calibration result and the second calibration result to obtain the target extrinsic parameters between each sensor. Specifically, for each sensor, a rigid body transformation matrix from the sensor to the vehicle coordinate system is constructed based on its first calibration result (translation vector) and second calibration result (rotation matrix) relative to the vehicle coordinate system.

[0035] In this application, when the vehicle is traveling on a straight, planar road, the sensors whose extrinsic parameters need to be calibrated are first identified, such as image acquisition sensors or inertial sensors. Then, the translational extrinsic parameters of the sensors whose extrinsic parameters need to be calibrated are first calibrated, followed by the rotational extrinsic parameters, thereby obtaining the target extrinsic parameters among the various sensors.

[0036] In some embodiments, the plurality of sensors includes at least one of the following sensors: Inertial measurement sensors, lidar sensors, image acquisition sensors, or millimeter-wave radar sensors.

[0037] The inertial measurement sensor, by fusing the angle obtained from gyroscope integration and the direction of gravity sensed by the accelerometer, can stably estimate the roll and pitch angles of the carrier relative to the Earth's horizontal plane. Combined with a magnetometer or GPS, it can also provide the heading angle.

[0038] By matching real-time scanned point clouds with pre-existing high-precision maps, LiDAR sensors enable vehicles to determine their precise location in a global coordinate system.

[0039] Image acquisition sensors help determine the precise location of a vehicle by matching the currently viewed scene with visual features in a high-precision map, such as lane lines, signs, and building outlines.

[0040] Millimeter-wave radar sensors can detect objects within a certain distance and angle range and output their distance, azimuth, and relative velocity.

[0041] In some embodiments, calibrating the translational extrinsic parameters of the plurality of sensors to obtain a first calibration result for each sensor relative to the vehicle coordinate system includes: Mark the vehicle coordinate system; The geometric feature points of each sensor are selected as the measurement points of the sensor; the geometric feature points are points that can be measured repeatedly. The distance between the origin of the vehicle coordinate system and the measurement point of the sensor is measured; the distance includes the distance between the origin of the vehicle coordinate system and the measurement point of the sensor in the horizontal axis direction, the vertical axis direction, and the longitudinal axis direction. The translation vector corresponding to the distance is used as the first calibration result.

[0042] In this application, after determining the vehicle's body coordinate system and the sensor's measurement points, the distance between the origin of the body coordinate system and the sensor's measurement points can be manually measured. The horizontal axis points directly forward of the vehicle, the vertical axis points to the driver's left, and the center axis points upward. Specifically, the horizontal axis distance is the distance between the measurement point and the origin along the vehicle's vertical axis (positive forward, negative backward). The vertical axis distance is the distance between the measurement point and the origin along the vehicle's horizontal axis (positive left, negative right). The center axis distance is the distance between the measurement point and the origin along the vehicle's vertical axis (positive upward, negative downward). These three distance components are combined into a three-dimensional translation vector, which represents the sensor's first calibration result relative to the vehicle's body coordinate system.

[0043] This application can ensure the consistency of the measurement process and the reliability of the results, achieving "controllable accuracy".

[0044] In some embodiments, the sensors whose extrinsic parameters need to be calibrated are inertial measurement sensors. The calibration of the rotational extrinsic parameters of multiple sensors to obtain a second calibration result for each sensor relative to the vehicle coordinate system includes: Under the preset scenario, acquire vehicle trajectory data and the first heading angle collected by the inertial measurement sensor; Based on the vehicle trajectory data, multiple positioning points of the vehicle are obtained and straight line fitting is performed to obtain a straight line trajectory. Based on the straight trajectory, the vehicle's second heading angle is obtained; Calculate the heading deviation angle based on the first heading angle and the second heading angle; The heading deviation angle is used as the second calibration result of the inertial measurement sensor.

[0045] In this application, under a preset straight road scenario, the vehicle is controlled to maintain straight-line driving, and a first heading angle sequence output by an inertial measurement sensor and vehicle trajectory data provided by a positioning system are simultaneously acquired. Based on the vehicle trajectory data, multiple positioning points of the vehicle during the straight-line driving phase are extracted, and the best-fit straight trajectory is obtained through a straight-line fitting algorithm. The azimuth angle of the best-fit straight trajectory in the navigation coordinate system is calculated, and this azimuth angle is determined as the actual heading angle of the vehicle, denoted as the second heading angle. The difference between the first heading angle and the second heading angle output by the inertial measurement sensor is calculated to obtain the heading angle deviation. The heading angle deviation is used as the second calibration result of the inertial measurement sensor relative to the vehicle coordinate system. It should be noted that, given the known changes in the heading of the inertial measurement sensor and the vehicle's heading, both should have a heading deviation that changes synchronously. Therefore, the difference between the two is the heading extrinsic parameter of the inertial measurement sensor and the vehicle.

[0046] In some embodiments, the sensor whose extrinsic parameters need to be calibrated is a lidar sensor. The calibration of the rotational extrinsic parameters of multiple sensors to obtain a second calibration result for each sensor relative to the vehicle coordinate system includes: In a preset scenario, laser point cloud data is collected using a lidar sensor while the vehicle is in motion. By filtering and fitting the ground point data in the laser point cloud data, the ground plane equation is obtained; Based on the ground plane equation, the plane normal vector is obtained; The angle between the plane normal vector and the theoretical normal vector is calculated to obtain the roll angle error of the lidar sensor; Filter target feature points in adjacent frames of the laser point cloud data; Based on the motion transformation of the target feature points, the driving trajectory of the lidar sensor relative to the vehicle coordinate system is determined; Based on the driving trajectory, the third heading angle of the vehicle is obtained, and based on the tangent direction of the driving trajectory, the fourth heading angle of the vehicle is obtained; wherein, the third heading angle is the heading angle output by the lidar sensor, and the fourth heading angle is the actual heading angle of the vehicle body; Based on the third and fourth heading angles, the heading angle error is obtained; Based on the roll angle error and heading angle error, the second calibration result of the lidar sensor relative to the vehicle coordinate system is determined.

[0047] In this application, under a preset straight road scenario, the vehicle is controlled to maintain straight-line travel, and the LiDAR is activated to collect multiple consecutive frames of LiDAR point cloud data during the vehicle's movement. For each frame of point cloud, based on the characteristics of the point cloud (such as reflection intensity, height information, or simple geometric rules), point cloud data belonging to the ground are selected. Then, using the ground point cloud, a plane equation is fitted using a robust algorithm such as least squares or RANSAC. The general form of this plane equation is: Ax + By + Cz + D = 0. From the fitted plane equation, the normal vector of the plane can be directly obtained. It should be noted that this vector is expressed in the LiDAR coordinate system. Ideally, when the LiDAR is installed correctly, the normal vector of the ground plane should be consistent with the Z-axis (pointing upwards) of the vehicle coordinate system. The angle between the measured normal vector and the theoretical normal vector is calculated. Since the vehicle is on a flat road, this angle is caused by the roll angle installation error of the LiDAR. The roll angle error can be calculated using the vector angle formula.

[0048] Stable and easily trackable feature points are selected from continuous multi-frame laser point clouds. The motion transformation matrix between adjacent frame point clouds is calculated using the iterative nearest-point algorithm or other feature-matching-based algorithms. By integrating these continuous inter-frame transformations, the motion trajectory of the lidar relative to its starting point during travel can be constructed. This trajectory reflects the lidar's own motion state. From the motion trajectory calculated through point cloud matching, the third heading angle of the lidar at each moment can be extracted. Since the vehicle is traveling on a straight road, its actual direction of travel should be consistent with the tangent direction of the trajectory. By differentiating or piecewise linearly fitting the fitted trajectory, the tangent direction at each point on the trajectory can be obtained, which is the vehicle's actual heading angle, i.e., the fourth heading angle. Comparing the heading angle sensed by the lidar (third heading angle) with the vehicle's actual heading angle (fourth heading angle), the systematic deviation between the two is the lidar's heading angle installation error. The roll angle error and heading angle error are the calibration results of the lidar sensor's rotational extrinsic parameters relative to the vehicle coordinate system.

[0049] This application requires no additional equipment and utilizes only natural scene data during normal vehicle operation to achieve automated, data-driven calibration, which greatly improves efficiency and reduces labor costs.

[0050] In some embodiments, the sensor whose extrinsic parameters need to be calibrated is an image acquisition sensor. The calibration of the rotational extrinsic parameters of multiple sensors to obtain a second calibration result for each sensor relative to the vehicle coordinate system includes: Under a preset scenario, acquire video images of the vehicle in motion; Lane lines are obtained for each frame of the video image based on a pre-built perception model. Determine the vanishing point and horizontal line of the target based on the intersection of the extensions of multiple lane lines; Based on the vanishing point of the target, calculate the first pitch angle and the fifth heading angle of the image acquisition sensor relative to the road direction, and calculate the roll angle of the image acquisition sensor relative to the ground plane based on the horizontal line; Based on the first pitch angle, the fifth heading angle, and the roll angle, the second calibration result of the image acquisition sensor relative to the vehicle coordinate system is determined.

[0051] In this application, a road with clearly defined and straight lane markings is selected, and the vehicle is controlled to maintain a straight line in the center of the lane. An image acquisition sensor is activated to capture a sequence of video images of the vehicle's movement. Each frame is then input into a pre-built, trained deep learning perception model. Figure 3 As shown, deep learning perception models can identify and extract lane lines in images at the pixel level, outputting the lane line contours or center lines. Figure 4 As shown, the intersection points of the extended lines between each pair of lane lines are calculated. The horizontal line in the image is a line that reflects the rotation of the image acquisition sensor around the vertical axis. It is obtained based on the directional line segment features such as the extracted lane lines. Robust algorithms such as RANSAC clustering are used to fit the most stable optimal horizontal line from multiple frames of image data.

[0052] The first pitch angle and fifth yaw angle of the image acquisition sensor relative to the road direction are calculated using the pixel coordinates of the vanishing point and the intrinsic parameters of the image acquisition sensor. Based on the equation of the fitted horizontal line in the image coordinate system, the angle between it and the horizontal axis of the image, i.e., the roll angle, is calculated. The fifth yaw angle and first pitch angle of the image acquisition sensor relative to the road direction, and the roll angle relative to the ground plane, are the extrinsic parameters of the image acquisition sensor coordinate system relative to the vehicle coordinate system. The second calibration result of the image acquisition sensor relative to the vehicle coordinate system is then presented.

[0053] This application eliminates the need for manual calibration board setup; it can be accomplished using natural road scenarios without requiring additional hardware.

[0054] In some embodiments, the sensor whose extrinsic parameters need to be calibrated is a millimeter-wave radar sensor. The calibration of the rotational extrinsic parameters of multiple sensors to obtain a second calibration result for each sensor relative to the vehicle coordinate system includes: Under a preset scenario, acquire the radar data frame sequence collected by the millimeter-wave radar sensor; The speed of each data frame in the radar data frame sequence is compared with the vehicle speed to determine the initial stationary object; Identify the azimuth distribution of the initial stationary object, and determine the direction of vehicle movement as the direction in which the azimuth distribution is most concentrated; The first yaw angle of the millimeter-wave radar sensor is determined based on the vehicle's direction of motion. Based on the first yaw angle, the radar data frame is converted to the vehicle coordinate system; Based on preset screening conditions, target stationary objects are selected from initial stationary objects; the screening conditions are that the relative velocity direction of the initial stationary objects is opposite to the vehicle's motion direction and the relative velocity magnitude is within a preset range from the vehicle's velocity magnitude. Based on the cosine function relationship between the radial velocity and azimuth angle of the stationary target object, a cosine function curve is fitted. Based on the cosine function curve, the second yaw angle is obtained; Calculate the difference between the second yaw angle and the first yaw angle; When the difference is less than the second preset value, the second yaw angle is taken as the target yaw angle; otherwise, the new yaw angle is calculated iteratively until the difference between the new yaw angle and the previous yaw angle is less than the second preset value. Then, the new yaw angle is output as the target yaw angle. Based on the target yaw angle, a second calibration result of the millimeter-wave radar sensor relative to the vehicle coordinate system is obtained.

[0055] In this application, a car is controlled to travel on a straight road, with abundant stationary objects on both sides of the road. These stationary objects can be guardrails, streetlights, traffic signs, etc. A sequence of radar data frames collected by a millimeter-wave radar sensor is acquired. Simultaneously, the vehicle's actual speed information is obtained from the vehicle's CAN bus. For each frame of radar data, the absolute value of the radial velocity of each detected target is compared with the absolute value of the vehicle's speed. Specifically, target points with speeds close to the vehicle's speed are initially identified as initial stationary objects, as their relative speeds primarily originate from the vehicle's own motion. Analyzing the azimuth distribution of all these initial stationary objects, the vast majority of truly stationary objects should be located on either side of the vehicle's direction of motion. Therefore, the direction with the densest azimuth distribution can be considered the vehicle's direction of motion. Furthermore, an initial, coarse first yaw angle is calculated. Using this first yaw angle, all target points detected by the radar are transformed from the radar coordinate system to the vehicle coordinate system. Specifically, the selection criteria are that the relative velocity direction of the initial stationary object is opposite to the vehicle's direction of motion, and the relative velocity magnitude is within a preset range compared to the vehicle's speed. This filters out misclassified dynamic objects and noise points.

[0056] In radar coordinates, the radial velocity of a stationary target V 1 and its azimuth angle Satisfy the cosine relation,

[0057] in, For vehicle speed, This is the actual yaw angle.

[0058] The (azimuth and radial velocity) data points of all stationary target objects selected in step 1 are aggregated and fitted with a cosine function curve. From the fitted curve, the phase parameter can be extracted, thereby calculating an updated and more accurate second yaw angle.

[0059] Calculate the difference between the updated yaw angle obtained in the current iteration (e.g., the second yaw angle) and the yaw angle from the previous iteration (e.g., the first yaw angle). If the difference is less than a second preset value, it indicates that the yaw angle estimation has stabilized and the iteration has converged. Set the currently updated yaw angle as the target yaw angle. Otherwise, use the currently updated yaw angle as the new initial value, jump back to step two, and continue performing the coordinate system transformation, screening of stationary objects, curve fitting, and updating of the yaw angle. Repeat this process until the difference between the new yaw angle and the previous yaw angle is less than the second preset value, and output the final target yaw angle.

[0060] This application requires no reflector or specific marker and can be completed using the natural environment.

[0061] In some embodiments, it also includes: The data collected by each sensor is fused based on the target extrinsic parameters between the sensors.

[0062] This application calibrates the extrinsic parameters of all sensors relative to the vehicle body (reference coordinate system). Then, based on this, matrix transformations are used to obtain the extrinsic parameter results between each sensor. Finally, various sensor fusion operations can be performed on this basis. The scenario implemented in this application is relatively simple: sensors are installed at various locations on the vehicle as required, and the final calibration result is output as long as the vehicle travels in a straight line. The technical solution provided in this application is simple and easy to implement, has high accuracy, and can significantly improve vehicle calibration efficiency while reducing manual intervention.

[0063] like Figure 5 As shown, this application provides a calibration device for sensor extrinsic parameters, comprising: The determination module 501 is used to determine multiple sensors whose extrinsic parameters need to be calibrated under a preset scenario; The first calibration module 502 is used to calibrate the translational extrinsic parameters of the multiple sensors to obtain the first calibration result of each sensor relative to the vehicle coordinate system. The second calibration module 503 is used to calibrate the rotational extrinsic parameters of the multiple sensors to obtain the second calibration result of each sensor relative to the vehicle coordinate system. The transformation module 504 is used to obtain the target extrinsic parameters between the various sensors through matrix transformation based on the first calibration result and the second calibration result.

[0064] According to embodiments of this application, this application also provides an electronic device and a readable storage medium.

[0065] The electronic device includes at least one processor and a memory communicatively connected to the at least one processor. The memory stores instructions executable by the at least one processor, which, when executed, enable the at least one processor to perform the sensor extrinsic parameter calibration method described in this application. The computer instructions are used to cause the computer to perform the sensor extrinsic parameter calibration method described in this application.

[0066] This application also provides a computer program product, including a computer program / instructions, which, when executed by a processor, implements the sensor extrinsic parameter calibration method of this application.

[0067] Figure 6 A schematic block diagram of an example electronic device 800 that can be used to implement embodiments of this application is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the application described and / or claimed herein.

[0068] like Figure 6 As shown, device 800 includes a computing unit 801, which can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) 802 or a computer program loaded from storage unit 808 into random access memory (RAM) 803. RAM 803 may also store various programs and data required for the operation of device 800. The computing unit 801, ROM 802, and RAM 803 are interconnected via bus 804. Input / output (I / O) interface 805 is also connected to bus 804.

[0069] Multiple components in device 800 are connected to I / O interface 805, including: input unit 806, such as keyboard, mouse, etc.; output unit 807, such as various types of monitors, speakers, etc.; storage unit 808, such as disk, optical disk, etc.; and communication unit 809, such as network card, modem, wireless transceiver, etc. Communication unit 809 allows device 800 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0070] The computing unit 801 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 801 performs the various methods and processes described above, such as the sensor extrinsic parameter calibration method. For example, in some embodiments, the sensor extrinsic parameter calibration method may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 808. In some embodiments, part or all of the computer program may be loaded and / or installed on device 800 via ROM 802 and / or communication unit 809. When the computer program is loaded into RAM 803 and executed by the computing unit 801, one or more steps of the sensor extrinsic parameter calibration method described above may be performed. Alternatively, in other embodiments, the computing unit 801 may be configured by any other suitable means (e.g., by means of firmware) to perform a calibration method for sensor extrinsics.

[0071] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0072] The program code used to implement the methods of this application may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing device, such that when executed by the processor or controller, the functions / operations specified in the flowcharts and / or block diagrams are implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0073] In the context of this application, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0074] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0075] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0076] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0077] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for calibrating extrinsic parameters of a sensor, characterized in that, The application relates to a method for calibrating external parameters of multiple sensors. In a preset scene, multiple sensors needing to be calibrated are determined; The translation external parameters of the multiple sensors are calibrated to obtain a first calibration result of each sensor relative to a vehicle body coordinate system; The rotation external parameters of the multiple sensors are calibrated to obtain a second calibration result of each sensor relative to the vehicle body coordinate system; Based on the first calibration result and the second calibration result, a target external parameter between each sensor is obtained through matrix transformation.

2. The method of claim 1, wherein, The multiple sensors at least include one of the following sensors: An inertial measurement sensor, a laser radar sensor, an image acquisition sensor or a millimeter wave radar sensor.

3. The method of claim 2, wherein, The calibration of the translation external parameters of the multiple sensors to obtain the first calibration result of each sensor relative to the vehicle body coordinate system comprises: Marking a vehicle body coordinate system; Selecting a geometric feature point of each sensor as a measurement point of the sensor; the geometric feature point is a point capable of repeated measurement; Measuring the distance between the origin of the vehicle body coordinate system and the measurement point of the sensor; the distance includes the distance between the origin of the vehicle body coordinate system and the measurement point of the sensor in the horizontal axis direction, the vertical axis direction and the vertical axis direction; The translation vector corresponding to the distance is taken as the first calibration result.

4. The method of claim 2, wherein, The sensor needing to be calibrated is an inertial measurement sensor, and the calibration of the rotation external parameters of the multiple sensors to obtain the second calibration result of each sensor relative to the vehicle body coordinate system comprises: In a preset scene, vehicle trajectory data and a first heading angle collected by the inertial measurement sensor are obtained; Based on the vehicle trajectory data, multiple positioning points of the vehicle are obtained and straight line fitting is performed to obtain a straight line trajectory; Based on the straight line trajectory, a second heading angle of the vehicle is obtained; Based on the first heading angle and the second heading angle, a heading deviation angle is calculated; The heading deviation angle is taken as the second calibration result of the inertial measurement sensor.

5. The method of claim 2, wherein, The sensor needing to be calibrated is a laser radar sensor, and the calibration of the rotation external parameters of the multiple sensors to obtain the second calibration result of each sensor relative to the vehicle body coordinate system comprises: In a preset scene, laser point cloud data of a vehicle during driving is collected by the laser radar sensor; Ground point data in the laser point cloud data is screened and fitted to obtain a ground plane equation; Based on the ground plane equation, a plane normal vector is obtained; An included angle between the plane normal vector and a theoretical normal vector is calculated to obtain a roll angle error of the laser radar sensor; Target feature points of adjacent frame point clouds in the laser point cloud data are screened; Based on the motion transformation of the target feature points, a driving trajectory of the laser radar sensor relative to the vehicle body coordinate system is determined; Based on the driving trajectory, a third heading angle of the vehicle is obtained, and based on the tangent direction of the driving trajectory, a fourth heading angle of the vehicle is obtained; wherein the third heading angle is a heading angle output by the laser radar sensor, and the fourth heading angle is a true heading angle of the vehicle body; Based on the third heading angle and the fourth heading angle, a heading angle error is obtained; Based on the roll angle error and the heading angle error, the second calibration result of the laser radar sensor relative to the vehicle body coordinate system is determined.

6. The method of claim 2, wherein, The sensor needing to calibrate the external parameter is an image acquisition sensor, the rotation external parameters of a plurality of the sensors are calibrated to obtain a second calibration result of each sensor relative to a vehicle body coordinate system, and the second calibration result comprises: In a preset scene, video images of vehicle driving are acquired; Based on a pre-constructed perception model, lane lines of each frame of image in the video images are acquired; Based on intersection points of extension lines of a plurality of lane lines, a target vanishing point and a horizontal line are determined; Based on the target vanishing point, a first pitch angle and a fifth heading angle of the image acquisition sensor relative to a road direction are calculated, and a roll angle of the image acquisition sensor relative to a ground plane is calculated based on the horizontal line; Based on the first pitch angle, the fifth heading angle and the roll angle, a second calibration result of the image acquisition sensor relative to the vehicle body coordinate system is determined.

7. The method of claim 2, wherein, The sensor needing to calibrate the external parameter is a millimeter wave radar sensor, the rotation external parameters of a plurality of the sensors are calibrated to obtain a second calibration result of each sensor relative to a vehicle body coordinate system, and the second calibration result comprises: In a preset scene, a radar data frame sequence acquired by the millimeter wave radar sensor is acquired; By comparing a speed of each data frame in the radar data frame sequence with a vehicle speed, an initial stationary object is determined; An azimuth angle distribution of the initial stationary object is identified, and a direction with the densest azimuth angle distribution is determined as a vehicle motion direction; Based on the vehicle motion direction, a first yaw angle of the millimeter wave radar sensor is determined; Based on the first yaw angle, the radar data frame is converted to the vehicle body coordinate system; Based on a preset screening condition, a target stationary object is screened from the initial stationary object; the screening condition is that a relative speed direction of the initial stationary object is opposite to the vehicle motion direction and a relative speed size is within a preset range of the vehicle speed size; Based on a cosine function relationship between a radial speed and an azimuth angle of the target stationary object, a cosine function curve is fitted; Based on the cosine function curve, a second yaw angle is obtained; A difference value between the second yaw angle and the first yaw angle is calculated; In response to the difference value being less than a second preset value, the second yaw angle is taken as a target yaw angle, otherwise a new yaw angle is iteratively calculated until a difference between the new yaw angle and a previous yaw angle is less than the second preset value, and the new yaw angle is output as the target yaw angle; Based on the target yaw angle, a second calibration result of the millimeter wave radar sensor relative to the vehicle body coordinate system is obtained.

8. The method of claim 1, wherein, The preset scene is The road on which the vehicle drives is a flat straight road.

9. The method of claim 1, wherein, Further comprising: Fusing data acquired by each sensor according to a target external parameter between the sensors.

10. An electronic device, comprising: Comprising: At least one processor; and a memory connected in communication with the at least one processor; wherein The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the method of any one of claims 1 to 9.

Images