A vehicle sensor calibration method, device, medium, vehicle-mounted system and vehicle

Abstract

The application discloses a vehicle sensor calibration method, device, medium, vehicle-mounted system and vehicle, and belongs to the technical field of computers. Mainly includes, vehicle-mounted radar calibration process, corner point reference position information acquisition process, corner point reference position information acquisition process, corner point re-projection error calculation process, judgment adjustment process, calibration process, the application calibrates the external parameter of the vehicle-mounted camera group by using the corner point in the daily driving place, which can be convenient and efficient, and the automatic calibration of the vehicle-mounted camera can be completed at a low cost.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a vehicle sensor calibration method, device, medium, vehicle system, and vehicle. Background Technology

[0002] Typically, automotive sensors such as LiDAR and cameras are rigidly mounted on autonomous vehicles. Ideally, once fixed, the relative positions of the sensors and the vehicle body remain unchanged; that is, after installation, only one calibration is needed to establish the coordinate transformation between them. However, in reality, various factors, such as vibrations from vehicle movement, prolonged driving on uneven roads, loosening of the fixing connections between the sensors and the vehicle body, and thermal expansion and contraction of the connections due to temperature changes, can alter the relative positions of the sensors and the vehicle body. In such cases, recalibration of the sensor positions is necessary.

[0003] Existing sensor calibration methods require vehicles to be driven into calibration fields for manual calibration, which is costly to build and time-consuming. Therefore, there is an urgent need for a low-cost and convenient automatic calibration method for vehicle sensors. Summary of the Invention

[0004] To address the problems existing in the prior art, this application mainly provides a vehicle sensor calibration method, device, medium, vehicle system, and vehicle. By utilizing the vehicle body's structural features and corner points in daily driving environments to calibrate the extrinsic parameters of vehicle sensors, it is convenient, efficient, and can complete the automatic calibration of vehicle cameras at a low cost.

[0005] To achieve the above objectives, this application adopts the following technical solution: a vehicle sensor calibration method, comprising: a vehicle-mounted radar calibration process, wherein the vehicle-mounted radar extrinsic parameters are calibrated using vehicle body structural features, wherein the vehicle body structural features are within the detection range of the vehicle-mounted radar and their position information in the vehicle coordinate system is obtainable; a corner reference position information acquisition process, wherein during vehicle operation, point cloud information of a corner point in a predetermined daily driving location is collected using a vehicle-mounted radar with accurate orientation calibration, and reference position information of the selected corner point is obtained based on the point cloud information of the selected corner point and the transformation relationship between the vehicle-mounted radar and the vehicle's coordinate system, wherein the number of corner points included in the predetermined daily driving location is not less than a preset threshold; and a corner reference position information acquisition process, wherein image information of the selected corner point is collected using a vehicle-mounted camera group configured on the vehicle, and reference position information of the selected corner point is obtained based on the image information of the selected corner point and the transformation relationship between the vehicle-mounted camera group and the vehicle's coordinate system. The system transforms the reference position information of the selected corner point by converting the system relationship. The vehicle-mounted camera group includes multiple cameras with overlapping fields of view. The corner reprojection error calculation process uses the reference position information of the selected corner point and the reference position information of the selected corner point to calculate the reprojection error of the selected corner point. The judgment and adjustment process judges whether the reprojection error of the selected corner point is greater than the preset reprojection error threshold. If it is greater, the extrinsic parameters of the vehicle-mounted camera group are adjusted so that the reprojection error of the selected corner point is not greater than the reprojection error threshold. The process continues to acquire the reference position information of the corner point, the reference position information of the corner point, and the reprojection error calculation process of the corner point for the next corner point in the predetermined daily driving location until the reprojection error of the selected corner point is not greater than the reprojection error threshold to obtain the accurate extrinsic parameters of the vehicle-mounted camera group. Finally, the calibration process uses the accurate extrinsic parameters of the vehicle-mounted camera group to calibrate the extrinsic parameters of the vehicle-mounted camera group.

[0006] Another technical solution adopted in this application is: providing a vehicle sensor calibration device, comprising: an onboard radar calibration module, used to calibrate the extrinsic parameters of the onboard radar using vehicle body structural features, wherein the position information of the vehicle body structural features within the detection range of the onboard radar and in the vehicle coordinate system is obtainable; a corner reference position information acquisition module, used to collect point cloud information of a corner point in a predetermined daily driving environment using an onboard radar with accurate calibration posture during vehicle operation, and obtain the reference position information of the selected corner point based on the point cloud information of the selected corner point and the transformation relationship between the onboard radar and the vehicle coordinate system, wherein the number of corner points included in the predetermined daily driving environment is not less than a preset number threshold; and a corner reference position information acquisition module, used to collect image information of the selected corner point using an onboard camera group configured on the vehicle, and obtain the reference position information of the selected corner point based on the image information of the selected corner point and the transformation relationship between the onboard camera group and the vehicle coordinate system. The system includes: reference position information for a selected corner point, wherein the vehicle-mounted camera group comprises multiple cameras with overlapping fields of view; a corner reprojection error calculation module, used to calculate the reprojection error of the selected corner point using the reference position information and reference position information of the selected corner point; a judgment and adjustment module, used to judge whether the reprojection error of the selected corner point is greater than a preset reprojection error threshold; if it is greater, the system adjusts the extrinsic parameters of the vehicle-mounted camera group so that the reprojection error of the selected corner point is not greater than the reprojection error threshold, and continues to perform the corner reference position information acquisition process, the corner reference position information acquisition process, and the corner reprojection error calculation process for the next corner point in the predetermined daily driving location until the reprojection error of the selected corner point is not greater than the reprojection error threshold to obtain the accurate extrinsic parameters of the vehicle-mounted camera group; and a calibration module, used to calibrate the extrinsic parameters of the vehicle-mounted camera group using the accurate extrinsic parameters of the vehicle-mounted camera group.

[0007] Another technical solution adopted in this application is to provide an in-vehicle system that includes the vehicle sensor calibration device described above.

[0008] Another technical solution adopted in this application is to provide a computer-readable storage medium storing computer instructions that are operated to execute a vehicle sensor calibration method as described above.

[0009] Another technical solution adopted in this application is to provide a vehicle that includes the vehicle-mounted system described above.

[0010] The beneficial effects that the technical solution of this application can achieve are: by using the structural features of the vehicle body to calibrate the external parameters of the vehicle radar, and further by using the corner points in the daily driving environment to calibrate the external parameters of the vehicle camera group, it is possible to complete the automatic calibration of vehicle sensors in a convenient, efficient and low-cost manner. Attached Figure Description

[0011] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1 This is a schematic flowchart illustrating a specific implementation of a vehicle sensor calibration method according to this application.

[0013] Figure 2 This is a schematic diagram of a specific embodiment of a vehicle sensor calibration device according to this application;

[0014] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0015] The preferred embodiments of this application will now be described in detail with reference to the accompanying drawings, so that the advantages and features of this application can be more easily understood by those skilled in the art, thereby providing a clearer and more definite definition of the scope of protection of this application.

[0016] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus that includes the element.

[0017] Autonomous vehicles are typically equipped with multiple sensors to perceive the external road environment, with common sensor types including LiDAR and cameras. Road environment data acquired by sensors such as LiDAR and cameras provides a valuable information source for real-time vehicle positioning, trajectory planning, and motion control. Therefore, the accuracy of sensor information is crucial for the functionality and safety of autonomous vehicles.

[0018] Typically, automotive sensors such as LiDAR and cameras are rigidly mounted on autonomous vehicles. Ideally, once fixed, the relative positions of the sensors and the vehicle body remain unchanged; that is, after installation, only one calibration is needed to establish the coordinate transformation between them. However, in reality, various factors, such as vibrations from vehicle movement, prolonged driving on uneven roads, loosening of the fixing connections between the sensors and the vehicle body, and thermal expansion and contraction of the connections due to temperature changes, can alter the relative positions of the sensors and the vehicle body. In such cases, recalibration of the sensor positions is necessary.

[0019] Existing sensor calibration methods require vehicles to be driven into calibration fields for manual calibration, which is costly to build and time-consuming. Therefore, there is an urgent need for a low-cost and convenient automatic calibration method for vehicle sensors.

[0020] This application calibrates the external parameters of the vehicle-mounted camera group by utilizing corner points in everyday driving environments, which is convenient, efficient, and cost-effective for automatic calibration of the vehicle-mounted camera.

[0021] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.

[0022] Figure 1 This paper illustrates a specific embodiment of a vehicle sensor calibration method according to this application.

[0023] exist Figure 1The vehicle sensor calibration method of this application includes: a vehicle-mounted radar calibration process S101, which calibrates the extrinsic parameters of the vehicle-mounted radar using vehicle body structural features, wherein the vehicle body structural features are within the detection range of the vehicle-mounted radar and their position information in the vehicle coordinate system is obtainable; a corner reference position information acquisition process S102, which, during vehicle operation, uses a vehicle-mounted radar with accurate orientation positioning to collect point cloud information of a corner in a predetermined daily driving environment, and obtains the reference position information of the selected corner based on the point cloud information of the selected corner and the transformation relationship between the vehicle-mounted radar and the vehicle's coordinate system, wherein the number of corners included in the predetermined daily driving environment is not less than a preset number threshold; and a corner reference position information acquisition process S103, which uses a vehicle-mounted camera group configured on the vehicle to collect image information of the selected corner, and obtains the reference position information of the selected corner based on the image information of the selected corner and the transformation relationship between the vehicle-mounted camera group and the vehicle coordinate system. The process includes: S104, which calculates the reprojection error of a selected corner point using the reference position information and the reference position information of the selected corner point; S105, which judges whether the reprojection error of the selected corner point is greater than a preset reprojection error threshold. If it is greater, the extrinsic parameters of the vehicle camera group are adjusted so that the reprojection error of the selected corner point is not greater than the reprojection error threshold. The process continues to acquire the reference position information of the corner point, the reference position information of the corner point, and the reprojection error of the corner point for the next corner point in the predetermined daily driving location until the reprojection error of the selected corner point is not greater than the reprojection error threshold to obtain the accurate extrinsic parameters of the vehicle camera group; and S106, which calibrates the extrinsic parameters of the vehicle camera group using the accurate extrinsic parameters of the vehicle camera group.

[0024] This application calibrates the extrinsic parameters of vehicle-mounted radar by utilizing the structural features of the vehicle body, and further calibrates the extrinsic parameters of the vehicle-mounted camera group by utilizing corner points in daily driving environments. This enables convenient, efficient, and low-cost automatic calibration of vehicle-mounted sensors.

[0025] The vehicle radar calibration process S101, which uses the vehicle body structure characteristics to calibrate the vehicle radar extrinsic parameters, can facilitate the convenient and efficient completion of the vehicle radar extrinsic parameter calibration, and further use the vehicle radar with accurate calibrated extrinsic parameters to calibrate the vehicle camera extrinsic parameters.

[0026] In a specific embodiment of this application, the above-mentioned vehicle radar calibration process S101 further includes a radar pose calibration process, which involves using the vehicle's vehicle radar to acquire point cloud information of calibration reference feature points located within the vehicle radar's monitoring range, and obtaining reference position information of the calibration reference feature points based on the point cloud information of the calibration reference feature points and the vehicle coordinate system transformation relationship between the vehicle radar and the vehicle; acquiring the actual position information of the calibration reference feature points; and calculating the error between the reference position information and the actual position information of the calibration reference feature points to obtain the position error of the vehicle radar, and determining whether the position error of the vehicle radar is greater than a preset position error threshold. If it is greater, the pose of the vehicle radar is recalibrated to obtain a vehicle radar with accurate pose; otherwise, the vehicle radar is determined to be a vehicle radar with accurate pose.

[0027] This embodiment utilizes the inherent structural features of the vehicle body to calibrate the radar pose, making the calibration process efficient, convenient, and cost-effective.

[0028] Specifically, calibration reference feature points include easily identifiable structural features, such as line features or point features. For example, the raised structural outline on the hood, the body outline, or corner points can be selected. For some models, the car logo is located on the hood, so the car logo can also be selected as the calibration reference feature.

[0029] In one specific embodiment of this application, the process of obtaining the true position information of the calibration reference feature points includes obtaining the true position information of the calibration reference feature points through measurement or product manuals, etc. During the vehicle's operation, the point cloud information of a corner point in a predetermined daily driving location is collected using a vehicle-mounted radar with accurate calibration posture. The corner reference position information acquisition process S102, which obtains the reference position information of the selected corner point based on the point cloud information of the selected corner point and the transformation relationship between the vehicle-mounted radar and the vehicle's body coordinate system, facilitates the subsequent calculation of the corner point reprojection error based on the reference position information of the selected corner point. Then, the extrinsic parameters of the corresponding camera group are adjusted according to the corner point reprojection error, thereby realizing the calibration of the extrinsic parameters of the vehicle-mounted camera group.

[0030] In a specific example of this application, the vehicle's speed during operation does not exceed a preset speed threshold. Specifically, the speed threshold can be 20 km / h or 15 km / h.

[0031] Specifically, at lower vehicle speeds, the process of radar and camera acquiring data and synchronizing data calculations can be more accurate.

[0032] In one specific embodiment of this application, the aforementioned daily driving location includes an underground parking garage.

[0033] Specifically, underground parking garages offer a stable environment and numerous structural reference points for calibration, such as the intersections of beams on the garage roof, the intersections of columns with the ground or roof, and wall corners. These can all be used as reference points for camera calibration, ensuring that the number of reference points does not fall below the preset threshold. Furthermore, the time allotted between a vehicle entering the garage, driving to a parking space, and parking itself is sufficient for the camera to complete automatic calibration. Specifically, the preset threshold can be eight reference points.

[0034] In one specific embodiment of this application, before the corner reference position information acquisition process, it is determined whether the number of corners in the daily driving environment is less than a preset number threshold. If it is not less than the preset number threshold, the corner reference position information acquisition process of one corner in the daily driving environment is performed.

[0035] In one specific embodiment of this application, the aforementioned daily driving location can be a street area where vehicles frequently pass through a relatively dense number of corners and where there is no minimum speed limit.

[0036] In one specific embodiment of this application, the vehicle's driving process includes the process of returning to the underground parking garage. Specifically, the vehicle's speed is typically slow after entering the garage, which satisfies the requirement for radar and cameras to obtain accurate corner position information.

[0037] In a specific example of this application, when the vehicle is stationary, corner information collected by the vehicle-mounted radar and camera group at the same location needs to be deleted. Specifically, corner position information at different locations facilitates subsequent iterative optimization of the vehicle-mounted camera's extrinsic parameters.

[0038] The corner reference position information acquisition process S103, which uses the vehicle-mounted camera group to collect image information of selected corner points and obtains reference position information of selected corner points based on the image information of selected corner points and the transformation relationship between the vehicle-mounted camera group and the vehicle body coordinate system, facilitates the subsequent calculation of corner point reprojection error based on the corner point reference position information of selected corner points. Then, the extrinsic parameters of the corresponding camera group are adjusted according to the corner point reprojection error, thereby realizing the calibration of the extrinsic parameters of the vehicle-mounted camera group.

[0039] In one specific embodiment of this application, the above-mentioned vehicle-mounted camera group can be multiple vehicle-mounted camera groups or a single vehicle-mounted camera group.

[0040] In one specific example of this application, each of the above-mentioned vehicle-mounted camera groups includes two vehicle-mounted cameras.

[0041] Specifically, autonomous vehicles typically have multiple (three or more) onboard cameras. Since the positional relationship between the cameras is fixed, the multiple cameras can be divided into several groups. Each camera group includes two cameras installed at different locations on the vehicle. This setup allows for a constraint relationship between the images captured by any two onboard cameras in each camera group, and also allows for the acquisition of depth information from the images, which is something that a single camera cannot do.

[0042] In optional embodiments of this application, the two vehicle-mounted cameras at the front of the vehicle can be divided into the same vehicle-mounted camera group, or the one vehicle-mounted camera at the front and one vehicle-mounted camera at the side can be divided into the same vehicle-mounted camera group, so as to avoid the images captured by the same vehicle-mounted camera group having no overlapping area and thus being unable to obtain the depth information of the image.

[0043] The corner reprojection error calculation process S104, which uses the reference position information of the selected corner point to calculate the reprojection error of the selected corner point, can facilitate the adjustment of the extrinsic parameters of the corresponding camera group based on the corner reprojection error, thereby realizing the calibration of the extrinsic parameters of the vehicle-mounted camera group.

[0044] Specifically, by using corner reprojection error as an observation, the extrinsic parameters of the corresponding camera group can be adjusted according to the magnitude of the reprojection error through iterative optimization.

[0045] In one specific embodiment of this application, the selected corner reprojection error includes the reprojection error between the vehicle-mounted radar and a vehicle-mounted camera group. This embodiment is applicable to situations where the extrinsic parameters of a vehicle-mounted camera group are calibrated at a time.

[0046] In one specific embodiment of this application, the selected corner reprojection error includes the reprojection error between the vehicle radar and multiple vehicle camera groups. This embodiment is suitable for situations where the extrinsic parameters of multiple vehicle camera groups are calibrated at once, which can facilitate the more efficient calibration of vehicle camera extrinsic parameters.

[0047] The process involves determining whether the reprojection error of the selected corner point is greater than a preset reprojection error threshold. If it is, the extrinsic parameters of the vehicle-mounted camera group are adjusted so that the reprojection error of the selected corner point is not greater than the reprojection error threshold. The process continues to acquire the corner point reference position information, corner point reference position information, and corner point reprojection error for the next corner point in the predetermined daily driving location. This process continues until the reprojection error of the selected corner point is not greater than the reprojection error threshold, thus obtaining the accurate extrinsic parameters of the vehicle-mounted camera group. In step S105, the extrinsic parameters of the corresponding camera group are adjusted based on the corner point reprojection error and the preset reprojection error threshold, thereby achieving the calibration of the extrinsic parameters of the vehicle-mounted camera group.

[0048] In one specific embodiment of this application, it is determined whether the reprojection error between the vehicle radar and a vehicle camera group is greater than a preset reprojection error threshold. If it is greater, the extrinsic parameters of the vehicle camera group are adjusted so that the reprojection error of the selected corner point is not greater than the reprojection error threshold.

[0049] In one specific embodiment of this application, it is determined whether the reprojection error between the vehicle radar and multiple vehicle camera groups is greater than a preset reprojection error threshold. If it is greater, the extrinsic parameters of the corresponding vehicle camera group are adjusted so that the reprojection error of the selected corner point is not greater than the reprojection error threshold, which is conducive to completing the calibration of the vehicle camera extrinsic parameters more efficiently.

[0050] In a specific example of this application, a corner point's point cloud information is acquired using a vehicle-mounted radar with accurate pose. The reference position information (X, Y, Z) of the corner point in the vehicle coordinate system is obtained using the transformation relationship between the radar and the vehicle coordinate system. An image containing the corner point is captured using a vehicle-mounted camera. A preset transformation matrix between the radar and the vehicle coordinate system is used to obtain the reference position information (X', Y', Z') of the corner point in the vehicle coordinate system acquired by the camera group. The selected corner point reprojection error is calculated based on the reference position information (X, Y, Z) and the reference position information (X', Y', Z'). Using the selected corner point reprojection error as an observation, the extrinsic parameters of the camera group are adjusted iteratively according to the magnitude of the reprojection error until the reprojection error reaches a reprojection error threshold. Before the reprojection error reaches the threshold, new corner points are continuously selected, and the above iterative optimization process is repeated until the reprojection error meets the threshold condition, thus completing the calibration.

[0051] The calibration process S106, which uses the accurate extrinsic parameters of the vehicle-mounted camera group to calibrate the extrinsic parameters of the vehicle-mounted camera group, can ultimately reduce system complexity, reduce computational load, reduce system latency, and improve user experience.

[0052] In one specific embodiment of this application, the vehicle sensor calibration method of this application further includes an automatic calibration triggering process, which determines whether the current driving location of the vehicle is a predetermined daily driving location. If the current driving location is a predetermined daily driving location, then the process of obtaining corner reference position information, obtaining corner reference position information, calculating corner reprojection error, determining and adjusting, and calibrating are performed; otherwise, no process is performed.

[0053] In one specific embodiment of this application, the above-mentioned automatic calibration triggering process further includes determining whether the calibration interval between the current time and the calibration time of the last calibration of the external parameters of the vehicle camera group is greater than a preset time threshold. If the current driving location is a predetermined daily driving location and the calibration interval is greater than the time threshold, then a corner reference position information acquisition process, a corner reference position information acquisition process, a corner reprojection error calculation process, a judgment and adjustment process, and a calibration process are performed; otherwise, no process is performed.

[0054] Specifically, the preset duration threshold can be 1 week.

[0055] The preset calibration trigger conditions enable the calibration of the vehicle-mounted camera group to be performed dynamically and automatically, avoiding safety accidents caused by untimely calibration due to human factors. Furthermore, the trigger conditions are flexible and controllable and can be set according to actual needs.

[0056] In one specific embodiment of this application, the vehicle sensor calibration method further includes a calibration failure data storage and utilization process. If the reprojection error of the selected corner point of the last corner point collected during a single passage of the vehicle through a predetermined driving location is greater than the reprojection error threshold, the adjusted extrinsic parameters corresponding to the last corner point collected during this process are stored. Furthermore, when the vehicle passes through the predetermined driving location again, the adjusted extrinsic parameters corresponding to the last corner point collected during this process are used to obtain the corner point reference position information.

[0057] Specifically, this embodiment is applicable to situations where the calibration task does not need to be completed in one go. If a sufficient number of corner points are not obtained during the first entry into the parking garage, the obtained corner point information can be stored and used together with the corner point information obtained during the next entry into the parking garage for a single calibration. This can save computational load and improve calibration efficiency.

[0058] In one specific embodiment of this application, the vehicle sensor calibration method of this application further includes an identifiable prompting process, wherein if the reprojection error of the selected corner point is greater than the reprojection error threshold, an identifiable prompt is given.

[0059] Specifically, this embodiment is applicable to calibration tasks that need to be completed in one go. If the reprojection error of the selected corner point is greater than the reprojection error threshold, it means that the number of currently selected corner points is insufficient. The driver can be prompted to continue driving the vehicle until enough corner points are obtained.

[0060] Figure 2 This application illustrates a vehicle sensor calibration device.

[0061] exist Figure 2The vehicle sensor calibration device shown includes: an onboard radar calibration module 201, used to calibrate the extrinsic parameters of the onboard radar using vehicle body structural features, wherein the position information of the vehicle body structural features within the detection range of the onboard radar and in the vehicle coordinate system is obtainable; a corner reference position information acquisition module 202, used to collect point cloud information of a corner point in a predetermined daily driving environment using an onboard radar with accurate calibration posture during vehicle operation, and obtain the reference position information of the selected corner point based on the point cloud information of the selected corner point and the transformation relationship between the onboard radar and the vehicle's coordinate system, wherein the number of corner points included in the predetermined daily driving environment is not less than a preset number threshold; and a corner reference position information acquisition module 203, used to collect image information of the selected corner point using an onboard camera group configured on the vehicle, and obtain the reference position information of the selected corner point based on the image information of the selected corner point and the transformation relationship between the onboard camera group and the vehicle coordinate system. The vehicle-mounted camera group includes multiple cameras with overlapping fields of view; a corner reprojection error calculation module 204 is used to calculate the reprojection error of the selected corner using the reference position information and the reference position information of the selected corner; a judgment and adjustment module 205 is used to judge whether the reprojection error of the selected corner is greater than a preset reprojection error threshold. If it is greater, the extrinsic parameters of the vehicle-mounted camera group are adjusted so that the reprojection error of the selected corner is not greater than the reprojection error threshold, and the process of obtaining the reference position information, the reference position information, and the reprojection error is continued for the next corner in the predetermined daily driving location until the reprojection error of the selected corner is not greater than the reprojection error threshold to obtain the accurate extrinsic parameters of the vehicle-mounted camera group; and a calibration module 206 is used to calibrate the extrinsic parameters of the vehicle-mounted camera group using the accurate extrinsic parameters of the vehicle-mounted camera group.

[0062] This application calibrates the extrinsic parameters of vehicle-mounted radar by utilizing the structural features of the vehicle body, and further calibrates the extrinsic parameters of the vehicle-mounted camera group by utilizing corner points in daily driving environments. This enables convenient, efficient, and low-cost automatic calibration of vehicle-mounted sensors.

[0063] The vehicle radar calibration module 201 is used to calibrate the extrinsic parameters of vehicle radar using the structural features of the vehicle body. It can facilitate the convenient and efficient calibration of the extrinsic parameters of vehicle radar, and further use the vehicle radar with accurate calibrated extrinsic parameters to calibrate the extrinsic parameters of vehicle camera.

[0064] The corner reference position information acquisition module 202 is used to accurately collect point cloud information of a corner point in a predetermined daily driving location during vehicle operation using a calibrated positioning attitude, and obtain the reference position information of the selected corner point based on the point cloud information of the selected corner point and the transformation relationship between the vehicle radar and the vehicle body coordinate system. This facilitates the subsequent calculation of the corner point reprojection error based on the reference position information of the selected corner point, and then adjusts the extrinsic parameters of the corresponding camera group according to the corner point reprojection error, thereby realizing the calibration of the extrinsic parameters of the vehicle camera group.

[0065] The corner reference position information acquisition module 203 is used to acquire image information of selected corner points using the vehicle-mounted camera group configured on the vehicle, and obtain the reference position information of the selected corner points based on the image information of the selected corner points and the transformation relationship between the vehicle-mounted camera group and the vehicle body coordinate system. This facilitates the subsequent calculation of the corner reprojection error based on the corner reference position information of the selected corner points, and then adjusts the extrinsic parameters of the corresponding camera group according to the corner reprojection error, thereby realizing the calibration of the extrinsic parameters of the vehicle-mounted camera group.

[0066] The corner reprojection error calculation module 204 is used to calculate the reprojection error of the selected corner point by using the reference position information of the selected corner point and the reference position information of the selected corner point. It can facilitate the adjustment of the extrinsic parameters of the corresponding camera group according to the corner reprojection error, and realize the calibration of the extrinsic parameters of the vehicle-mounted camera group.

[0067] The module 205 is used to determine whether the reprojection error of the selected corner point is greater than a preset reprojection error threshold. If it is greater, the extrinsic parameters of the vehicle camera group are adjusted so that the reprojection error of the selected corner point is not greater than the reprojection error threshold. The module then continues to acquire the corner reference position information, the corner reference position information, and the corner reprojection error calculation for the next corner point in the predetermined daily driving location until the reprojection error of the selected corner point is not greater than the reprojection error threshold. This module can adjust the extrinsic parameters of the corresponding camera group based on the corner reprojection error and the preset reprojection error threshold, thereby achieving the calibration of the extrinsic parameters of the vehicle camera group.

[0068] The calibration module 206 is used to calibrate the extrinsic parameters of the vehicle camera group using accurate extrinsic parameters. This can ultimately reduce system complexity, reduce computational load, reduce system latency, and improve user experience.

[0069] The vehicle sensor calibration device provided in this application can be used to perform the vehicle sensor calibration method in any of the above embodiments. Its principle and technical effect are similar, and will not be described again here.

[0070] In one specific embodiment of this application, the functional modules of the vehicle sensor calibration device of this application may be directly in hardware, in software modules executed by a processor, or in a combination of both.

[0071] Software modules may reside in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disks, removable disks, CD-ROMs, or any other form of storage medium known in this art. An exemplary storage medium is coupled to the processor, enabling the processor to read information from and write information to the storage medium.

[0072] The processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic, discrete hardware components, or any combination thereof. A general-purpose processor can be a microprocessor, but alternatively, it can be any conventional processor, controller, microcontroller, or state machine. The processor can also be implemented as a combination of computing devices, such as a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors incorporating a DSP core, or any other such configuration. Alternatively, the storage medium can be integrated with the processor. The processor and storage medium can reside in an ASIC. The ASIC can reside in the user terminal. Alternatively, the processor and storage medium can reside as discrete components in the user terminal.

[0073] In one specific embodiment of this application, an in-vehicle system includes the vehicle sensor calibration device described above.

[0074] In one specific embodiment of this application, a vehicle is provided that includes the above-described vehicle-mounted system.

[0075] In another specific embodiment of this application, a computer-readable storage medium is provided, which stores computer instructions that are operated to perform the vehicle sensor calibration method in any of the above embodiments.

[0076] In another specific embodiment of this application, a computer device includes a processor and a memory, the memory storing computer instructions that are operated to perform the vehicle sensor calibration method in any of the above embodiments.

[0077] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0078] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0079] The above are merely embodiments of this application and do not limit the scope of this patent application. Any equivalent structural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of this application.

Claims

1. A method for calibrating vehicle sensors, characterized in that, include: The vehicle-mounted radar calibration process utilizes vehicle body structural features to calibrate the vehicle-mounted radar's extrinsic parameters. The vehicle body structural features are located within the detection range of the vehicle-mounted radar and their position information in the vehicle coordinate system can be obtained. The corner reference position information acquisition process involves using the vehicle-mounted radar with accurate positioning during vehicle operation to collect point cloud information of a corner in a predetermined daily driving location. The reference position information of the selected corner is obtained based on the point cloud information of the selected corner and the transformation relationship between the vehicle-mounted radar and the vehicle's body coordinate system. The number of corners included in the predetermined daily driving location is not less than a preset number threshold. The corner reference position information acquisition process involves using the vehicle-mounted camera group configured on the vehicle to collect image information of the selected corner point, and obtaining the reference position information of the selected corner point based on the image information of the selected corner point and the transformation relationship between the vehicle-mounted camera group and the vehicle coordinate system. The vehicle-mounted camera group includes multiple cameras with overlapping fields of view. The corner reprojection error calculation process involves using the reference position information of the selected corner point and the reference position information of the selected corner point to calculate the reprojection error of the selected corner point. The adjustment process involves determining whether the reprojection error of the selected corner point exceeds a preset reprojection error threshold. If it does, the extrinsic parameters of the vehicle-mounted camera group are adjusted to ensure that the reprojection error of the selected corner point does not exceed the threshold. The process then continues to acquire the corner reference position information, calculate the corner reprojection error, and perform the corner reference position information acquisition process for the next corner point in the predetermined daily driving location until the reprojection error of the selected corner point does not exceed the threshold, thus obtaining the accurate extrinsic parameters of the vehicle-mounted camera group. The calibration process involves using the accurate extrinsic parameters of the vehicle-mounted camera group to calibrate its extrinsic parameters.

2. The vehicle sensor calibration method according to claim 1, characterized in that, Also includes: The calibration is automatically triggered. The system determines whether the current driving location of the vehicle is the predetermined daily driving location. If the current driving location is the predetermined daily driving location, the system performs the following processes: corner reference position information acquisition, corner reprojection error calculation, judgment and adjustment, and calibration. Otherwise, the process is not performed.

3. The vehicle sensor calibration method according to claim 2, characterized in that, The automatic calibration triggering process further includes determining whether the calibration interval between the current time and the last calibration time of the external parameters of the vehicle-mounted camera group is greater than a preset time threshold. If the current driving location is the predetermined daily driving location and the calibration interval is greater than the time threshold, then the following processes are performed: obtaining corner reference position information, obtaining corner reference position information, calculating corner reprojection error, making a judgment and adjustment, and performing the calibration process. Otherwise, the following processes are not performed.

4. The vehicle sensor calibration method according to claim 1, characterized in that, Also includes: The calibration failure data storage and utilization process is as follows: if the reprojection error of the selected corner point of the last corner point collected during the single passage of the vehicle through the predetermined driving location is greater than the reprojection error threshold, then the adjusted extrinsic parameters corresponding to the last corner point of the current collection are stored. as well as When the vehicle passes through the predetermined driving location again, the corner reference position information is obtained by using the adjusted extrinsic parameters corresponding to the last corner point collected in this study.

5. The vehicle sensor calibration method according to claim 1, characterized in that, Also includes: The process of providing identifiable prompts involves providing a prompt if the reprojection error of the selected corner point is greater than the reprojection error threshold.

6. The vehicle sensor calibration method according to claim 1, characterized in that, The vehicle-mounted radar calibration process further includes: In the radar pose calibration process, the vehicle's onboard radar is used to acquire point cloud information of calibration reference feature points located within the detection range of the onboard radar. The reference position information of the calibration reference feature points is obtained based on the point cloud information of the calibration reference feature points and the transformation relationship between the onboard radar and the vehicle's body coordinate system. Obtain the true location information of the calibration reference feature points; and, The position error of the vehicle-mounted radar is obtained by calculating the error between the reference position information and the actual position information of the calibration reference feature point. It is then determined whether the position error of the vehicle-mounted radar is greater than a preset position error threshold. If it is greater, the pose of the vehicle-mounted radar is recalibrated to obtain the vehicle-mounted radar with accurate longitude and longitude positioning. Otherwise, the vehicle-mounted radar is identified as the vehicle-mounted radar with accurate longitude and longitude positioning.

7. A vehicle sensor calibration device, characterized in that, include: The vehicle-mounted radar calibration module is used to calibrate the extrinsic parameters of the vehicle-mounted radar using vehicle body structural features, wherein the position information of the vehicle body structural features within the detection range of the vehicle-mounted radar and in the vehicle coordinate system can be obtained. The corner reference position information acquisition module is used to collect point cloud information of a corner in a predetermined daily driving location using the vehicle-mounted radar with accurate positioning during vehicle operation, and obtain the reference position information of the selected corner based on the point cloud information of the selected corner and the transformation relationship between the vehicle-mounted radar and the vehicle's body coordinate system, wherein the number of corners included in the predetermined daily driving location is not less than a preset number threshold. The corner reference position information acquisition module is used to acquire image information of the selected corner point using the vehicle-mounted camera group configured on the vehicle, and obtain the reference position information of the selected corner point based on the image information of the selected corner point and the transformation relationship between the vehicle-mounted camera group and the vehicle coordinate system, wherein the vehicle-mounted camera group includes multiple cameras with overlapping fields of view; The corner reprojection error calculation module is used to calculate the reprojection error of the selected corner point using the reference position information of the selected corner point and the reference position information of the selected corner point. The judgment and adjustment module is used to determine whether the reprojection error of the selected corner point is greater than a preset reprojection error threshold. If it is greater, the extrinsic parameters of the vehicle-mounted camera group are adjusted so that the reprojection error of the selected corner point is not greater than the reprojection error threshold. The process of acquiring the corner point reference position information, calculating the corner point reprojection error, and obtaining the corner point reference position information for the next corner point in the predetermined daily driving location continues until the reprojection error of the selected corner point is not greater than the reprojection error threshold, thus obtaining the accurate extrinsic parameters of the vehicle-mounted camera group. The calibration module is used to calibrate the extrinsic parameters of the vehicle-mounted camera group using the accurate extrinsic parameters of the vehicle-mounted camera group.

8. A vehicle-mounted system, characterized in that, The vehicle system includes the vehicle sensor calibration device as described in claim 7.

9. A computer-readable storage medium storing computer instructions, characterized in that, The computer instructions are operated to perform the vehicle sensor calibration method according to any one of claims 1-6.

10. A vehicle, characterized in that, Including the vehicle-mounted system as described in claim 8.

Images