Calibration method, storage medium, electronic device and vehicle

Abstract

The application relates to a calibration method, a storage medium, an electronic device and a vehicle. The calibration method comprises the following steps: in response to a calibration trigger instruction, a plurality of sets of calibration board data are collected, each set of calibration board data comprises laser radar point cloud data and camera image data; the relative external parameters between the laser radar and the camera are calculated based on the point cloud data and the camera image data; and the relative external parameters are updated to the coordinate system of the vehicle to complete the calibration. Based on the above, the application can realize rapid and efficient joint calibration of the laser radar and the camera, the process is simple and general, complex calibration equipment and professional sites are not required, and the application is suitable for the needs of the after-sales scene.

Description

Technical Field

[0001] This application relates to the field of calibration technology, and more particularly to a calibration method, storage medium, electronic device, and vehicle. Background Technology

[0002] With the widespread adoption of intelligent assisted driving technology, the relative poses of LiDAR and cameras are prone to shift after vehicle maintenance, collision repair, or sensor replacement, requiring recalibration to ensure perception accuracy. Existing calibration solutions mostly rely on professional calibration labs, which are costly and complex, making them unsuitable for ordinary after-sales scenarios such as 4S stores. Simple calibration methods suffer from insufficient calibration accuracy, poor robustness in point cloud feature extraction, reliance on specific coding markers for image features, and weak synchronization of multi-sensor data. They cannot simultaneously meet the requirements of low cost, ease of operation, high precision, and strong robustness, severely hindering the large-scale implementation of vehicle after-sales calibration. Summary of the Invention

[0003] This application provides a calibration method, storage medium, electronic device, and vehicle, which can solve at least one technical problem in the prior art.

[0004] Accordingly, this application provides a calibration method, which includes: in response to a calibration trigger command, collecting multiple sets of calibration board data, each set of calibration board data including lidar point cloud data and camera image data; calculating the relative extrinsic parameters between the lidar and the camera based on the point cloud data and the camera image data; and updating the relative extrinsic parameters to the vehicle's coordinate system to complete the calibration.

[0005] In one embodiment of this application, the step of calculating the relative extrinsic parameters between the lidar and the camera based on the point cloud data and the camera image data includes: extracting the three-dimensional coordinates of the corresponding features of the calibration board based on the point cloud data, and extracting the two-dimensional pixel coordinates of the corresponding features of the calibration board based on the camera image data; and calculating the relative extrinsic parameters based on the three-dimensional coordinates and the two-dimensional pixel coordinates.

[0006] In one embodiment of this application, the step of extracting the three-dimensional coordinates of the calibration board features based on the point cloud data includes: filtering the point cloud data to obtain a point cloud region containing the calibration board; fitting the point cloud data within the point cloud region to obtain a calibration board plane, and transforming the calibration board plane to a preset reference plane; and determining the three-dimensional coordinates based on the point cloud data of the transformed calibration board plane.

[0007] In one embodiment of this application, the step of extracting the two-dimensional pixel coordinates of the calibration board corresponding to the camera image data includes: preprocessing the camera image data to obtain the image region of the calibration board; and calculating the two-dimensional pixel coordinates based on the image region of the calibration board and the preset size information of the calibration board.

[0008] In one embodiment of this application, the step of calculating the relative extrinsic parameters based on the three-dimensional coordinates and the two-dimensional pixel coordinates includes: solving for initial extrinsic parameters based on the three-dimensional coordinates and the two-dimensional pixel coordinates; and optimizing the initial extrinsic parameters based on the reprojection error to obtain the relative extrinsic parameters.

[0009] In one embodiment of this application, the multiple sets of calibration board data are obtained by placing the same calibration board at multiple preset positions within the common field of view of the lidar and the camera.

[0010] In one embodiment of this application, the calibration plate includes visual markers for positioning and a plurality of hollowed-out circular holes. The calibration plate is arranged perpendicular to the ground and rotated relative to the vehicle by a preset angle.

[0011] Accordingly, embodiments of this application provide an electronic device, including a processor and a memory, wherein the memory stores a computer program, and the processor executes the computer program to implement the calibration method described in any of the above claims.

[0012] Accordingly, embodiments of this application provide a computer storage medium, wherein the computer program, when executed by a processor, implements the calibration method described in any of the above claims.

[0013] Accordingly, embodiments of this application provide an electronic device, including a processor and a memory, wherein the memory stores a computer program, and the processor executes the computer program to implement the calibration method described in any of the above claims.

[0014] Accordingly, this application provides a vehicle that includes a lidar, a camera, and the aforementioned electronic equipment.

[0015] This application provides a calibration method, storage medium, electronic device, and vehicle. By responding to a calibration trigger command, it collects synchronized LiDAR point cloud and camera image data, calculates relative extrinsic parameters based on the two types of data, and updates them to the vehicle coordinate system. This achieves rapid and efficient joint calibration of LiDAR and camera, with a simple and universal process that does not require complex calibration equipment or specialized sites. It is suitable for after-sales scenarios, can quickly restore sensor pose accuracy, and ensures the reliability and safety of intelligent driving perception systems. At the same time, it has strong superordinate capabilities, is compatible with various calibration detail optimization schemes, and has a wide protection range. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a flowchart illustrating one embodiment of the calibration method of this application;

[0018] Figure 2 This is a schematic diagram of one embodiment of the calibration device of this application;

[0019] Figure 3 This is a schematic diagram of one embodiment of the calibration system of this application;

[0020] Figure 4 This is a structural schematic diagram of one embodiment of the visual marker of this application;

[0021] Figure 5 This is a flowchart illustrating one embodiment of step S200 of this application;

[0022] Figure 6 This is a flowchart illustrating one embodiment of step S210 of this application;

[0023] Figure 7 This is a schematic diagram of one embodiment of point cloud data in this application;

[0024] Figure 8 This is a flowchart illustrating another embodiment of step S210 of this application;

[0025] Figure 9 This is a flowchart illustrating another embodiment of step S220 of this application;

[0026] Figure 10 This is a schematic diagram of the structure of one embodiment of the electronic device of this application;

[0027] Figure 11 This is a structural schematic diagram of one embodiment of the vehicle described in this application. Detailed Implementation

[0028] The present application will now be described in further detail with reference to the accompanying drawings and embodiments. It should be noted that the following embodiments are merely illustrative of the present application and do not limit its scope. Similarly, the following embodiments are only some, not all, embodiments of the present application, and all other embodiments obtained by those skilled in the art without inventive effort are within the scope of protection of this application.

[0029] In the description of this application, it should be understood that the terms "upper," "lower," "left," "right," "front," "rear," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or relative positional relationship shown in the accompanying drawings. They are used only for the convenience of describing this application and for simplification, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this application. Unless otherwise specified, the above-mentioned orientational descriptions can be flexibly set in practical applications, provided that the relative positional relationships shown in the accompanying drawings are satisfied.

[0030] The terms "first" and "second" are configured for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this application, unless otherwise stated, "a plurality of" means two or more.

[0031] In the description of this application, it should be noted that, unless otherwise expressly specified and limited, the terms "installation," "connection," "linking," and "communication" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection. They can refer to a direct connection or an indirect connection through an intermediate medium, or a connection within two components. Those skilled in the art can understand the specific meaning of the above terms in this application according to the specific circumstances.

[0032] In embodiments of this application, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, article, or apparatus that includes that element.

[0033] In the embodiments of this application, the terms "exemplary" or "for example" are used to indicate that something is an example, illustration, or description. Any embodiment or design that is described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design. Specifically, the use of the terms "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0034] The following is a detailed analysis of the proposed solution with reference to the accompanying drawings:

[0035] Please see Figure 1 , Figure 1This is a flowchart illustrating one embodiment of the calibration method of this application, as shown below. Figure 1 The calibration method of this application includes the following steps:

[0036] S100, in response to calibration trigger command, collects multiple sets of calibration board data, each set of calibration board data including LiDAR point cloud data and camera image data.

[0037] It is understood that the calibration method described in this application can be applied in after-sales service. When a vehicle undergoes repair, replacement, or repositioning of its LiDAR or camera in an after-sales scenario, the repair personnel establish a communication connection between the vehicle and the after-sales diagnostic system (VDS). The VDS then issues calibration trigger commands through its human-machine interface. The VDS is a dedicated diagnostic device and software system developed by the OEM for after-sales scenarios, typically consisting of a diagnostic host, a dedicated diagnostic interface, diagnostic software, and a human-machine interface. It communicates with the vehicle domain controller and sensor ECUs via wired / wireless means to trigger the calibration process, issue commands, collect status data, display progress, issue calibration parameters, and provide feedback on calibration results. It serves as the operation entry point and control center for after-sales calibration.

[0038] Furthermore, after the vehicle domain controller receives the calibration trigger command from the vehicle, it automatically activates the data acquisition functions of the LiDAR and the vehicle camera, enabling the two sensors to enter a synchronous acquisition state.

[0039] During the calibration process, based on real-time prompts from the VDS system, the calibration board can be placed sequentially at multiple preset calibration positions within the shared field of view of the LiDAR and camera. In other words, in this embodiment, multiple sets of calibration board data are obtained by placing the same calibration board at multiple preset positions within the shared field of view of the LiDAR and camera. Furthermore, at each preset position, the LiDAR and camera simultaneously acquire data from the same calibration board, ensuring that each set of calibration board data includes LiDAR point cloud data and camera image data from the same time, scene, and pose. This guarantees strict temporal and spatial alignment between 3D spatial information and 2D image information, avoiding feature matching errors caused by asynchronous acquisition or inconsistent scenes, and providing stable, reliable, and highly consistent raw data for subsequent extrinsic parameter calculation.

[0040] In the above implementation, the calibration process is uniformly triggered by the VDS system to achieve synchronous data acquisition between the lidar and the camera, ensuring that each set of data is highly aligned in time and space, effectively avoiding calculation errors caused by acquisition timing deviations or scene differences, and improving the stability and reliability of the entire calibration process.

[0041] It is understood that the calibration method in this application only requires one calibration plate to complete the joint calibration, which is simple to operate and can achieve the calibration accuracy of the calibration interval. Further combining... Figure 2 , Figure 2 This is a schematic diagram of one embodiment of the calibration device of this application, as shown below. Figure 2 Specifically, the calibration device 100 in this embodiment includes a movable bracket 110 and a replaceable calibration plate 120.

[0042] The movable bracket 110 is equipped with casters at its bottom, allowing it to move as needed. Figure 3 The vehicle was smoothly pushed between the 1-5 marked positions, among which... Figure 3 This is a schematic diagram of one embodiment of the calibration system of this application. In this embodiment, the movable bracket 110 adopts a lifting structure, and the height of the calibration plate 120 is adjusted according to the sensor installation height of different vehicle models, so that the calibration plate is always in the common field of view of the camera and the lidar; the calibration plate adopts a quick-release structure, which can be directly disassembled and replaced.

[0043] In the above embodiments, the bracket is highly versatile and easy to move, and can be adapted to different vehicle models and sensor heights. It supports the expansion of other sensor calibration functions by simply replacing the calibration board, which greatly reduces hardware costs and waste in later upgrades.

[0044] Further reading Figure 2 The calibration plate 120 includes visual markers A for positioning and multiple hollow circular holes B. The visual markers on the calibration plate 120 can be configured as coded patterns for quickly and robustly locating the position, orientation, and scale of the calibration plate in camera images. The coded patterns provide stable and detectable corner features, which determine the outer boundary, perspective distortion, and spatial orientation of the calibration plate. Combined with the preset physical dimensions of the calibration plate, the two-dimensional pixel coordinates of the center of the hollow circular holes in the image can be accurately calculated, avoiding the errors and instabilities caused by directly detecting the holes and improving calibration accuracy and robustness.

[0045] In specific implementations, visual markers may include, but are not limited to, ArUco codes, AprilTag codes, checkerboard patterns, CharuCo boards, circular array codes, or custom visual coding markers, as long as they can be stably detected in the image and used for calibration board positioning. This application does not impose any limitations on this. Figure 4 , Figure 4 This is a schematic diagram of a structural embodiment of the visual marker of this application. In a specific application scenario of this application, the visual marker A can be as follows: Figure 4 The ArUco code or AprilTag code shown.

[0046] Furthermore, the calibration plate 120 can be arranged perpendicular to the ground and rotated at a preset angle relative to the vehicle. Setting the calibration plate perpendicular to the ground ensures that the surface of the calibration plate is substantially perpendicular to the main optical axis of the LiDAR and the camera, guaranteeing that the LiDAR point cloud can uniformly illuminate the surface of the calibration plate. This avoids uneven point cloud distribution, difficulties in planar extraction, or distortion of circular hole features caused by the calibration plate tilting or falling over. At the same time, it allows the camera to capture a complete calibration plate area without severe perspective distortion, which is beneficial to improving the stability and accuracy of image feature detection.

[0047] Simultaneously, the calibration board is rotated relative to the vehicle by a preset angle (e.g., 45°) to ensure that it is neither parallel nor perpendicular to the edge of the camera image. This avoids problems such as blurred corner point positioning and degraded attitude calculation caused by the image edge being parallel to the calibration board edge. By rotating it by a certain angle, the calibration board can be arranged in an oblique position with a clear perspective relationship in the image, providing stronger geometric constraints for camera attitude calculation and improving the stability and accuracy of calibration board pose estimation.

[0048] Furthermore, the oblique layout enables the lidar point cloud to form a richer point distribution on the calibration plate plane, avoiding insufficient plane fitting accuracy caused by the point cloud being arranged in a single direction, and further improving the reliability of point cloud plane extraction, edge segmentation and circle center coordinate calculation.

[0049] In summary, setting the calibration board to be perpendicular to the ground and rotated at a preset angle relative to the vehicle can simultaneously optimize the quality of the lidar point cloud distribution and the observation conditions of the camera image features. This allows both sensors to acquire observation data with sufficient geometric constraints, low distortion, and clear features, thereby significantly improving the robustness of the joint calibration and the accuracy of the final extrinsic parameter calculation.

[0050] S200 calculates the relative extrinsic parameters between the lidar and the camera based on point cloud data and camera image data.

[0051] Please refer to further information. Figure 5 , Figure 5 This is a flowchart illustrating an implementation method of step S200 of this application, as shown below. Figure 5 Step S200 of this application further includes the following sub-steps:

[0052] S210 extracts the 3D coordinates of the corresponding features of the calibration board based on point cloud data, and extracts the 2D pixel coordinates of the corresponding features of the calibration board based on camera image data.

[0053] Please combine further Figure 6 , Figure 6 This is a flowchart illustrating an embodiment of step S210 of this application, as shown below. Figure 6 Step S210 of this application further includes the following sub-steps:

[0054] S211, filter the point cloud data to obtain the point cloud region containing the calibration board.

[0055] Understandably, after acquiring the LiDAR point cloud data, the raw point cloud data is first preprocessed. By using a pass-through filter, reasonable numerical ranges are set in the X, Y, and Z dimensions to filter out background points, ground points, and distant noise points that are far from the calibration board. Only the point cloud of the region of interest (ROI), including the calibration board, is retained, reducing the interference of invalid data on subsequent calculations and improving processing efficiency.

[0056] S212, Fit the point cloud data within the point cloud region to obtain the calibration plate plane, and transform the calibration plate plane to a preset reference plane.

[0057] Furthermore, based on the point cloud region of the calibration board obtained after filtering, the RANSAC algorithm is used to perform plane fitting processing on the point cloud data in this region. This robustly identifies and separates the planar model where the calibration board is located from the actual point cloud data containing noise, speckles, and local missing data, and determines the position and orientation information of the calibration board in three-dimensional space.

[0058] Furthermore, after obtaining the calibration plate plane, the plane is uniformly transformed to the preset reference plane of Z=0 through coordinate transformation, thereby eliminating the coordinate deviation caused by the difference in the actual placement posture of the calibration plate, unifying the coordinate system used for subsequent feature extraction and calculation, and ensuring the consistency and accuracy of feature calculation.

[0059] S213, determine the three-dimensional coordinates based on the point cloud data of the transformed calibration plate plane.

[0060] Furthermore, after completing the planar coordinate transformation, edge detection and clustering processing are performed on the point cloud data within the calibration plate plane. The overall outer contour boundary of the calibration plate and the set of inner edge points corresponding to each hollowed-out circular hole are extracted from the point cloud. The center point is then calculated based on the geometric distribution characteristics of the edge points. Further, combined with... Figure 7 , Figure 7 This is a schematic diagram illustrating one implementation method of point cloud data in this application, specifically for... Figure 7 To address issues such as stringiness, incompleteness, and irregular outlines of circular holes that are common in actual point clouds, a robust extraction method compatible with non-ideal edges is adopted. This method can stably obtain the 3D coordinates of the center of each circular hole without relying on the circular holes being standard circles, providing high-precision and high-reliability 3D geometric features for subsequent calculation of the relative extrinsic parameters between the LiDAR and the camera.

[0061] The above implementation method, by sequentially performing pass-through filtering, RANSAC plane fitting, coordinate system normalization, edge point clustering, and robust center extraction on the point cloud, can effectively filter out noise and interference data in real after-sales scenarios, robustly locate the calibration board plane and unify the calculation benchmark, while being compatible with non-ideal situations such as incomplete point clouds and irregular circular holes, significantly improving the stability, anti-interference, and accuracy of 3D feature extraction, and providing a solid and reliable data foundation for subsequent high-precision extrinsic parameter calculation.

[0062] Please combine further Figure 8 , Figure 8 This is a flowchart illustrating another embodiment of step S210 of this application, as shown below. Figure 8 Step S210 of this application further includes the following sub-steps:

[0063] S211a, preprocess the camera image data to obtain the image area of ​​the calibration board.

[0064] Furthermore, for the acquired camera image data, image preprocessing operations such as grayscale conversion, noise reduction, and contrast enhancement are first performed to weaken interference factors such as changes in ambient lighting, vehicle body reflection, and surrounding debris, highlighting the overall outline and structural features of the calibration board.

[0065] Furthermore, after preprocessing, the image region where the calibration board is located is located and segmented from the overall image using image segmentation and contour detection algorithms. This eliminates the influence of irrelevant areas such as the background, ground, and vehicle surroundings, providing a clean and well-defined image foundation for subsequent pixel coordinate calculations.

[0066] S212a, Two-dimensional pixel coordinates are calculated based on the image area of ​​the calibration plate and the preset size information of the calibration plate.

[0067] Furthermore, after determining the image area of ​​the calibration board, the system combines the pre-stored physical dimensions, hole spacing, geometric layout, and other preset size information of the calibration board in the system. Based on the actual projection position and scale of the calibration board in the image, the system calculates the two-dimensional pixel coordinates corresponding to the center of each hole on the calibration board, thereby achieving fast and stable extraction of image features. This method does not rely on the detection of complex coding patterns and has the characteristics of low computational load, high real-time performance, and strong applicability.

[0068] The above implementation method, by preprocessing the image and locating the calibration plate area, and then calculating the two-dimensional pixel coordinates in combination with the preset size information of the calibration plate, can effectively suppress environmental interference, simplify the image feature extraction process, reduce algorithm complexity and dependence on specific markers while ensuring extraction accuracy, and greatly improve the stability, versatility and real-time performance of image feature extraction.

[0069] S220, the relative extrinsic parameters are calculated based on the three-dimensional coordinates and the two-dimensional pixel coordinates.

[0070] Please combine further Figure 9 , Figure 9 This is a flowchart illustrating another embodiment of step S220 of this application, as shown below. Figure 9 Step S220 of this application, as shown, further includes the following sub-steps:

[0071] S221, solve for the initial extrinsic parameters based on the three-dimensional coordinates and two-dimensional pixel coordinates.

[0072] Furthermore, after obtaining the 3D coordinates of the center of the circular hole on the calibration plate and the corresponding 2D pixel coordinates in the image, the Perspective-n-Point (PnP) algorithm is used to calculate the initial pose between the sensors based on multiple sets of precisely matched 3D-2D point pairs. The PnP algorithm utilizes the known 3D spatial point coordinates and their projected pixel coordinates on the image to establish a perspective projection constraint relationship from space to image. Through numerical solution, the initial extrinsic parameters of the LiDAR relative to the camera are obtained. These initial extrinsic parameters include a rotation matrix and a translation vector, which can initially reflect the true spatial position and attitude relationship between the LiDAR and the camera. This provides stable, reliable, and convergent initial parameters for subsequent nonlinear fine optimization, avoiding divergence or getting trapped in local optima during the optimization process.

[0073] S222, relative extrinsic parameters are obtained by optimizing the initial extrinsic parameters based on the reprojection error.

[0074] Furthermore, after obtaining the initial extrinsic parameters, a loss function is constructed with reprojection error as the optimization objective. The 3D center coordinates extracted from the LiDAR point cloud are projected onto the camera image plane using the current initial extrinsic parameters to obtain the corresponding projected pixel coordinates. These projected pixel coordinates are then compared with the observed center pixel coordinates obtained from actual image detection to calculate the reprojection error. With minimizing the reprojection error as the optimization objective, a nonlinear optimization method is used to iteratively adjust the rotation matrix and translation vector in the extrinsic parameters, continuously correcting the projection deviation until the extrinsic parameters gradually converge to the global optimum. Ultimately, high-precision, high-stability relative extrinsic parameters between the LiDAR and the camera that accurately reflect the spatial relationship between the sensors are obtained.

[0075] In the above implementation, by first using the PnP algorithm to solve the initial extrinsic parameters and then performing nonlinear fine optimization based on the reprojection error, a two-stage solution method can be used to quickly obtain stable and reliable initial pose values. Furthermore, the accuracy of the extrinsic parameters can be further improved through iterative optimization, enabling the post-sales calibration results to reach the accuracy level of the calibration interval. At the same time, the robustness, convergence, and reliability of the solution process are guaranteed.

[0076] S300 updates the relative external parameters to the vehicle's coordinate system to complete the calibration.

[0077] Understandably, after the domain controller completes the calculation and optimization of the relative extrinsic parameters between the LiDAR and the camera, the final high-precision extrinsic parameters are written into the vehicle's perception fusion system or the domain controller's storage area, replacing the vehicle's original old calibration parameters. Based on the updated extrinsic parameters, the vehicle coordinate system re-establishes an accurate spatial mapping relationship between the LiDAR's 3D perception data and the camera's 2D image data. This enables intelligent driving-related functions such as target detection, trajectory prediction, and obstacle fusion to operate based on precise sensor pose relationships, ensuring the accuracy and reliability of the perception system's output.

[0078] Furthermore, after the extrinsic parameters are updated, the VDS vehicle diagnostic system provides real-time feedback to the operator on the calibration completion status, indicating that the entire after-sales calibration process for the LiDAR and camera is complete. In this embodiment, by automatically updating and distributing calibration parameters, the sensor extrinsic parameters are made effective quickly, ensuring that the vehicle perception system rapidly returns to a high-precision working state after after-sales maintenance or sensor adjustments, thereby improving the safety and reliability of the intelligent driving system.

[0079] In the above embodiments, synchronous LiDAR point cloud and camera image data are collected in response to calibration trigger commands. Relative extrinsic parameters are calculated based on the two types of data and updated to the vehicle coordinate system, realizing rapid and efficient joint calibration of LiDAR and camera. The process is simple and universal, requiring no complex calibration equipment or professional site, adapting to after-sales scenario needs, and can quickly restore sensor pose accuracy, ensuring the reliability and safety of intelligent driving perception system. At the same time, it has strong superposition capabilities, is compatible with various calibration detail optimization schemes, and has a wide protection range.

[0080] Please combine further Figure 10 , Figure 10 This is a schematic diagram of one embodiment of the electronic device of this application, specifically:

[0081] The electronic device 300 may include components such as a processor 301 with one or more processing cores, a memory 302 with one or more storage media, a power supply 303, and an input unit 304. Those skilled in the art will understand that... Figure 10 The electronic device structure shown does not constitute a limitation on the electronic device and may include more or fewer components than shown, or combine certain components, or have different component arrangements. Wherein:

[0082] The processor 301 is the control center of the electronic device. It connects various parts of the electronic device via various interfaces and lines, and performs various functions and processes data by running or executing computer programs and / or modules stored in the memory 302, and by calling data stored in the memory 302. Optionally, the processor 301 may include one or more processing cores; optionally, the processor 301 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications, and the modem processor mainly handles wireless communication. It is understood that the modem processor may also not be integrated into the processor 301.

[0083] The memory 302 can be used to store computer programs and modules. The processor 301 executes various functional applications and vehicle control by running the computer programs and modules stored in the memory 302. The memory 302 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, computer programs required for at least one function (such as calibration methods), etc.; the data storage area may store data created based on the use of the electronic device, etc. In addition, the memory 302 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory 302 may also include memory electronics to provide the processor 301 with access to the memory 302.

[0084] The electronic device also includes a power supply 303 that supplies power to the various components. Optionally, the power supply 303 can be logically connected to the processor 301 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system. The power supply 303 may also include one or more DC or AC power supplies, recharging systems, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components.

[0085] The electronic device may also include an input unit 304, which can be used to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function control.

[0086] Although not shown, the electronic device may also include a display unit, etc., which will not be described in detail here. Specifically, in this embodiment, the processor 301 in the electronic device loads the executable files corresponding to the processes of one or more computer programs into the memory 302 according to the following instructions, and the processor 301 runs the computer programs stored in the memory 302 to realize various functions, such as:

[0087] In response to a calibration trigger command, multiple sets of calibration board data are collected, each set including LiDAR point cloud data and camera image data; the relative extrinsic parameters between the LiDAR and the camera are calculated based on the point cloud data and the camera image data; the relative extrinsic parameters are updated to the vehicle's coordinate system to complete the calibration.

[0088] Therefore, the electronic device provided in this application embodiment collects synchronous LiDAR point cloud and camera image data in response to calibration trigger commands, calculates relative extrinsic parameters based on the two types of data and updates them to the vehicle coordinate system, realizing rapid and efficient joint calibration of LiDAR and camera. The process is simple and universal, requiring no complex calibration equipment or professional site, adapting to after-sales scenario needs, and can quickly restore sensor pose accuracy, ensuring the reliability and safety of intelligent driving perception system. At the same time, it has strong superposition capabilities, is compatible with various calibration detail optimization schemes, and has a wide protection range.

[0089] For details on the specific implementation methods and corresponding beneficial effects of each of the above operations, please refer to the detailed description of the calibration method above, which will not be repeated here.

[0090] To this end, embodiments of this application provide a computer storage medium storing a computer program that can be loaded by a processor to execute the steps in any of the calibration methods provided in embodiments of this application. For example, the computer program can execute the following steps: in response to a calibration trigger command, acquiring multiple sets of calibration board data, each set of calibration board data including LiDAR point cloud data and camera image data; calculating the relative extrinsic parameters between the LiDAR and the camera based on the point cloud data and the camera image data; and updating the relative extrinsic parameters to the vehicle's coordinate system to complete the calibration.

[0091] Therefore, the storage medium provided in this application embodiment can collect synchronous LiDAR point cloud and camera image data in response to calibration trigger commands, calculate relative extrinsic parameters based on the two types of data and update them to the vehicle coordinate system, thereby realizing fast and efficient joint calibration of LiDAR and camera. The process is simple and universal, requiring no complex calibration equipment or professional site, adapting to after-sales scenario needs, and can quickly restore sensor pose accuracy, ensuring the reliability and safety of intelligent driving perception system. At the same time, it has strong superposition capabilities, is compatible with various calibration detail optimization schemes, and has a wide protection range.

[0092] For details on the specific implementation methods and corresponding beneficial effects of the above operations, please refer to the previous embodiments, which will not be repeated here.

[0093] The storage medium may include: read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0094] Since the computer program stored in the storage medium can execute the steps in any calibration method provided in the embodiments of this application, the beneficial effects that the calibration method provided in the embodiments of this application can achieve can be realized. For details, please refer to the previous embodiments, which will not be repeated here.

[0095] Please combine further Figure 11 , Figure 11 This is a structural schematic diagram of one embodiment of the vehicle described in this application, as shown below. Figure 11 This embodiment provides a vehicle 400, which is a motor vehicle equipped with intelligent driving functions, specifically including a lidar, a camera, and the aforementioned electronic equipment 300.

[0096] The lidar and camera are arranged at the front of the vehicle as environmental perception sensors to collect point cloud data and image data of the environment in front of the vehicle. The electronic device 300 is integrated into the domain controller of the vehicle 400. Its memory stores a computer program. When the processor executes the program, it can implement the lidar and camera calibration method in any of the above embodiments. After the vehicle sensor position is changed, repaired or replaced, the after-sales joint calibration of lidar and camera can be completed quickly, and the calculated relative external parameters are updated to the vehicle coordinate system to ensure the accuracy and reliability of intelligent driving functions such as perception fusion and target detection.

[0097] In the above embodiments, synchronous LiDAR point cloud and camera image data are collected in response to calibration trigger commands. Relative extrinsic parameters are calculated based on the two types of data and updated to the vehicle coordinate system, realizing rapid and efficient joint calibration of LiDAR and camera. The process is simple and universal, requiring no complex calibration equipment or professional site, adapting to after-sales scenario needs, and can quickly restore sensor pose accuracy, ensuring the reliability and safety of intelligent driving perception system. At the same time, it has strong superposition capabilities, is compatible with various calibration detail optimization schemes, and has a wide protection range.

[0098] The above provides a detailed description of a calibration method, storage medium, electronic device, and vehicle provided in the embodiments of this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only configured to help understand the method and core ideas of this application. At the same time, those skilled in the art will recognize that there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

[0099] Although embodiments of this application have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of this application, the scope of which is defined by the claims and their equivalents.

Claims

1. A calibration method, characterized in that, The calibration method includes: In response to the calibration trigger command, multiple sets of calibration board data are collected. Each set of calibration board data includes LiDAR point cloud data and camera image data. The relative extrinsic parameters between the lidar and the camera are calculated based on the point cloud data and the camera image data. The relative extrinsic parameters are updated to the vehicle's coordinate system to complete the calibration.

2. The calibration method according to claim 1, characterized in that, The process of calculating the relative extrinsic parameters between the lidar and the camera based on the point cloud data and the camera image data includes: The three-dimensional coordinates of the corresponding features of the calibration board are extracted based on the point cloud data, and the two-dimensional pixel coordinates of the corresponding features of the calibration board are extracted based on the camera image data. The relative extrinsic parameters are calculated based on the three-dimensional coordinates and the two-dimensional pixel coordinates.

3. The calibration method according to claim 2, characterized in that, The step of extracting the three-dimensional coordinates of the calibration board features based on the point cloud data includes: The point cloud data is filtered to obtain a point cloud region containing the calibration board; The point cloud data within the point cloud region is fitted to obtain a calibration plate plane, and the calibration plate plane is transformed to a preset reference plane; The three-dimensional coordinates are determined based on the point cloud data of the transformed calibration plate plane.

4. The calibration method according to claim 2, characterized in that, The step of extracting the two-dimensional pixel coordinates of the calibration board based on the camera image data includes: The camera image data is preprocessed to obtain the image region of the calibration board; The two-dimensional pixel coordinates are calculated based on the image area of ​​the calibration board and the preset size information of the calibration board.

5. The calibration method according to claim 2, characterized in that, The process of calculating the relative extrinsic parameters based on the three-dimensional coordinates and the two-dimensional pixel coordinates includes: The initial extrinsic parameters are determined based on the three-dimensional coordinates and the two-dimensional pixel coordinates. The relative extrinsic parameters are obtained by optimizing the initial extrinsic parameters based on the reprojection error.

6. The calibration method according to claim 1, characterized in that, The multiple sets of calibration board data are obtained by placing the same calibration board at multiple preset positions within the common field of view of the lidar and the camera.

7. The calibration method according to claim 1, characterized in that, The calibration plate includes visual markers for positioning and multiple hollow circular holes. The calibration plate is arranged perpendicular to the ground and rotated at a preset angle relative to the vehicle.

8. A computer storage medium, characterized in that, When the computer program is executed by the processor, it implements the calibration method according to any one of claims 1-7.

9. An electronic device, characterized in that, It includes a processor and a memory, the memory storing a computer program, and the processor executing the computer program to implement the calibration method according to any one of claims 1-7.

10. A vehicle, characterized in that, The vehicle includes a lidar, a camera, and the electronic device as described in claim 9.

Images