Calibration system, calibration method and electronic equipment

Abstract

The invention relates to the technical field of equipment calibration, in particular to a calibration system, a calibration method and electronic equipment. The calibration system comprises: a mobile base, which comprises at least one mobile track, and the mobile track comprises an initial position; the carrier is configured to move along the moving track, the carrier is provided with a fixing device, and the fixing device is configured to detachably install to-be-calibrated equipment; the calibration part is fixedly arranged at a preset distance away from the initial position; the calibration piece comprises at least two types of marks, and each type of mark is used for calibrating one type of sensor; under the condition that the to-be-calibrated equipment is calibrated, the calibration piece is located in the detection range of the to-be-calibrated equipment, and the detection direction of the to-be-calibrated equipment faces the calibration piece; the carrier is configured to drive the to-be-calibrated device to move along the moving track, the to-be-calibrated device is configured to acquire detection information of the calibration piece corresponding to different positions on the moving track, and the positions of the to-be-calibrated devices at different positions relative to the calibration piece are different. Therefore, the problems of low efficiency and poor precision caused by manual intervention can be reduced, and the problem of errors can be reduced.

Description

Technical Field

[0001] This application relates to the field of equipment calibration technology, specifically to a calibration system, calibration method, and electronic equipment. Background Technology

[0002] With the rapid development of humanoid robot technology, in order to better achieve autonomous navigation and intelligent obstacle avoidance, it is usually necessary to integrate multiple types of sensors on the robot, such as LiDAR, cameras, and inertial measurement units (IMUs). Each sensor outputs data (such as images, point clouds, etc.) based on its own independent coordinate system. In order to achieve high-precision data fusion, these sensors must be calibrated efficiently and accurately to obtain high-quality raw data and pose transformation relationships between different sensor coordinate systems, so as to unify all sensor data into a single coordinate system for processing.

[0003] In related technologies, manual calibration is performed separately for the camera's intrinsic parameters, the radar and camera's extrinsic parameters, and the IMU and radar's extrinsic parameters. For example, by moving a calibration plate used for the camera's intrinsic parameters while holding it, the camera's intrinsic parameters are calibrated using images captured by the camera. Then, the radar and camera's extrinsic parameters, and the IMU and radar's extrinsic parameters are calibrated separately.

[0004] Therefore, the calibration process for cameras, radars, and IMUs currently relies mainly on manual operation, which is inefficient and can lead to inconsistent calibration accuracy due to manual intervention. Furthermore, different calibration processes require different calibration boards, and changing calibration boards is complex and prone to errors. Summary of the Invention

[0005] To address the aforementioned issues, embodiments of this application provide a calibration system, calibration method, and electronic device that reduce the inefficiency and poor accuracy caused by manual intervention, and also reduce the likelihood of errors.

[0006] To achieve the above objectives, in a first aspect, embodiments of this application provide a calibration system, comprising: a movable base including at least one movable track, the movable track including an initial position; a carrier configured to move along the movable track, the carrier having a fixing device configured to detachably mount a device to be calibrated; a calibration element fixedly disposed at a preset distance from the initial position; the calibration element including at least two types of markings, each type of marking being used for calibrating a type of sensor; when calibrating the device to be calibrated, the calibration element is located within the detection range of the device to be calibrated, and the detection direction of the device to be calibrated is towards the calibration element; the carrier is configured to drive the device to be calibrated to move along the movable track, the device to be calibrated is configured to acquire detection information of the calibration element corresponding to different positions on the movable track, the positions of the device to be calibrated relative to the calibration element are different at different positions.

[0007] The calibration system provided in this application embodiment can achieve joint calibration of multiple sensors in a device to be calibrated. By simultaneously collecting the detection information required for multi-sensor calibration during the movement of the device to be calibrated relative to the calibration component, multiple sensors can be calibrated collaboratively after a single acquisition, avoiding errors introduced by multiple calibrations. Furthermore, it can reduce errors caused by manual intervention in the calibration process, improving the automation level and repeatability accuracy of the calibration process.

[0008] In one possible implementation, the mobile base includes: a platform with its top surface at a first height, the platform including an initial position and an exit; a ramp including a first end and a second end, the first end being connected to the exit; the height of the first end gradually decreasing towards the second end; a mobile track including the initial position and a path on the ramp from the first end to the second end; and a vehicle configured to start from the platform, enter the ramp through the first end, and move along the ramp to the second end. Thus, during the movement of the vehicle along the ramp from the first end to the second end, the device to be calibrated undergoes multi-axis motion relative to the calibration component, causing the relative position between the device to be calibrated and the calibration component to change with multiple degrees of freedom, thereby obtaining richer detection information for calibration.

[0009] In one possible implementation, there is at least one ramp; the platform's exit corresponds one-to-one with the ramp; if there are multiple ramps, the ramps extend in different directions, and the first end of each ramp connects to a different exit of the platform. In this way, different ramps correspond to different extension directions, allowing the device to be calibrated to move relative to the calibration component in more directions during its movement, thereby obtaining multi-angle detection information of the calibration component and improving the comprehensiveness and accuracy of the calibration.

[0010] In one possible implementation, the carrier further includes a mobile chassis, a fixed device mounted on the mobile chassis, and the equipment to be calibrated detachably mounted on the fixed device; the mobile chassis is configured to drive the fixed device to move along a mobile track. In this way, the mobile chassis can flexibly drive the equipment to be calibrated along the mobile track, providing power for the automated movement of the equipment and facilitating the automation of the calibration process.

[0011] In one possible implementation, the device to be calibrated is applied to the machine body; the fixing device includes: a frame, mounted on a mobile chassis; at least one fixing part, mounted on the frame; when calibrating the device, the machine body is fixedly mounted on the fixing part. In this way, the machine body is securely mounted on the frame via the fixing part, stabilizing the posture of the device to be calibrated during the calibration process, reducing detection errors caused by equipment shaking, and facilitating quick assembly and disassembly, thus improving calibration efficiency.

[0012] In one possible implementation, the device to be calibrated includes at least two of a first sensor, a second sensor, and a third sensor. The first sensor is configured to acquire an image of the calibrated part; the second sensor is configured to acquire a point cloud of the image of the calibrated part; and the third sensor is configured to acquire its own motion data. In this way, multiple sensors work collaboratively to simultaneously acquire the visual texture, 3D point cloud, and pose data of the calibrated part, achieving multimodal data fusion during calibration and improving calibration accuracy and environmental adaptability.

[0013] In one possible implementation, the calibration component includes: a first type of marker configured for intrinsic parameter calibration of a first sensor to be calibrated; a second type of marker configured for extrinsic parameter calibration of both the first and second sensors to be calibrated; and a third type of marker configured for extrinsic parameter calibration of both sensors. This coordinated arrangement of the three types of markers provides dedicated feature information for both intrinsic and extrinsic parameter calibration, enabling joint calibration of multiple sensors to be completed with a single data acquisition from the same calibration component. This avoids errors caused by changing calibration components and improves calibration efficiency and consistency.

[0014] In one possible implementation, the calibration system further includes: electronic equipment configured to control the movement of the carrier; the electronic equipment is also configured to calibrate the device to be calibrated based on the detection information. In this way, the electronic equipment can achieve centralized scheduling and automated control of the calibration process, thereby automating the calibration process and reducing errors caused by manual intervention.

[0015] Secondly, embodiments of this application provide a calibration method, comprising: controlling a carrier to move along a moving track of a movable base; the movable base includes at least one moving track, the moving track including an initial position; the carrier includes a fixing device, and the device to be calibrated is detachably mounted on the fixing device; during the movement of the carrier along the moving track, acquiring at least two sets of detection information; wherein each set of detection information includes at least two sensing information, each sensing information being acquired by a sensor to be calibrated included in the device to be calibrated; the positions on the moving track corresponding to each set of detection information are different; and calibrating each sensor to be calibrated in the device to be calibrated based on the detection information.

[0016] The calibration method provided in this application allows for the movement of a carrier along a track, and the device to be calibrated can be moved by controlling the carrier's movement. Multiple sets of detection information from various angles of the calibration components are collected from the device to be calibrated, enabling joint calibration of each sensor within the device. This achieves automated calibration of the device, reducing errors caused by manual intervention and improving both calibration efficiency and accuracy.

[0017] In one possible implementation, the calibration element is fixed at a preset distance from the initial position, located within the detection range of the device to be calibrated, with the detection direction of the device facing the calibration element. The relative positions of the device to be calibrated and the calibration element differ for each set of detection information. Each set of detection information includes at least two of a first sensing information, a second sensing information, and a third sensing information. The first sensing information is an image of the calibration element acquired by the first sensor to be calibrated in the device; the second sensing information is a point cloud of the calibration element acquired by the second sensor to be calibrated in the device; and the third sensing information is the motion data of the third sensor to be calibrated in the device. In this way, by acquiring the sensing data of each sensor to be calibrated simultaneously, spatiotemporal alignment of multimodal data can be achieved, improving the accuracy of joint calibration.

[0018] In one possible implementation, the calibration component includes at least two types of markers, each type used for calibrating a specific type of sensor. The calibration component includes a first type of marker, a second type of marker, and a third type of marker. Based on detection information, each sensor to be calibrated in the calibration device is calibrated, including: calibrating the intrinsic parameters of the first sensor to be calibrated based on relevant information from the first type of marker in the first sensing information; calibrating the extrinsic parameters of the second sensing information and the first sensor to be calibrated based on relevant information from the second and third types of markers in the first sensing information, and relevant information from the second type of marker in the second sensing information; and calibrating the extrinsic parameters of the second sensing information and the third sensor to be calibrated based on the second and third sensing information. Thus, based on the first, second, and third sensing information, the intrinsic and extrinsic parameters of the first, second, and third sensors to be calibrated are jointly calibrated, respectively, achieving joint calibration of the calibration device.

[0019] In one possible implementation, the mobile base includes a platform and a ramp; the platform includes an initial position and an exit, and the height of the top surface of the platform is a first height; the ramp includes a first end and a second end, the first end being connected to the exit; the height of the first end gradually decreases from the second end; controlling the vehicle to move along the mobile track of the mobile base includes: controlling the vehicle to stay at the initial position for a preset time; controlling the vehicle to move through the first end of the ramp to the second end of the ramp; and controlling the vehicle to move through the second end of the ramp back to the initial position. This allows the device to be calibrated to collect multi-angle detection information from the calibration component, facilitating calibration of the device based on this detection information.

[0020] In one possible implementation, when there are multiple ramps, the ramps extend in different directions, and the first end of each ramp connects to a different exit of the platform; after controlling the vehicle to move to the initial position via the second end of the ramp, the implementation further includes controlling the vehicle to enter other ramps from the initial position. This allows for the acquisition of more detection information from different angles on the calibration component.

[0021] Thirdly, embodiments of this application provide a chip system, including: a memory and one or more processors; the memory is coupled to the processors; wherein the memory stores computer program code, the computer program code including computer instructions, and when the computer instructions are executed by the processor, the electronic device performs the calibration method provided by the second aspect above and any possible implementation thereof.

[0022] Fourthly, embodiments of this application provide an electronic device, including: a chip system as described in the third aspect above.

[0023] Fifthly, embodiments of this application provide a computer-readable storage medium including computer instructions that, when executed on an electronic device, cause the electronic device to perform the calibration method provided in the second aspect above and any possible implementation thereof.

[0024] Sixthly, embodiments of this application provide a computer program product that, when run on a computer, causes the computer to execute the calibration method provided in the second aspect above and any possible implementation thereof.

[0025] It is worth noting that the beneficial effects that the technical solutions provided in the third to sixth aspects above can achieve can be referred to the beneficial effects that the technical solutions provided in the second aspect above and any possible implementation thereof can achieve, and this application will not repeat them here. Attached Figure Description

[0026] To more clearly illustrate the technical solution of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0027] Figure 1 This is a schematic diagram of a calibration system provided in an embodiment of this application; Figure 2A This is a schematic diagram of the first application scenario for calibrating the device to be calibrated, provided in an embodiment of this application. Figure 2B This is a schematic diagram of a second application scenario for calibrating the device to be calibrated, provided in an embodiment of this application. Figure 2C This is a schematic diagram of a third application scenario for calibrating the device to be calibrated, provided in an embodiment of this application. Figure 2D This is a schematic diagram of the fourth application scenario for calibrating the device to be calibrated, provided in the embodiments of this application; Figure 2E This is a schematic diagram of the fifth application scenario for calibrating the device to be calibrated, provided in the embodiments of this application.

[0028] Figure 3 This is a schematic flowchart of the calibration method provided in the embodiments of this application; Figure 4 This is a simplified structural diagram of the mobile base provided in the embodiments of this application; Figure 5 This is a schematic diagram of the structure of the vehicle provided in the embodiments of this application; Figure 6 This is a simplified structural diagram of the mobile chassis provided in the embodiments of this application; Figure 7 This is a schematic diagram of the fixing device provided in the embodiment of this application; Figure 8 This is a schematic diagram of the structure of the calibration component provided in the embodiments of this application; Figure 9 This is a schematic diagram of the chip system provided in an embodiment of this application.

[0029] Figure label: 100. Calibration system; 110. Moving base; 111. Moving track; 112. Initial position; 113. Platform; 114. Ramp; 114a. First end; 114b. Second end; 120. Vehicle; 121. Fixing device; 121a. Frame; 121b. Head fixing part; 121c. Hip fixing part; 121d. Leg fixing part; 122. Mobile chassis; 122a. Shell; 122b. Movable wheel set; 130. Calibration parts; 131. Class I markings; 132. Class II markings; 133. Class III markings; 200. Equipment to be calibrated; 900. Chip system; 901. Processor; 902. Interface circuit. Detailed Implementation

[0030] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. Other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are all within the protection scope of this application.

[0031] In the following description, the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined with "first," "second," etc., may explicitly or implicitly include one or more of that feature. In the description of this application, unless otherwise stated, "a plurality of" means two or more.

[0032] Furthermore, in this application, directional terms such as "upper," "lower," "inner," and "outer" are defined relative to the indicated placement of the components in the accompanying drawings. It should be understood that these directional terms are relative concepts, used for relative description and clarification, and can change accordingly depending on the placement of the components in the accompanying drawings.

[0033] Currently, high-precision motion control systems for robots, drones, and other similar devices typically include sensor arrays. These sensor arrays acquire real-time images, point clouds, and motion state information from the external environment, providing the system with fundamental data for environmental perception and pose estimation. These sensors usually include cameras, LiDAR, and IMUs, and the data collected by each sensor must be precisely calibrated before effective fusion can be achieved.

[0034] Traditional calibration methods typically involve pairwise calibration of sensors, that is, determining the external parameter relationships between the camera and the lidar, the lidar and the IMU, and the camera and the IMU one by one. This requires manual replacement of calibration objects, which is cumbersome and prone to errors.

[0035] For example, in the process of calibrating the camera's intrinsic parameters, the main method is to hold the calibration board and change its position and orientation relative to the camera, and collect multiple sets of images for calibration. This process is cumbersome and prone to human error.

[0036] In the process of calibrating the extrinsic parameters of the camera and LiDAR, the calibration board is first replaced with the calibration object required for the camera and LiDAR. Then, the position and orientation of the calibration object are manually adjusted, and camera images and LiDAR point cloud data are acquired respectively. Finally, the extrinsic parameter calibration is completed based on feature matching. This process is not only time-consuming, but also requires repeated adjustments to the orientation of the calibration object to obtain sufficient effective data, further increasing the operational complexity.

[0037] When jointly calibrating a camera, lidar, and IMU, traditional methods require sequential pairwise calibration of each sensor, followed by matrix transfer to calculate initial extrinsic parameters. This process is lengthy and prone to significant error accumulation. Furthermore, the need to change the calibration object during each calibration process further reduces calibration efficiency, and additional errors can easily be introduced between different calibration stages due to environmental changes or minor equipment movements.

[0038] In summary, traditional calibration methods suffer from drawbacks such as cumbersome operation, low efficiency, and significant error accumulation, making it difficult to meet the requirements of high-precision motion control systems for rapid and accurate calibration. Furthermore, they cannot automate calibration processes, limiting their practical application in dynamic scenarios and large-scale deployments.

[0039] To address the aforementioned technical problems, embodiments of this application provide a calibration system, calibration method, and electronic device. By employing an integrated calibration device and an automated control module, the integrated operation of multi-sensor joint calibration is achieved, reducing the problems of low efficiency and poor accuracy caused by manual intervention, and also reducing the problem of errors.

[0040] Figure 1 This is a schematic diagram of a calibration system provided in an embodiment of this application. Figure 2AThis is a schematic diagram of the first application scenario for calibrating the device to be calibrated, provided in an embodiment of this application. Figure 2B This is a schematic diagram illustrating a second application scenario for calibrating the device to be calibrated, as provided in an embodiment of this application. Figure 2C This is a schematic diagram illustrating a third application scenario for calibrating the device to be calibrated, as provided in this application embodiment. Figure 2D This is a schematic diagram of the fourth application scenario for calibrating the device to be calibrated, provided in an embodiment of this application. Figure 2E This is a schematic diagram of the fifth application scenario for calibrating the device to be calibrated, provided in the embodiments of this application.

[0041] In some embodiments, such as Figure 1 , Figures 2A to 2E As shown, the calibration system 100 includes a mobile base 110, a carrier 120, and a calibration component 130.

[0042] For ease of explanation, a three-dimensional Cartesian coordinate system (xyz) is constructed in the calibration system 100. The x-axis and y-axis define the direction of movement in the horizontal plane, and the z-axis extends vertically. The origin of the three-dimensional coordinate system is the center point of the initial position 112 of the moving base 110.

[0043] In some examples, a moving track 111 is provided on the moving base 110, and the calibration component 130 is fixedly disposed relative to the moving base 110. The moving track 111 is used to allow the device to be calibrated 200 to move along a preset path to change the relative position between the device to be calibrated 200 and the calibration component 130, thereby facilitating the device to be calibrated 200 to collect multi-angle detection information from the calibration component 130.

[0044] For example, the number of moving tracks 111 is at least one.

[0045] For example, the number of moving tracks 111 can be set as needed. This increases the range of movement of the device to be calibrated 200 relative to the calibration component 130 by setting more moving tracks 111, thereby improving the coverage and density of spatial sampling points during calibration and increasing the observation diversity of the device to be calibrated 200 on the calibration component 130. Furthermore, setting multiple moving tracks 111 can reduce the risk of calibration blind spots caused by a single moving path.

[0046] For example, the moving track 111 includes an initial position 112, which serves as the starting positioning reference for the device 200 to be calibrated. This allows each calibration process to be repeated based on the initial position 112, thereby improving the consistency and repeatability of the calibration process.

[0047] For example, during each calibration process of the device to be calibrated 200, the device to be calibrated 200 starts from the initial position 112, moves along the moving track 111 to multiple preset acquisition points, automatically triggers the data acquisition command, and acquires multimodal sensing data of the calibration component 130 in different poses.

[0048] For example, during each calibration process of the device to be calibrated 200, as the device to be calibrated 200 starts from the initial position 112 and moves along the moving track 111, it continuously scans the calibration component 130 to obtain continuous detection information of the calibration component 130, such as video recording, thereby realizing complete data recording of the dynamic calibration process.

[0049] For example, during the data acquisition process, if the acquired information is continuous detection information of the calibration component 130, key frames can be extracted from the continuous data stream according to the preset acquisition time points, and then the calibration device 200 can be calibrated.

[0050] It is worth noting that the specific type of the device 200 to be calibrated in this application embodiment is not limited, and the device 200 to be calibrated includes various sensors such as lidar, camera, and IMU.

[0051] In some examples, the carrier 120 is configured to move along a moving track 111. The carrier 120 has a fixing device 121 configured to detachably mount the device to be calibrated 200. Thus, the device to be calibrated 200 can be mounted on the carrier 120 via the fixing device 121, allowing the device to move along the moving track 111 with the carrier 120 to different positions on the track 111, changing the position of the device to be calibrated 200 relative to the calibration component 130, thereby facilitating the acquisition of multi-angle detection information from the calibration component 130.

[0052] For example, the carrier 120 can move back and forth along the moving track 111 so that the device to be calibrated 200 can acquire multiple data from the same calibration component 130 under different back and forth paths, thereby enhancing the diversity and robustness of the detection information.

[0053] In some examples, the calibration element 130 is fixed at a preset distance from the initial position 112. In this way, the relative position between the calibration element 130 and the initial position 112 remains constant, so as to establish a stable calibration reference.

[0054] For example, each time the device to be calibrated 200 is started from the initial position 112, the relative positional relationship between the device to be calibrated 200 and the calibration component 130 can be changed, thereby obtaining detection information from different perspectives. In this way, a consistent calibration process and reference standard can be maintained each time a different device to be calibrated 200 is calibrated, improving the reliability of the calibration results.

[0055] For example, when calibrating the device 200, the calibration element 130 is located within the detection range of the device 200, and the detection direction of the device 200 is towards the calibration element 130.

[0056] For example, such as Figures 2A to 2E As shown, the detection direction of the device to be calibrated 200 is oriented towards the geometric center of the calibration component 130. In this way, by presetting the detection direction, measurement noise caused by angular deviation can be effectively reduced, and the consistency of the collected detection data can be improved.

[0057] For example, the calibration device 200 can be automatically aligned with the calibration component 130 by image recognition. That is, the calibration component 130 can be kept within the detection range of the calibration device 200 during the movement of the calibration device 200. There is no need to pre-set a fixed angle. The system can dynamically adjust the pose based on real-time image feedback to ensure that the calibration component 130 is always within the field of view of the calibration device 200.

[0058] For example, the preset distance between the calibration element 130 and the initial position 112 can be comprehensively set based on the physical size of the calibration element 130, the sensor field of view of the device to be calibrated 200, and the effective working distance of the device to be calibrated 200. In this way, when the device to be calibrated 200 moves on the moving track 111, the calibration element 130 is always within the field of view of the device to be calibrated 200, and the device to be calibrated 200 can always completely capture the effective feature area of ​​the calibration element 130, while avoiding the problems of near-field distortion or insufficient far-field resolution.

[0059] For example, the viewing angle of the device 200 to be calibrated is 60° horizontally and 45° vertically. The distance between the calibration component 130 and the initial position 112 can be set to the median range of the effective working distance of the sensor to ensure that the calibration component 130 occupies an appropriate proportion in the center of the field of view.

[0060] For example, the preset distance can also be combined with the feature size setting of the calibration component 130. At the preset distance, the calibration component 130 covers no less than 30% of the effective pixel area on the imaging plane to ensure the reliability of feature extraction from the calibration component 130.

[0061] For example, the preset distance setting can also take into account the coordinated optimization of LiDAR point cloud density and camera image resolution, avoiding sensor saturation or occlusion due to too close a distance, and also preventing data sparsity due to too far a distance.

[0062] For example, the calibration component 130 includes at least two types of markers, each type used for calibrating a specific type of sensor. This allows for the adaptation of different sensor sensing characteristics to the corresponding marker types, enabling joint calibration of multiple sensors. Furthermore, by setting multiple types of markers on the calibration component 130, response data from different types of sensors can be acquired simultaneously in the same calibration scenario, improving calibration efficiency and spatial consistency, and reducing errors caused by replacing the calibration component 130.

[0063] For example, the carrier 120 is configured to move the device to be calibrated 200 along the moving track 111. The device to be calibrated 200 is configured to acquire detection information of the calibration component 130 at different positions on the moving track 111. The positions of the device to be calibrated 200 relative to the calibration component 130 are different at different positions.

[0064] The calibration system 100 provided in this application embodiment can achieve joint calibration of more than 200 sensors in a device to be calibrated. By synchronously collecting the detection information required for multi-sensor calibration during the movement of the device to be calibrated 200 relative to the calibration component 130, multiple sensors can be calibrated collaboratively after a single acquisition, avoiding errors introduced by multiple calibrations. Furthermore, it can reduce errors caused by manual intervention in the calibration process and improve the automation level and repeatability accuracy of the calibration process.

[0065] For example, when calibration of the device 200 is required, the device 200 is mounted on the fixing device 121 of the carrier 120, and the carrier 120 is adjusted so that the device 200 is in the initial position 112. The carrier 120 is controlled to move along the moving track 111 to move the device 200 along the moving track 111. When the device 200 is in the initial position 112, the sensors of the device 200 are activated to start collecting the detection information of the calibration component 130 and its own motion parameters, and continuously and synchronously recording the data stream and corresponding spatial position of each sensor during the movement until the entire calibration path is completed.

[0066] In this way, multimodal data of the calibration component 130 under different angles and distances can be fully acquired throughout the entire movement process, ensuring coverage of the effective working range of the sensor. This facilitates the joint calibration of more than 200 sensors of the calibration device in subsequent processes, realizing an automated calibration process.

[0067] In some embodiments, the calibration system 100 further includes an electronic device. The electronic device is communicatively connected to the carrier 120 and the device 200 to be calibrated.

[0068] For example, the electronic device is configured to control the movement of the carrier 120, and the electronic device is also configured to calibrate the device 200 to be calibrated based on the detection information.

[0069] In this way, the movement of the carrier 120 can be controlled by electronic devices to drive the device to be calibrated 200 to run automatically along the moving track 111, and to acquire the detection information of the device to be calibrated 200 for calibration, so as to realize the automated calibration process of the device to be calibrated 200 and reduce the error caused by manual intervention.

[0070] It is worth noting that the specific implementation of the electronic device in this application is not limited. It can be a mobile phone, computer, industrial control computer, server or embedded system, as long as it has data processing and communication capabilities.

[0071] It is understood that the embodiments of this application do not limit the communication connection method between the electronic device and the carrier 120 and the device to be calibrated 200, and can adopt wired or wireless communication methods. Among them, wireless communication methods include Wi-Fi, Bluetooth or 5G network, etc., and wired communication methods include Ethernet, USB or RS485, etc.

[0072] Figure 3 This is a flowchart illustrating the calibration method provided in the embodiments of this application.

[0073] In some embodiments, during the calibration of the device 200 to be calibrated, the calibration method provided in this application embodiment is executed to automate the calibration of the device 200. The executing entity of this calibration method can be an electronic device, such as... Figure 3 As shown, the calibration method includes the following steps S1-S3.

[0074] S1 controls the vehicle to move along the moving track of the mobile base.

[0075] In this step, the carrier 120 is controlled to move along the preset moving track 111 to drive the device to be calibrated 200 to move, so that the device to be calibrated 200 can collect the detection information of the calibration component 130 corresponding to multiple positions on the moving track 111.

[0076] Furthermore, as the device to be calibrated 200 moves along the moving track 111, the spatial coordinates of different positions on the moving track 111 are different. In other words, the spatial position of the device to be calibrated 200 changes continuously as the position changes during the movement, thereby obtaining multi-dimensional data of the calibration component 130 from different spatial perspectives.

[0077] This allows the device to be calibrated 200 to perform multi-axis motion relative to the calibration component 130, enabling all-round observation in the pitch, yaw, and roll directions.

[0078] S2, during the movement of the vehicle along the moving track, acquire at least two sets of detection information.

[0079] Each set of detection information includes at least two sensor data points, each acquired by a sensor to be calibrated within the device to be calibrated 200. The positions on the moving track 111 corresponding to each set of detection information are different.

[0080] In this step, the device to be calibrated 200 can be controlled to continuously collect detection information while the vehicle 120 moves along the moving track 111, or the collection can be triggered when the device to be calibrated 200 moves to a preset calibration point to obtain detection data at a specific location.

[0081] S3, based on the detection information, calibrates each sensor in the calibration equipment.

[0082] In this embodiment, the calibration method provided in this application allows the carrier 120 to move the device to be calibrated 200 along the moving track 111. The device to be calibrated 200 collects detection information from multiple angles of the calibration component 130, obtaining multiple sets of detection information for each sensor in the device to be calibrated 200. This achieves automated calibration of the device to be calibrated 200, reducing errors caused by manual intervention and improving calibration efficiency and accuracy.

[0083] Figure 4 This is a simplified structural diagram of the mobile base provided in an embodiment of this application. Figure 4 (a) is a schematic diagram of the movable base 110 including a ramp 114. Figure 4 (b) is a schematic diagram of the movable base 110 including four ramps 114.

[0084] In some embodiments, such as Figure 4 As shown, the mobile base 110 includes a platform 113 and a ramp 114.

[0085] In some examples, platform 113 includes an initial position 112. In this way, by setting platform 113, the vehicle 120 can be stably stationed at the initial position 112, so that the device 200 to be calibrated has a uniform reference attitude at the beginning of the calibration phase.

[0086] For example, ramp 114 includes a first end 114a and a second end 114b, with the first end 114a connected to the exit of platform 113. Thus, platform 113 and ramp 114 together form a moving track 111, that is, the moving track 111 includes an initial position 112 and a path on ramp 114 from the first end 114a to the second end 114b.

[0087] For example, the second end 114b of the ramp 114 is connected to the ground or another platform 113 to form a continuous movement path.

[0088] For example, the slope of ramp 114 can be set according to the power of vehicle 120 to ensure that vehicle 120 has sufficient power to move along the second end 114b of ramp 114 to the first end 114a of ramp 114, reducing movement obstruction caused by insufficient power. In this way, the tilt angle of ramp 114 is optimized to ensure that vehicle 120 remains stable during movement, avoiding the impact of bumps on the accuracy of sensor data acquisition.

[0089] For example, the surface of ramp 114 has an anti-slip texture, which improves the safety of vehicle 120 during driving.

[0090] For example, the height of the top surface of platform 113 is a first height. The height of the first end 114a of ramp 114 is the first height, and the height of the second end 114b is the second height, with the first height being greater than the second height, to form an inclined channel for the vehicle 120 to move up and down. In this way, ramp 114 is connected to platform 113, allowing the vehicle 120 to drive out of or into platform 113 along ramp 114, thereby realizing multi-angle calibration of the device 200 at different height positions.

[0091] For example, during the process of the device to be calibrated 200 moving from the first end 114a to the second end 114b of the ramp 114, the height and distance of the device to be calibrated 200 relative to the calibration component 130 change. That is, the device to be calibrated 200 performs multi-axis motion, and the Euler angles of the device to be calibrated 200 need to be adjusted, such as at least one of the pitch angle, yaw angle and roll angle, so that the detection direction of the device to be calibrated 200 is oriented toward the calibration component 130, so that the device to be calibrated 200 can detect the calibration component 130.

[0092] In some examples, the extension direction of ramp 114 can be set along a direction that is only close to or far from calibrator 130, or a lateral offset can be superimposed on the direction that is close to or far from calibrator 130 to cover more spatial calibration points. In this way, by adjusting the extension path of ramp 114, the device to be calibrated 200 can pass through more calibration positions during movement, thereby collecting detection data at different distances and angles, improving the comprehensiveness and accuracy of calibration.

[0093] For example, taking the calibration component 130 with its center set on the x-axis, the extension direction of the ramp 114 on the xoy plane can be at least one of the positive x-axis direction, the negative x-axis direction, the positive y-axis direction, and the negative y-axis direction. In this way, the positive and negative x-axis directions provide the vehicle 120 with a movement track 111 that only moves closer to or away from the calibration component 130, while the positive and negative y-axis directions introduce lateral offset, causing the vehicle 120 to undergo lateral displacement while moving away from or closer to the calibration component 130, thereby covering calibration requirements in more spatial dimensions.

[0094] In other words, as the calibration device 200 moves along the ramp 114 set in the positive or negative x-axis direction on the xoy plane, the coordinates of the calibration device 200 on the x-axis and z-axis change, causing the calibration device 200 to move on four axes. Furthermore, the pitch angle of the calibration device 200 also needs to be adjusted as it moves to ensure that its detection direction is always aligned with the calibration component 130.

[0095] During the movement of the calibration device 200 along the ramp 114 set in the positive or negative y-axis direction of the extension direction on the xoy plane, the coordinates of the calibration device 200 on the y-axis and z-axis change, causing the calibration device 200 to move on four axes. Furthermore, the yaw angle of the calibration device 200 needs to be adjusted accordingly to keep the detection direction always pointing towards the calibration component 130.

[0096] During the movement of the device to be calibrated 200 along the ramp 114 set in the direction of the extension direction on the xoy plane with an angle of 45° between the x-axis and the y-axis, the device to be calibrated 200 undergoes positional changes on the three coordinate axes of x, y, and z, realizing multi-axis coordinated motion in the six degrees of freedom.

[0097] It is worth noting that the extension direction of ramp 114 on the xoy plane can be flexibly configured according to actual calibration requirements. It can be set along a single coordinate axis or a diagonal path can be used to achieve multi-axis coupled motion.

[0098] In some embodiments, step S1 in the calibration method includes the following steps S11-S13.

[0099] S11, control the vehicle 120 to stay at the initial position 112 for a preset time.

[0100] For example, the preset time ranges from 5 seconds to 15 seconds, so as to complete the initial detection of the calibration device 200 on the calibration component 130 in this step and obtain the benchmark calibration data.

[0101] S12, control the vehicle 120 to move from the first end 114a of the ramp 114 to the second end 114b of the ramp 114.

[0102] S13, control vehicle 120 to move to initial position 112 via the second end 114b of ramp 114.

[0103] In this embodiment, by controlling the carrier 120 to move back and forth along the ramp 114, the device to be calibrated 200 mounted on the carrier 120 fixing device 121 is moved relative to the calibration component 130, changing the relative position between the device to be calibrated 200 and the calibration component 130. This allows the device to be calibrated 200 to collect multi-angle detection information from the calibration component 130, so that the device to be calibrated 200 can be calibrated based on the detection information.

[0104] For example, during the movement of the vehicle 120 along the ramp 114, at least four-axis motion is performed, including translational motion along the x-axis, y-axis, and z-axis. The number of axes of the multi-axis motion of the vehicle 120 is related to the extension direction of the ramp 114 in the xoy plane.

[0105] It is worth noting that the at least two sets of detection information can correspond to either the initial position 112 or any position on the ramp 114, or they can correspond to different positions on the ramp 114. As long as the detection information corresponds to different positions on the moving track 111, the detection information of the calibration component 130 at different angles can be obtained.

[0106] For example, the moving speed of the vehicle 120 can be dynamically adjusted according to the calibration accuracy requirements. The stability of data acquisition is ensured by moving the vehicle 120 at low speed, and the calibration efficiency is improved by moving the vehicle 120 at high speed, thereby optimizing the calibration accuracy and efficiency.

[0107] In some embodiments, the calibration method further includes step S14 after step S13.

[0108] S14, control vehicle 120 to move from initial position 112 to other ramps 114.

[0109] In this way, the control vehicle 120 moves from the initial position 112 to the next moving track 111, repeating the above calibration process to obtain more detection information of the calibration component 130 at different angles.

[0110] For example, the next moving track 111 may be a circular, straight, or intersecting ramp 114 path to cover more spatial attitude combinations.

[0111] It is worth noting that the other ramps 114 can be set up in the same or different ways as the previous ramp 114, and this application does not impose any restrictions on this.

[0112] Furthermore, the movement mode of the vehicle 120 in other slopes 114 can be the same as or different from the movement mode in the previous slope 114. The motion parameters, such as movement speed, dwell time and movement trajectory, can be adjusted according to calibration requirements.

[0113] The following example uses a movable base 110 comprising four ramps 114, each ramp 114 extending in the xoy plane in the positive x-axis direction, negative x-axis direction, positive y-axis direction, and negative y-axis direction, with the calibration element 130 positioned in the positive x-axis direction. Continuing with the example... Figures 2A to 2E The workflow of the calibration system 100 is described as shown.

[0114] After installing the device 200 to be calibrated onto the fixing device 121 of the carrier 120, and ensuring a secure connection between the device 200 and the carrier 120, as follows... Figure 2A As shown, the vehicle 120 is controlled to stop at the initial position 112 for a first preset time to collect detection data for calibration in the initial state.

[0115] like Figure 2B As shown, the control vehicle 120 moves through the exit of platform 113 to a ramp 114 extending in the positive x-axis direction on the xoy plane, and slowly moves down ramp 114 to gradually approach the calibration component 130. During this movement, real-time detection data of the device to be calibrated 200 at different pitch states is collected, ensuring that its detection direction is always aligned with the calibration component 130 located in the positive x-axis direction. When the vehicle 120 reaches the lowest point along the current ramp 114, it returns to the initial position 112 along the ramp 114, completing one cycle of data acquisition.

[0116] like Figure 2C As shown, the carrier 120 is moved to a ramp 114 extending in the negative x-axis direction on the xoy plane. It then slowly moves downwards along ramp 114, gradually moving away from the calibration component 130. Simultaneously, it collects detection data from the device 200 under different pitch angles and dynamically adjusts its pitch angle to ensure the detection direction remains aligned with the calibration component 130. Once the carrier 120 reaches its lowest point, it is controlled to return to its initial position 112 along the original path, completing the reciprocating calibration process along the negative x-axis ramp 114.

[0117] like Figure 2DAs shown, the control vehicle 120 moves to a ramp 114 extending in the positive y-axis direction on the xoy plane, and slowly descends along the ramp 114. This causes the device to be calibrated 200 to shift in the y-axis direction, while its pitch and roll angles change accordingly. Real-time data is collected under different attitudes, and the yaw angle is dynamically adjusted to ensure the detection direction remains aligned with the calibration component 130 located in the positive x-axis direction. After reaching the lowest point, the control vehicle 120 returns to its initial position 112 along the original path, completing the reciprocating calibration along the ramp 114 in the positive y-axis direction.

[0118] like Figure 2E As shown, the control vehicle 120 moves to a ramp 114 extending in the negative y-axis direction on the xoy plane, and slowly descends along the ramp 114, causing the device to be calibrated 200 to experience roll and pitch angle changes during the reverse y-axis displacement. Multi-attitude detection data is continuously collected, and the yaw angle is corrected in real time to maintain accurate pointing of the calibration component 130. After the vehicle 120 descends to the lowest point, it is controlled to return to the initial position 112 along the original path, completing the reciprocating calibration of the ramp 114 in the negative y-axis direction.

[0119] After the vehicle 120 has completed its journey along all the ramps 114, all the sensors of the device 200 to be calibrated are jointly calibrated based on the acquired detection information.

[0120] For example, the calibration component 130 can also be set at any other position in the coordinate system while satisfying the preset distance from the initial position 112. The setting of the movement path can be adapted to the actual coordinates of the calibration component 130, and this application does not limit this.

[0121] In some examples, the overall shape of the moving track 111 can be straight or curved, and multiple track types can be combined to adapt to different calibration requirements.

[0122] For example, if the moving track 111 is arc-shaped, the ramp 114 can be set to an arc-shaped structure accordingly. The ramp 114 can also be set to a straight structure, but a wider ramp 114 is required to accommodate the space of the arc-shaped moving track 111.

[0123] It is worth noting that this application embodiment does not impose specific restrictions on the arrangement form, number, and shape of the moving track 111 on the moving base 110. At least one moving track 111 can be provided on the moving base 110. The track layout of the moving base 110 can be set according to the motion characteristics of the device 200 to be calibrated, so as to ensure the diversity and coverage of the collected data.

[0124] For example, the number of ramps 114 is at least one, and the number of exits of platform 113 corresponds one-to-one with the number of ramps 114.

[0125] For example, when there are multiple ramps 114, each ramp 114 extends in a different direction, and the first end 114a of each ramp 114 is connected to a different exit of the platform 113. In this way, each ramp 114 can independently form a moving track 111 with a different orientation to adapt to diverse calibration path requirements.

[0126] For example, multiple ramps 114 can extend along the x-axis, y-axis, or a combination thereof to form a multi-dimensional network of movement paths. The vehicle 120 can flexibly switch between starting from the initial platform 113 and reaching multiple spatial calibration points by selecting different exits to enter the corresponding ramps 114.

[0127] For example, the slope, length and curvature of each ramp 114 can be designed individually according to actual calibration requirements to adapt to the mobility and calibration accuracy requirements of different equipment.

[0128] Figure 5 This is a schematic diagram of the structure of the vehicle provided in the embodiments of this application.

[0129] In some embodiments, such as Figure 5 As shown, the carrier 120 includes a mobile chassis 122 and a fixing device 121. The fixing device 121 is disposed on the mobile chassis 122 and is used for detachably mounting the equipment 200 to be calibrated.

[0130] Figure 6 This is a simplified structural diagram of the mobile chassis provided in the embodiments of this application.

[0131] In some examples, such as Figure 6 As shown, the mobile chassis 122 includes a housing 122a and a set of movable wheels 122b. The set of movable wheels 122b is mounted on the bottom of the housing 122a and is used to ensure the smooth operation of the carrier 120 on the moving track 111. The fixing device 121 is disposed on the housing 122a. In this way, when the set of movable wheels 122b moves relative to the housing 122a along the moving path, the housing 122a moves along the set of movable wheels 122b, thereby driving the carrier 120 to move along the moving track 111, thereby changing the position of the device to be calibrated 200, that is, changing the relative position between the device to be calibrated 200 and the calibration component 130, so as to obtain the detection information of the calibration component 130 at different angles.

[0132] For example, the movable wheel set 122b may employ omnidirectional wheels or Mecanum wheels to support the flexible steering and precise positioning of the vehicle 120 on the track.

[0133] Figure 7 This is a schematic diagram of the fixing device provided in the embodiment of this application.

[0134] In some examples, such as Figure 7 As shown, the fixing device 121 includes a frame 121a and at least one fixing part. The fixing part is disposed on the frame 121a and is used for detachably mounting the equipment 200 to be calibrated.

[0135] In some examples, the device to be calibrated 200 is applied to the main body of the machine, that is, the device to be calibrated 200 is a sensing module on the main body of the machine.

[0136] It is worth noting that the specific type of the machine body is not limited in the embodiments of this application. The machine body is a device such as a robot, a robot dog, or a drone that is equipped with multiple sensors such as LiDAR, camera, and IMU.

[0137] For example, the shape of the frame 121a can be adapted to the shape characteristics of the device 200 to be calibrated, such as using a U-shaped frame, an L-shaped frame or a frame structure to provide stable support.

[0138] For example, such as Figure 7 As shown, when the main body of the machine is a humanoid robot, the frame 121a can be shaped like a chair to support the humanoid robot in a sitting position, so that it can maintain a stable posture during the calibration process.

[0139] Furthermore, the fixing part can be set at multiple positions on the frame 121a to clamp or adsorb the device 200 to be calibrated from different directions, ensuring that it does not shift or vibrate during movement.

[0140] For example, such as Figure 7 As shown, the fixing part may include a head fixing part 121b, a hip fixing part 121c and a leg fixing part 121d, which correspond to the installation positions of the head, hip and leg of the humanoid robot, respectively, to achieve multi-point coordinated fixing.

[0141] In this way, by constraining multiple parts of the machine body, the attitude deviation of the device to be calibrated 200 caused by the movement of the carrier 120 during the calibration process is effectively suppressed, thereby improving the stability of the calibration data.

[0142] For example, the fixing part is provided with an adjustable clip structure for detachable connection with the device 200 to be calibrated.

[0143] For example, clamping structures such as telescopic clamping arms or elastic buckles. This allows for quick clamping and positioning of equipment of different sizes.

[0144] For example, the fixing part is provided with a non-adjustable locking structure, which is used for detachable connection with the device 200 to be calibrated. In this way, the locking structure can ensure that the device 200 to be calibrated remains stably connected during movement, avoiding the impact of vibration or inertial displacement on the calibration accuracy.

[0145] For example, the locking structure can be tightened by thread, electromagnetic adsorption or pneumatic pressing to adapt to the installation requirements of different equipment 200 to be calibrated.

[0146] For example, a shock-absorbing structure can be provided between the frame 121a and the mobile chassis 122. In this way, the swaying of the frame 121a relative to the mobile chassis 122 can be reduced by the shock-absorbing structure, and the vibration generated when the mobile chassis 122 moves can be reduced and transmitted to the device to be calibrated 200, thereby further improving the stability and accuracy of the calibration process.

[0147] For example, the shock absorption structure may be an elastic rubber pad, a spring damping assembly or a hydraulic buffer device, and an appropriate solution may be selected according to the actual working conditions. This application does not limit this.

[0148] In some embodiments, the device to be calibrated 200 includes at least two of a first sensor to be calibrated, a second sensor to be calibrated, and a third sensor to be calibrated. Thus, by combining multi-dimensional sensors, the device to be calibrated 200 can perform multi-dimensional perception and data fusion of the environment, thereby improving the accuracy of spatial position, motion state, and environmental characteristics analysis during the calibration process.

[0149] In some examples, the first sensor to be calibrated is configured to acquire an image of the calibrator 130.

[0150] For example, the first sensor to be calibrated is a camera. In this way, environmental images can be captured by the camera.

[0151] In some examples, the second sensor to be calibrated is configured to acquire point clouds of images from the calibrator 130.

[0152] For example, the second sensor to be calibrated is radar. This allows for the detection of environmental point cloud data using radar, facilitating the analysis of the distance and location information of surrounding obstacles.

[0153] In some examples, the third sensor to be calibrated is configured to acquire motion data of the third sensor itself.

[0154] For example, the third sensor to be calibrated is an inertial measurement unit (IMU). In this way, the motion data of the third sensor to be calibrated, such as acceleration and angular velocity information, can be collected in real time through the IMU.

[0155] For example, the key to calibration between cameras, radar, and inertial measurement units lies in establishing a unified benchmark for multiple sensors in both spatial and temporal dimensions. This can be achieved by synchronously acquiring data streams from each sensor and using methods such as feature point matching or motion trajectory alignment to jointly optimize external parameters and time offsets.

[0156] For example, when calibrating the intrinsic parameters of a camera, the image of the corresponding mark on the calibration component 130 can be referenced. By analyzing the projection deformation of the preset pattern on the calibration component 130 in the image, the focal length, principal point offset, and distortion coefficient can be calculated, thereby completing the accurate solution of the intrinsic parameter matrix.

[0157] For example, when calibrating the extrinsic parameters of cameras and radars, the images and point clouds marked on the calibration board can be used to jointly optimize the consistency between the spatial distribution of image feature points and point cloud, and solve the rotation and translation matrices between them, thus achieving spatial alignment across modal sensors. Combined with a time synchronization mechanism, time delay deviations during data acquisition are further eliminated, ensuring the accuracy and real-time performance of multi-source information fusion.

[0158] For example, when calibrating the extrinsic parameters of radar and inertial measurement units (IMUs), an optimization model can be constructed by analyzing the geometric constraints between the motion trajectory recorded by the IMU and the environmental features observed by the radar, and the spatial transformation parameters between the two can be solved. Simultaneously, a timestamp alignment algorithm is introduced to compensate for data transmission delays between sensors, ensuring calibration accuracy in dynamic scenarios.

[0159] In some embodiments, each set of detection information includes at least two of the first sensing information, the second sensing information, and the third sensing information.

[0160] The first sensing information is the image of the calibration component 130 acquired by the first sensor to be calibrated in the calibration device 200; the second sensing information is the point cloud of the calibration component 130 acquired by the second sensor to be calibrated in the calibration device 200; and the third sensing information is the motion data of the third sensor to be calibrated in the calibration device 200 itself.

[0161] This facilitates the joint calibration of the internal and external parameters of the first, second, and third sensors to be calibrated based on the spatiotemporal correspondence between the first, second, and third sensing information.

[0162] In other words, corresponding to the various sensors to be calibrated included in the calibration device 200, the calibration component 130 is provided with at least two types of markings to calibrate different types of sensors respectively.

[0163] Figure 8 This is a schematic diagram of the calibration component provided in the embodiments of this application.

[0164] In some embodiments, such as Figure 8 As shown, the calibration component 130 includes a first type of mark 131, a second type of mark 132, and a third type of mark 133.

[0165] In some examples, the first type of marker 131 is configured for intrinsic parameter calibration of the first sensor to be calibrated. That is, the first type of marker 131 is used for calibrating the camera's intrinsic parameters.

[0166] For example, the first type of mark 131 can adopt the Aprilgrid code, which consists of multiple ordered square black and white patterns, has a unique coded identifier, and can be quickly identified and accurately located by the camera.

[0167] In this way, by detecting the position and deformation of the Aprilgrid code in the image, intrinsic parameters such as the camera's focal length, principal point offset, and radial distortion can be effectively calculated, thereby improving calibration accuracy and robustness.

[0168] For example, the calibration element 130 adopts a plate-like structure, and the Aprilgrid code is set in the central area of ​​the calibration element 130 to ensure that the camera can still capture the calibration pattern completely when viewed from different angles.

[0169] In some examples, the second type of marker 132 and the third type of marker 133 are configured for extrinsic parameter calibration of the first and second sensors to be calibrated. That is, the second type of marker 132 and the third type of marker 133 are used to achieve extrinsic parameter calibration of the camera and radar.

[0170] For example, the second type of marker 132 can adopt a circular hole structure. In this way, by using the geometric features formed by the edge of the circular hole in the radar point cloud and the features in the camera image, and by aligning the three-dimensional center coordinates of the circular hole in the point cloud with the two-dimensional center coordinates of the circular hole in the image, the correspondence between the camera pixel coordinate system and the radar point cloud coordinate system can be established, thereby solving for the extrinsic parameters of both.

[0171] It is worth noting that multiple circular holes can be used to provide sufficient constraints for solving the extrinsic parameter matrix. For example... Figure 8 As shown, four circular holes are evenly distributed on the diagonal of the calibration component 130, forming a symmetrical layout, and a first-class mark 131 is provided in the central area enclosed by the four circular holes.

[0172] For example, the third type of marker 133 can use ArUco codes, such as four 6×6 dictionaries distributed at the four corners of the calibration board, with the starting position of the upper left corner of each dictionary located at (50, 100, 250, 1000) mm on the calibration board. Each of the four ArUco codes has an independent ID identifier: the ArUco code ID of the upper left corner is 0, the upper right corner is 1, the lower left corner is 2, and the lower right corner is 3. The side length of each ArUco code is 150 mm.

[0173] In this way, the pose of the calibration board in the camera coordinate system can be quickly determined using the image of the third type of marker 133.

[0174] For example, along one diagonal of the calibration plate, there are ArUco codes, a circular hole, an Aprilgrid code, another circular hole, and another ArUco code arranged in sequence.

[0175] In some embodiments, step S3 includes the following steps S31 to S33.

[0176] S31, based on the relevant information of the first type of marker 131 in the first sensing information, calibrate the intrinsic parameters of the first sensor to be calibrated.

[0177] In this step, the relevant information of the first type of marker 131 in the first sensing information is the image of the first type of marker 131. By detecting information such as the corner position of the Aprilgrid code in the image, the intrinsic parameters of the camera, including focal length, principal point coordinates and distortion coefficient, can be calculated.

[0178] For example, in step S31, the Zhang Zhengyou calibration method is used to calibrate the camera intrinsic parameters. Using at least 10 frames of Aprilgrid images in different poses, the sub-pixel level coordinates of each corner point are extracted, constraint equations are constructed to solve for the initial values ​​of the intrinsic parameters, and then the reprojection error is minimized through nonlinear optimization to obtain accurate camera intrinsic parameters.

[0179] S32, based on the relevant information of the second type of marker 132 and the third type of marker 133 in the first sensing information, and the relevant information of the second type of marker 132 in the second sensing information, perform external parameter calibration on the second sensing information and the first sensor to be calibrated.

[0180] In this step, the relevant information of the second type of marker 132 in the first sensing information is the image of the second type of marker 132, the relevant information of the third type of marker 133 in the first sensing information is the image of the third type of marker 133, and the relevant information of the second type of marker 132 in the second sensing information is the point cloud data of the second type of marker 132.

[0181] For example, in step S32, the attitude of the calibration component 130 is first identified based on the image of the third type of marker 133 in the second sensing information. Based on the identified attitude of the calibration component 130, and combined with the image of the third type of marker 133 in the first sensing information, the two-dimensional center coordinates of the third type of marker 133 are identified. Combined with the point cloud data of the third type of marker 133 in the second sensing information, the three-dimensional center coordinates of the third type of marker 133 are identified. The two-dimensional and three-dimensional center coordinates of the third type of marker 133 are aligned, and the extrinsic parameters of the camera and radar are calculated based on the PnP algorithm.

[0182] S33, based on the second and third sensing information, performs external parameter calibration on the second sensing information and the third sensor to be calibrated.

[0183] In this step, the second sensing information is the point cloud information of the environment, which includes not only the point cloud of the calibration component 130, but also the point cloud data of surrounding static objects. The third sensing information is the position information collected by the inertial measurement unit.

[0184] For example, in step S33, the point cloud registration results are fused with the IMU pre-integration constraints through a joint optimization framework to construct a factor graph optimization model, solve the extrinsic parameter matrix and time synchronization parameters between the radar and the IMU, and achieve high-precision spatial and temporal joint calibration.

[0185] In this embodiment, based on the first sensing information, the second sensing information, and the third sensing information, the internal and external parameters of the first sensor to be calibrated, the second sensor to be calibrated, and the third sensor to be calibrated are jointly calibrated to achieve the joint calibration of the device to be calibrated 200.

[0186] In some examples, each set of detection information includes a first image of the first class label 131. Step S31 includes the following steps S311-S314.

[0187] S311, detect the corner coordinates of the first class of marker 131 in each first image.

[0188] S312, Construct the initial projection model of the camera.

[0189] S313, based on the initial projection model and corner coordinates, constructs an optimization function.

[0190] The optimization function is used to minimize the projection error of the initial projection model.

[0191] S314 uses a nonlinear optimization algorithm to solve the optimization function and obtains the intrinsic parameters of the first sensor to be calibrated when the projection error is minimized.

[0192] In this embodiment, based on the above steps, high-precision camera intrinsic parameters can be obtained, improving the overall accuracy of subsequent calibration.

[0193] In some examples, each set of detection information includes a second image and a first point cloud. The second image includes images of second-class marker 132 and third-class marker 133, and the first point cloud includes the corresponding point cloud of third-class marker 133. Step S32 includes the following steps S321-S324.

[0194] S321, Based on the first point cloud, determine the first center coordinates of the second type of marker 132. The first center coordinates are the center point coordinates of the second type of marker 132 in the three-dimensional coordinate system.

[0195] For example, step S321 includes: filtering the first point cloud to reduce noise in the first point cloud; extracting the target point cloud corresponding to the calibration component 130 in the first point cloud; clustering multiple first target points where the plane of the calibration component 130 in the target point cloud is located using the PCA algorithm; extracting multiple second target points corresponding to the first boundary of the calibration plate from the multiple first target points; reconstructing the first boundary from the multiple second target points; and extracting the coordinates of the center point corresponding to the first boundary to obtain the first center coordinates.

[0196] S322, Based on the second image, determine the second center coordinates of the second type of marker 132. The second center coordinates are the coordinates of the center point of the second type of marker 132 in the two-dimensional coordinate system.

[0197] For example, step S322 includes: preprocessing the second image; identifying the third type of marker 133 in the second image to obtain the pose of the calibration component 130; segmenting the second image to obtain sub-images corresponding to the calibration component 130; extracting the second boundary of the second marker in the sub-image using a density-based spatial clustering algorithm (DBSCAN algorithm); and extracting the coordinates of the center point corresponding to the second boundary to obtain the second center coordinates.

[0198] For example, preprocessing the second image includes: grayscale processing of the second image; and filtering the grayscale-processed second image. This effectively reduces image noise and improves the accuracy of subsequent marker recognition.

[0199] S323, align the first center coordinate and the second center coordinate.

[0200] S324, Based on the aligned first and second center coordinates, and using the PnP extrinsic parameter calculation method, determine the extrinsic parameters of the first and second sensors to be calibrated.

[0201] In this embodiment, the correspondence between the radar's three-dimensional coordinate system and the camera's two-dimensional coordinate system is constructed using the first and second center coordinates, and then the extrinsic parameter transformation matrix of the two systems is solved. Furthermore, by introducing the PnP algorithm and utilizing the known three-dimensional-two-dimensional point pair matching relationship, rotation and translation parameters are accurately estimated, achieving high-precision extrinsic parameter calibration between sensors and providing a reliable spatial alignment foundation for subsequent multi-sensor fusion.

[0202] In some examples, each set of detection information also includes a second point cloud and motion parameters. Step S33 includes the following steps S331-S336.

[0203] S331, calculate the first pose change of the motion data within the time interval between two consecutive second point clouds.

[0204] S332, compensates for the distortion of the second point cloud based on the first pose change.

[0205] S333 uses a point cloud matching algorithm to determine the second pose change between two second point clouds that are adjacent in time after compensation for distortion.

[0206] S334 uses initial extrinsic parameters to construct the residual function between the first pose change and the second pose change.

[0207] S335, adjust the initial extrinsic parameters to the target extrinsic parameters to reduce the residuals corresponding to the residual function.

[0208] S336, under the condition of minimum residual, determines the target extrinsic parameters as the extrinsic parameters of the radar and inertial measurement unit.

[0209] In this embodiment, motion data provided by the inertial measurement unit (IMU) is introduced to compensate for distortions in the point cloud caused by sensor motion. Furthermore, a point cloud matching algorithm is used to iteratively solve for pose changes, constructing a residual optimization model based on initial extrinsic parameters. Further, by minimizing the residual between the first and second pose changes, fine-tuning of the extrinsic parameters is achieved. When the residual converges to its minimum, the optimal spatial transformation relationship between the radar and the IMU is determined, thus obtaining the optimal extrinsic parameters between the radar and the IMU, completing the joint calibration of the radar and the IMU.

[0210] Figure 9 This is a schematic diagram of the chip system provided in an embodiment of this application.

[0211] like Figure 9 As shown, this application embodiment also provides a chip system 900, such as a SoC, which includes at least one processor 901 and at least one interface circuit 902. The processor 901 and the interface circuit 902 can be interconnected via lines. For example, the interface circuit 902 can be used to receive signals from other devices (e.g., the memory of an electronic device). As another example, the interface circuit 902 can be used to send signals to other devices (e.g., the processor 901 or the touchscreen of an electronic device). Exemplarily, the interface circuit 902 can read instructions stored in the memory and send the instructions to the processor 901. When the instructions are executed by the processor 901, the electronic device can perform the steps in the above embodiments. Of course, the chip system may also include other discrete devices, which are not specifically limited in this application embodiment.

[0212] This application also provides an electronic device, which includes a chip system 900 as described in the above embodiments.

[0213] This application also provides a computer-readable storage medium including computer instructions that, when executed on the electronic device, cause the electronic device to perform various functions or steps performed by the electronic device in the above method embodiments.

[0214] This application also provides a computer program product that, when run on an electronic device, causes the electronic device to perform various functions or steps performed by the electronic device in the above method embodiments.

[0215] Through the above description of the embodiments, those skilled in the art can clearly understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0216] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another apparatus, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0217] The units described as separate components may or may not be physically separate. A component shown as a unit can be one or more physical units; that is, it can be located in one place or distributed in multiple different locations. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0218] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0219] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, essentially or in other words, the parts that contribute to the prior art, or all or part of the technical solutions, can be embodied in the form of a software product. This software product is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0220] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0221] It should be noted that those skilled in the art, upon considering the specification and practicing the application disclosed herein, will readily conceive of other embodiments of this application. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and embodiments are to be considered exemplary only, and the true scope of this application is indicated by the claims.

[0222] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. A calibration system, characterized in that, include: The movable base (110) includes at least one movable track (111) with an initial position (112) on the movable track (111). A vehicle (120) is configured to move along the moving track (111), the vehicle (120) having a fixing device (121) configured to detachably mount the device to be calibrated (200). A calibration component (130) is fixedly disposed at a preset distance from the initial position (112); the calibration component (130) includes at least two types of markings, each type of marking being used for the calibration of a type of sensor; when calibrating the device to be calibrated (200), the calibration component (130) is located within the detection range of the device to be calibrated (200), and the detection direction of the device to be calibrated (200) is directed toward the calibration component (130). The carrier (120) is configured to move the device to be calibrated (200) along the moving track (111). The device to be calibrated (200) is configured to acquire detection information of the calibration component (130) at different positions on the moving track (111). The positions of the device to be calibrated (200) relative to the calibration component (130) are different at different positions.

2. The calibration system according to claim 1, characterized in that, The mobile base (110) includes: Platform (113), the height of the top surface of the platform (113) is a first height, the platform (113) includes the initial position (112) and the exit; The ramp (114) includes a first end (114a) and a second end (114b), the first end (114a) being connected to the outlet; the height of the first end (114a) gradually decreases from the height of the second end (114b); The moving track (111) includes the initial position (112) and the path on the ramp (114) from the first end (114a) to the second end (114b); The vehicle (120) is configured to start from the platform (113), enter the ramp (114) through the first end (114a) and move along the ramp (114) to the second end (114b).

3. The calibration system according to claim 2, characterized in that, The number of the slopes (114) is at least one; The exit of the platform (113) is set up in a one-to-one correspondence with the ramp (114); When there are multiple ramps (114), the ramps (114) extend in different directions, and the first end (114a) of each ramp (114) is connected to a different outlet of the platform (113).

4. The calibration system according to claim 1, characterized in that, The vehicle (120) also includes: A mobile chassis (122) is provided on the mobile chassis (122), and the device to be calibrated (200) is detachably provided on the fixing device (121). The mobile chassis (122) is configured to drive the fixed device (121) to move along the mobile track (111).

5. The calibration system according to claim 4, characterized in that, The device to be calibrated (200) is applied to the machine body; the fixing device (121) includes: The frame (121a) is mounted on the mobile chassis (122). At least one fixing part is provided on the frame (121a). When calibrating the device to be calibrated (200), the main body of the machine is fixedly mounted on the fixing part.

6. The calibration system according to claim 1, characterized in that, The device to be calibrated (200) includes at least two of a first sensor to be calibrated, a second sensor to be calibrated, and a third sensor to be calibrated; The first sensor to be calibrated is configured to acquire an image of the calibrator (130); The second sensor to be calibrated is configured to acquire a point cloud of an image of the calibrator (130); The third sensor to be calibrated is configured to acquire its own motion data.

7. The calibration system according to claim 6, characterized in that, The calibration component (130) includes: The first type of marker (131) is configured to be used for the first sensor to be calibrated to perform intrinsic parameter calibration; The second type of marker (132) is configured to be used for external parameter calibration of the first sensor to be calibrated and the second sensor to be calibrated; The third type of marker (133) is configured for external parameter calibration of the first sensor to be calibrated and the second sensor to be calibrated.

8. The calibration system according to any one of claims 1-7, characterized in that, Also includes: An electronic device configured to control the movement of the vehicle (120); The electronic device is also configured to calibrate the device to be calibrated (200) based on the detection information.

9. A calibration method, characterized in that, include: The control vehicle (120) moves along the moving track (111) of the moving base (110); the moving base (110) includes at least one of the moving tracks (111), and the moving track (111) includes an initial position (112); the vehicle (120) includes a fixing device (121), and the device to be calibrated (200) is detachably mounted on the fixing device (121). During the movement of the vehicle (120) along the moving track (111), at least two sets of detection information are acquired; wherein each set of detection information includes at least two sensor information, and each sensor information is acquired by a sensor to be calibrated included in the device to be calibrated (200); the positions on the moving track (111) corresponding to each set of detection information are different; Based on the detection information, each sensor in the device to be calibrated (200) is calibrated.

10. The calibration method according to claim 9, characterized in that, The calibration component (130) is fixedly disposed at a preset distance from the initial position (112). The calibration component (130) is located within the detection range of the device to be calibrated (200), and the detection direction of the device to be calibrated (200) is towards the calibration component (130). The relative positions of the device to be calibrated (200) and the calibration component (130) corresponding to each set of detection information are different. Each set of detection information includes at least two of the following: first sensing information, second sensing information, and third sensing information; the first sensing information is an image of the calibration component (130) acquired by the first sensor to be calibrated in the calibration device (200); the second sensing information is a point cloud of the calibration component (130) acquired by the second sensor to be calibrated in the calibration device (200); and the third sensing information is motion data of the third sensor to be calibrated in the calibration device (200) acquired by the third sensor to be calibrated itself.

11. The calibration method according to claim 10, characterized in that, The calibration component (130) includes at least two types of markings, each type of marking being used for the calibration of a type of sensor; the calibration component (130) includes a first type of marking (131), a second type of marking (132), and a third type of marking (133); the calibration of each sensor to be calibrated in the device to be calibrated (200) based on the detection information includes: Based on the relevant information of the first type of marker (131) in the first sensing information, the intrinsic parameters of the first sensor to be calibrated are calibrated; Based on the relevant information of the second type of marker (132) and the third type of marker (133) in the first sensing information, and the relevant information of the second type of marker (132) in the second sensing information, external parameter calibration is performed on the second sensing information and the first sensor to be calibrated. Based on the second sensing information and the third sensing information, external parameter calibration is performed on the second sensing information and the third sensor to be calibrated.

12. The calibration method according to claim 9, characterized in that, The mobile base (110) includes a platform (113) and a ramp (114); the platform (113) includes the initial position (112) and an exit, and the height of the top surface of the platform (113) is a first height; the ramp (114) includes a first end (114a) and a second end (114b), the first end (114a) being connected to the exit; the height from the first end (114a) to the second end (114b) gradually decreases; the control vehicle (120) moves along the mobile track (111) of the mobile base (110), including: Control the vehicle (120) to remain at the initial position (112) for a preset time; Control the vehicle (120) to move from the first end (114a) of the ramp (114) to the second end (114b) of the ramp (114). Control the vehicle (120) to move to the initial position (112) via the second end (114b) of the ramp (114).

13. The calibration method according to claim 12, characterized in that, When there are multiple ramps (114), the ramps (114) extend in different directions, and the first end (114a) of each ramp (114) is connected to a different exit of the platform (113); after controlling the vehicle (120) to move to the initial position (112) via the second end (114b) of the ramp (114), the method further includes: Control the vehicle (120) to move from the initial position (112) to other slopes (114).

14. A chip system, characterized in that, include: A memory and one or more processors; the memory is coupled to the processors; wherein the memory stores computer program code, the computer program code including computer instructions, which, when executed by the processor, cause the electronic device to perform the calibration method as described in any one of claims 9 to 13.

15. An electronic device, characterized in that, include: The chip system as described in claim 14.

16. A computer-readable storage medium, characterized in that, Includes computer instructions that, when executed on an electronic device, cause the electronic device to perform the calibration method as described in any one of claims 9 to 13.

17. A computer program product, characterized in that, When the computer program product is run on a computer, the computer performs the calibration method as described in any one of claims 9 to 13.

Images