Calibration room, calibration method, calibration device, calibration system and storage medium

Abstract

The invention provides a calibration room, a calibration method, a calibration device, a calibration system and a storage medium. The calibration room comprises a first area, a second area, a third area, a fourth area and a fifth area, a calibration position is arranged in the fifth area, the calibration position is used for parking a vehicle, and the vehicle is used for loading a sensor to be calibrated; the first area, the second area, the third area and the fourth area are located on the periphery of the calibration position, calibration plates are arranged in the first area, the second area, the third area and / or the fourth area, and the calibration plates are used for calibrating parameters of a sensor to be calibrated; the calibration plate is located in the view field range of the sensor to be calibrated, and the calibration plate is provided with marks and / or calibration holes. The method is suitable for parameter calibration of various sensors, has high universality, and can meet the requirements of automatic driving vehicle production line calibration for robust, accurate and high-efficiency internal and external parameter calibration.

Description

Technical Field

[0001] The present disclosure relates to a calibration room, a calibration method, a calibration device, a calibration system and a storage medium. Background Art

[0002] Sensor calibration in the autonomous driving field involves calibrating the internal and external parameters of sensors such as cameras, Lidar (Light Detection and Ranging), millimeter-wave radar, inertial navigation, and integrated navigation. This ensures the coordinates of each sensor are unified and provides reliable input for perception algorithms. Calibration methods for cameras and other sensors are mainly divided into target-based calibration and non-target-based calibration methods. Non-target-based calibration methods can be further divided into environmental scene-based calibration methods and motion-based calibration methods.

[0003] At present, in the target-based calibration method, problems such as high spatial distribution requirements of the calibration plate, insufficient number of feature points, and poor feature consistency still exist. As a result, the target-based camera calibration has low robustness and versatility. At the same time, the success rate and accuracy of the calibration are difficult to guarantee, and it cannot be applied to mass production calibration in the field of autonomous driving. Summary of the Invention

[0004] In view of this, the present disclosure provides a calibration room, a calibration method, a calibration device, a calibration system and a storage medium to improve the robustness, accuracy and versatility of a target-based calibration method.

[0005] According to a first aspect of the present disclosure, a calibration room is provided, wherein the calibration room includes a first area, a second area, a third area, a fourth area and a fifth area; a calibration position is provided in the fifth area, the calibration position is used to park a vehicle, and the vehicle is used to load a sensor to be calibrated; the first area, the second area, the third area and the fourth area are located around the calibration position, and the first area, the second area, the third area and / or the fourth area are provided with a calibration plate, and the calibration plate is used to calibrate the parameters of the sensor to be calibrated; the calibration plate is located within the field of view of the sensor to be calibrated and is provided with marks and / or calibration holes.

[0006] It can be seen from the above technical solution that the calibration room of the disclosed embodiment can arrange calibration plates in the forward space, lateral space and rearward space of the vehicle as needed, which can meet the calibration needs of various sensors. It is suitable for parameter calibration of various sensors such as short-focus cameras, long-focus cameras, fisheye cameras, lidar, inertial navigation, combined navigation, etc. It can also be used for pairwise calibration and joint calibration between sensors. It has high versatility and can meet the calibration needs of autonomous driving vehicle production lines for robust, accurate and efficient internal and external parameter calibration. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0008] Figure 1 A schematic diagram of the spatial structure of a calibration room provided in an embodiment of the present disclosure;

[0009] Figure 2 An example diagram of the deployment of a calibration room and its calibration board provided in an embodiment of the present disclosure;

[0010] Figure 3 This is an example diagram of a calibration plate involved in an embodiment of the present disclosure;

[0011] Figure 4 A schematic diagram of characteristic points on a calibration plate involved in an embodiment of the present disclosure;

[0012] Figure 5 A schematic diagram showing the height of a calibration plate and its distance and azimuth angle relative to a vehicle involved in an embodiment of the present disclosure;

[0013] Figure 6 A flow chart of a calibration method provided in an embodiment of the present disclosure;

[0014] Figure 7 A schematic flow chart of another calibration method provided in an embodiment of the present disclosure;

[0015] Figure 8 A schematic block diagram of a calibration device in the form of an electronic device provided by an embodiment of the present disclosure;

[0016] Figure 9 A schematic diagram of the structure of the calibration system provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0017] The following will clearly and completely describe the technical solutions in the embodiments of the present disclosure in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present disclosure, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present disclosure without making any creative efforts shall fall within the scope of protection of the present disclosure.

[0018] The terms used in the embodiments of the present disclosure are for the purpose of describing specific embodiments only and are not intended to limit the present disclosure. The singular forms "a," "an," "the," and "the" used in the embodiments of the present disclosure and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise.

[0019] It should be understood that the term "and / or" as used herein is merely a description of the relationship between associated objects, indicating that three possible relationships exist. For example, "A and / or B" can represent: A exists alone, A and B exist simultaneously, or B exists alone. Furthermore, the character " / " in this document generally indicates that the associated objects are in an "or" relationship.

[0020] The word "if," as used herein, may be interpreted as "at the time of" or "when" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrases "if it is determined" or "if (stated condition or event) is detected" may be interpreted as "when it is determined" or "in response to the determination" or "when detecting (stated condition or event)" or "in response to detecting (stated condition or event)," depending on the context.

[0021] Explanation of terms:

[0022] Vehicle coordinate system: For example, a three-dimensional rectangular coordinate system, with the origin typically being the vehicle's origin. In one example, the vehicle's origin is located on the vehicle's center axis, with the positive x-axis pointing perpendicular to the vehicle's frontal axis and to the right. The positive y-axis is along the vehicle's frontal axis, and the positive z-axis is opposite to the direction of gravity. The xy coordinate plane is a horizontal plane perpendicular to the direction of gravity, and the heading angle yaw is consistent with the vehicle's forward direction.

[0023] Reference coordinate system: For example, a total station coordinate system. In one example, the reference coordinate system can be a three-dimensional rectangular coordinate system. The origin of the reference coordinate system can be set to be consistent with the origin of the coordinate system between calibrations. The positive x-axis is perpendicular to the vehicle's front direction and points to the right of the vehicle. The positive y-axis is along the vehicle's front direction. The positive z-axis is opposite to the direction of gravity. The xy coordinate axis plane is a horizontal plane perpendicular to the direction of gravity. In specific applications, the reference coordinate system can be ensured to be consistent with the coordinate system between calibrations by setting the translation amount, placing it on the center axis of the alignment device, and adjusting it.

[0024] Calibration room coordinate system: This can be, for example, a three-dimensional rectangular coordinate system. If there are multiple calibration rooms, each calibration room has a corresponding calibration room coordinate system. In one example, the xy plane of the calibration room coordinate system is a plane perpendicular to the direction of gravity, the heading angle yaw is oriented toward the center axis of the centering device, and the origin of the calibration room coordinate system can be the center of the front end of the centering device. For example, the orientation of the calibration room coordinate system can be: the positive direction of the x-axis is toward the right side of the vehicle, the positive direction of the y-axis is consistent with the direction of the vehicle's front, and the z-axis is perpendicular to the xy plane and its positive direction is opposite to the direction of gravity.

[0025] Calibration plate coordinate system: Each calibration plate has its own calibration plate coordinate system, such as a three-dimensional rectangular coordinate system. In one example, the origin of the calibration plate coordinate system can be the center of the calibration plate surface, with the positive x-axis pointing to the right of the calibration plate, the positive y-axis pointing upward, the z-axis perpendicular to the xy plane and pointing outward, the x-axis and y-axis planes parallel to the calibration plate surface, and the z-axis perpendicular to the calibration plate surface.

[0026] Marker coordinate system: Each marker has a corresponding marker coordinate system. The marker coordinate system is, for example, a three-dimensional rectangular coordinate system with a specific point in the marker (the center point or a vertex of the marker) as the origin. In some embodiments, the area formed by the horizontal axis x and the vertical axis y of the three-dimensional rectangular coordinate system is the area where the marker is located, and the z-axis of the three-dimensional rectangular coordinate system is perpendicular to the area where the marker is located. For example, the x-axis is pointing to the right of the calibration plate, the y-axis is pointing to the top of the calibration plate, and the z-axis is perpendicular to the xy plane and points outward.

[0027] Sensor coordinate system: A coordinate system with a specific point on the sensor as its origin, such as a 3D rectangular coordinate system. Each sensor to be calibrated has its own corresponding sensor coordinate system (referred to as the sensor coordinate system).

[0028] Camera coordinate system: A coordinate system with the optical center of the camera lens as its origin, which can be a three-dimensional rectangular coordinate system.

[0029] Pixel coordinate system: Usually defined in such a way that the origin is located in the upper left corner of the image, the horizontal axis (u) is parallel to the horizontal axis of the camera coordinate system to the right, and the vertical axis (v) is parallel to the vertical axis of the camera coordinate system downward.

[0030] LiDAR coordinate system: a coordinate system with the optical center of the LiDAR as its origin, such as a three-dimensional rectangular coordinate system.

[0031] The extrinsic parameters of A and B include the transformation relationship between the A coordinate system and the B coordinate system, and the extrinsic parameters of A include the transformation relationship between the A coordinate system and other coordinate systems. The transformation relationship is not limited to rotation matrices and translation vectors.

[0032] External parameters between the camera and the calibration: including the transformation relationship between the camera coordinate system and the calibration coordinate system. This transformation relationship may include a rotation matrix and a translation vector. The rotation matrix and the translation vector determine the position and orientation of the camera in the calibration space.

[0033] External parameters between the lidar and the calibration room: including the transformation relationship between the lidar coordinate system and the calibration room coordinate system. The transformation relationship may include a rotation matrix and a translation vector. The rotation matrix and translation vector determine the position and orientation of the lidar in the calibration room.

[0034] Extrinsic parameters between the navigation device and the calibration: These include the transformation relationship between the navigation coordinate system and the calibration coordinate system. This transformation relationship includes the rotation matrix and translation variables. The rotation matrix and translation vector determine the position of the navigation device within the calibration space. Navigation devices include, but are not limited to, inertial navigation or integrated navigation. Integrated navigation includes, but is not limited to, inertial measurement units (IMUs) and global navigation satellite systems (GNSSs).

[0035] High reflectivity points: Feature points with high laser intensity values ​​in the point cloud data. High reflectivity points on the calibration plate can be feature points with higher reflectivity than surrounding points on the calibration plate. High reflectivity points can be formed by placing high-reflectivity materials at specific locations on the calibration plate.

[0036] Calibration room

[0037] An embodiment of the present disclosure provides a calibration room, which can be used for parameter calibration of various sensors, specifically including internal parameter calibration and / or external parameter calibration. Figure 1 FIG. 1 shows an exemplary structural diagram of a calibration room according to an embodiment of the present disclosure. Figure 2 A schematic diagram of exemplary calibration plate deployment in a calibration room in an embodiment of the present disclosure is shown, wherein the black-filled quadrilateral shape represents the calibration plate.

[0038] See also Figure 1 and Figure 2 The calibration room 100 may include: a first area 110 , a second area 120 , a third area 130 , a fourth area 140 and a fifth area 150 .

[0039] The first area 110, the second area 120, the third area 130, the fourth area 140 and the fifth area 150 are all three-dimensional spaces, each area has a first dimension along the direction of gravity and two dimensions perpendicular to the direction of gravity, and the two dimensions perpendicular to the direction of gravity are: the second dimension parallel to the direction of the vehicle's front and the third dimension perpendicular to the direction of the vehicle's front 160.

[0040] A calibration position is provided in the fifth area 150 , and the calibration position is used to park a vehicle 160 , and the vehicle 160 can be used to load a sensor to be calibrated.

[0041] The first area 110, the second area 120, the third area 130, and the fourth area 140 may be located around the fifth area 150, respectively. One or more calibration plates may be disposed in any one or more of the first area 110, the second area 120, the third area 130, and the fourth area 140. The calibration plates may be used to calibrate the parameters of the sensor to be calibrated. The calibration plates are located within the field of view of the sensor to be calibrated so that the sensor to be calibrated can collect data related to the calibration plates (e.g., images, point clouds, etc.) to achieve calibration of its parameters.

[0042] The first area can be located in the first direction of the calibration position, the second area can be located in the second direction of the calibration position, the third area can be located in the third direction of the calibration position, and the fourth area can be located in the fourth direction of the calibration position. The first direction is opposite to the third direction, and the second direction is opposite to the fourth direction. The first direction is the direction of the vehicle's front, the second direction and the fourth direction are both perpendicular to the vehicle's direction of travel, and the third direction is opposite to the vehicle's direction of travel. Thus, the calibration room 100 is divided into multiple areas with the calibration position as a reference object, making it easy to arrange calibration plates in the three-dimensional space of the front, rear, left and right sides of the vehicle, thereby meeting the actual calibration needs of various sensors and various application scenarios.

[0043] In one example, the fifth area 150 may be the middle area of ​​the calibration room 100, and the first area 110, the second area 120, the third area 130, and the fourth area 140 may be areas surrounding the fifth area 150 in the calibration room 100. That is, the first area 110 may be the front area of ​​the vehicle in the calibration room 100, the second area 120 may be the left area of ​​the vehicle in the calibration room 100, the third area 130 may be the right area of ​​the vehicle in the calibration room 100, and the fourth area 140 may be the rear area of ​​the vehicle in the calibration room 100. In specific applications, the first area 110, the second area 120, the third area 130, and the fourth area 140 may be adjacent to each other, or may be spaced apart as needed.

[0044] The calibration room is more conducive to forming a standardized calibration operation process. By arranging calibration plates (also called targets) suitable for feature extraction of various types of sensors in the calibration room, the robustness and accuracy of sensor parameter calibration can be improved. It can also be connected to the vehicle production process to realize sensor calibration before the vehicle leaves the factory and ensure the safety of autonomous driving.

[0045] The calibration plate and related details involved in the embodiments of the present disclosure are described in detail below.

[0046] A calibration plate, also known as a target, is primarily used for sensor calibration. The plate can be equipped with markings, calibration holes, and / or any other shapes or objects useful for sensor parameter calibration. The number of markings and calibration plates per plate is unlimited and can be flexibly configured based on actual calibration needs. To better capture valid features during calibration, the markings and calibration holes on the plate can be arranged symmetrically.

[0047] The marker can be printed, drawn or pasted on the calibration plate. The marker can be a regular shape such as a quadrilateral, triangle, circle or other. The marker can also be an irregular shape. In some embodiments of the present disclosure, the marker can be but is not limited to a QR code, a picture, etc. Each marker can have a unique marker ID, which can be used to uniquely represent a marker. For example, the marker can be but is not limited to an ArUco marker, i.e., an ArUco QR code.

[0048] The calibration hole is preferably a regular shape. For example, the calibration hole can be, but is not limited to, a circular through hole (referred to as a circular hole), a square through hole, a rectangular through hole, or a through hole of any other shape.

[0049] Figure 3 An example diagram of a calibration plate according to an embodiment of the present disclosure is shown. Figure 3 The calibration plate 300 is provided with four symmetrically distributed marks and four symmetrically distributed circular holes. The four marks can be QR codes or pictures. Each mark can have the same size, and each circular hole can have the same size.

[0050] Each calibration plate may have a unique board identifier (board id), which is used to uniquely identify a calibration plate. Different calibration plates may be distinguished by the board identifier, which includes but is not limited to the number, position, name, or other information that can uniquely represent the calibration plate. For example, the board identifier of each calibration plate may be numbered according to the type of calibration plate and the markings on the calibration plate. Different calibration plates may be provided with different markings and / or different calibration holes. Different calibration plates or different markings on the same calibration plate may be distinguished by the marking identifier, and different calibration holes may be distinguished or not distinguished as needed.

[0051] Marker identifiers and plate identifiers can be associated. To facilitate management, when designing a calibration room, you can generate the required number and style of calibration plates based on the plate identifier values ​​from low to high according to predetermined rules, based on the number of calibration plates required. At this point, the calibration plate has the plate identifier and the marker identifiers of the multiple marks it contains. The correspondence between the plate identifier and the marker identifier can be saved as the calibration plate's attribute information for subsequent calibration processes.

[0052] The characteristic points of the calibration plate may include but are not limited to a first characteristic point about the mark and a second characteristic point about the structural contour of the calibration plate. The second characteristic point may include one or both of the following: a first reference point about the outer contour of the calibration plate, a second reference point about the calibration hole in the calibration plate.

[0053] The first feature point can be pre-selected as needed. In order to more accurately capture the features of the first feature point, the first feature point can be a feature point on the outline of the mark, for example, Figure 4 For example, the first feature point can be Figure 4 In the example, the four corner points p5 to p8 of the quadrilateral are marked.

[0054] The second feature point can also be pre-selected according to actual needs. Similarly, in order to facilitate accurate capture of feature points, the second feature point can preferably be a structural feature point about the structural contour of the calibration plate. Specifically, the second feature point may include but is not limited to: a first reference point about the outer contour of the calibration plate and / or a second reference point about the calibration hole in the calibration plate. The first reference point may include but is not limited to the edge corner point, high reflection point, etc. of the calibration plate. The second reference point can be a corner point of a polygonal calibration hole or the center point of a circular calibration hole. The corner point of the embodiment of the present disclosure is the vertex.

[0055] In one example, if the calibration plate is not provided with a calibration hole, the second feature point may include the first reference point. For example, the feature point on the contour of the calibration plate may include, for example Figure 4 The four edge corner points p1 to p4 of the quadrilateral calibration plate in the example.

[0056] In one example, if a calibration hole is provided on the calibration plate, the second feature point may include a first reference point and a second reference point. Figure 4 For example, the second feature points may include four edge corner points p1 to p4 of the calibration plate and a center point p9 of a circular calibration hole on the calibration plate.

[0057] The attribute information of the calibration plate may include but is not limited to the plate identification of the calibration plate, the outline size of the calibration plate, the number of marks on the calibration plate, the number of calibration holes on the calibration plate, the mark identification of each mark on the calibration plate, the correspondence between the plate identification and the mark identification, and other information.

[0058] The attribute information of the calibration plate also includes attribute information of the markers. The attribute information of the markers may include, but is not limited to, the plate identification of the calibration plate, the position information of the markers, the marker identification, the outline size, the number of first feature points, and the relative positional relationship of each first feature point with respect to a specific corner point of the calibration plate. The positional information of the markers may include the relative positional relationship of the center point or multiple corner points of different markers with respect to a specific corner point of the calibration plate, as well as the relative positional relationship between different markers.

[0059] The calibration plate's attribute information also includes the attribute information of the calibration holes. This attribute information may include, but is not limited to, the plate identifier of the calibration plate, the location information of the calibration hole, the calibration hole's outline dimensions, and the number of structural feature points of the calibration hole. The calibration hole's location information includes the relative positional relationship of the calibration hole's center point or multiple corner points relative to a specific corner point of the calibration plate, as well as the relative positional relationship between different calibration holes.

[0060] In one example, the position information of the mark can also be represented by the adjacent edge distance between the mark and the calibration plate, and the adjacent edge distance between the mark and the mark. Similarly, the position information of the calibration hole can also be represented by the adjacent edge distance between the calibration hole and the calibration plate, the adjacent edge distance between the calibration hole and the mark, and the adjacent edge distance between the calibration hole and the calibration hole. Among them, the adjacent edge distance between the mark and the calibration plate includes the distance between the edge of the mark and the adjacent edge of the calibration plate, and the adjacent edge distance between the mark and the mark includes the distance between the edge of the mark and the adjacent edges of other marks. The adjacent edge distance between the calibration hole and the calibration plate includes the distance between the center point or other point of the calibration hole and the adjacent edge of the calibration plate.

[0061] Outline dimension information can be represented by side length. For example, the outline dimension information of a calibration plate may include the length and width of the calibration plate, the outline dimension information of each mark may include the length and width of the mark, and the outline dimension information of a calibration hole may include the diameter or radius of a circular hole, or the length and width of a quadrilateral hole.

[0062] The information of the second feature point may include the position and identification of the feature point on the outline of the calibration plate, the position and identification of the center point of the calibration hole, such as the position and identification of multiple corner points and / or the center point of the calibration hole.

[0063] The information of the first feature point may include the position and identification of the first feature point. Figure 3 Taking the calibration plate as an example, if the first feature points are selected as the four corner points of the mark, the information of the corresponding feature points may include the identifiers of the four corner points on the mark and their distances to the edge of the calibration plate.

[0064] The attribute information of the calibration plate, the attribute information of the mark, and the attribute information of the calibration hole can be stored in association with each other through the plate identifier, the mark identifier, and the like.

[0065] In specific applications, different types of calibration plates can be designed as needed to calibrate the parameters of different sensors in different scenarios. To facilitate feature capture and calculation of various sensor parameter calibrations, different calibration plates, different markings on the plates, and different calibration holes on the plates can be of the same or different sizes, and this disclosure does not impose any restrictions on this.

[0066] In the embodiments of the present disclosure, the sensors to be calibrated may include, but are not limited to, one or more of the following: cameras, lidars, millimeter-wave radars, inertial navigation, and integrated navigation. Cameras include, but are not limited to, fisheye cameras, short-focus cameras, and long-focus cameras. Lidars include, but are not limited to, short-range lidars, medium-range lidars, and long-range lidars.

[0067] To adapt to specific application scenarios and the calibration requirements of various sensors, the types of calibration plates involved in the embodiments of the present disclosure may include, but are not limited to, one of the following: a first calibration plate, a second calibration plate, a third calibration plate, a fourth calibration plate, and a fifth calibration plate. The first calibration plate is provided with a first number of marks, the second calibration plate is provided with a second number of marks, the third calibration plate is provided with a third number of marks and a third number of calibration holes, the fourth calibration plate is provided with a fourth number of marks, and the fifth calibration plate is provided with a fifth number of calibration holes. The values ​​of the first number, the second number, the third number, the fourth number, and the fifth number are different, and their specific values ​​can be flexibly set according to actual needs. The sizes of the first calibration plate, the second calibration plate, the third calibration plate, the fourth calibration plate, and the fifth calibration plate can be the same or different. The number of each type of calibration plate in the calibration room can also be flexibly set according to needs, that is, one or more first calibration plates, one or more second calibration plates, one or more third calibration plates, one or more fourth calibration plates, and one or more fifth calibration plates can be set in the calibration room. The number of calibration plates of each type set in each area can also be flexibly adjusted as needed. By setting up multiple types of calibration plates in the calibration room, parameter calibration of various sensors can be achieved, thus meeting the calibration needs of various specific application scenarios.

[0068] In one example, the first and second calibration plates are each provided with only multiple marks, without calibration holes, and are thus pure mark plates, or multi-mark plates. The fourth calibration plate is provided with a large mark covering most of the fourth calibration plate, without calibration holes, and is also a pure mark plate, or a single mark plate. The third calibration plate is a mixed plate, provided with both one or more marks and one or more calibration holes. The fifth calibration plate is provided with no marks but with multiple calibration holes, and is a pure calibration hole plate, or a multi-hole plate.

[0069] In one example, the first calibration plate may be a 3x3 calibration plate, and the second calibration plate may be a 2x2 calibration plate. Specific specifications of the various calibration plates may be as follows:

[0070] 3x3 calibration plate: The reserved plate identifier has a value range of 0-99, and the reserved mark identifier has a value range of 0-299. There are 9 marks, each of which can include 4 corner points, which can provide 4x9 mark corner point features with ID identification;

[0071] 2x2 calibration plate: The value range of the reserved plate identifier is 100-199, and the value range of the reserved mark identifier is 300-499. It has 4 markers and can provide 4x4 marker corner point features with ID identifiers;

[0072] Mixed calibration plate: The value range of the reserved plate identifier is 200-299, the value range of the reserved mark identifier is 500-699, with m marks and m calibration holes (m is a positive integer), which can provide mx4 mark corner point features with ID identifiers and m calibration hole center point features;

[0073] Single-mark board: The value range of the reserved board identifier is 300-399, and the value range of the reserved mark identifier is 700-799. It has 1 mark and can provide 1x4 mark corner point features with ID identification. The mark of the single-mark board is larger than the mark of the multi-mark board to provide clearer mark features;

[0074] Multi-hole calibration plate: The value range of the reserved plate identification is 400-499, with n calibration holes (n≥2), which can provide n calibration hole center point features with ID identification.

[0075] It should be understood that different types of calibration plates can be arranged in each area of ​​the calibration room, and multiple calibration plates of the same type can be set. In order to facilitate different types of calibration plates, the embodiment of the present disclosure divides the identification intervals for different types of calibration plates in advance, and the calibration plates of the same type take values ​​in this interval. Similarly, for different types of calibration plates, the value intervals of the marking identifiers can also be preset. Then, by determining the type of calibration plate, the value intervals of the marking identifiers in the calibration plate can be known, and the marking identifiers finally calculated can be verified. For example, the preset value interval of the mixed calibration plate is 200-299, and the plate identification of the mixed calibration plate selected in the calibration room can be 205, 210, 215, 220, etc.

[0076] In some embodiments, the calibration plate identification can also be determined according to the hole distribution by customizing the hole distribution of the calibration plate, that is, different hole distributions represent different calibration plate identifications. Each calibration plate has multiple hole positions, each calibration plate has a hole at at least one hole position, each hole position has a corresponding binary code according to whether there is a hole at the hole position, each calibration plate has a corresponding plate code according to the binary codes of the multiple hole positions, and each plate code is associated with a unique plate identification. The binary code includes 0 and 1, and the code is 1 if there is a hole and 0 if there is no hole, and vice versa. For example, for a 4*4 calibration plate, there are 16 hole positions in 4*4. If the calibration plate has holes at all 16 hole positions, the plate code of the calibration plate is "1111111111111", and the plate code has a corresponding plate identification. If the calibration plate has no hole at hole 1 and has holes at the other 15 holes, the plate code of the calibration plate is "011111111111", which also has a corresponding plate identifier. If the calibration plate has no holes at hole 3, hole 5, hole 11, or hole 16, the plate code of the calibration plate is "1101011111011110", which also has a corresponding plate identifier.

[0077] The image data collected by the camera and / or the point cloud data collected by the lidar can be used to identify which hole positions in the calibration plate have holes and which do not, thereby obtaining the plate code of the calibration plate and then the corresponding plate identification of the calibration plate.

[0078] Moreover, the calibration room of the embodiment of the present disclosure may include multiple calibration plates with holes (hereinafter referred to as calibration plates with holes), each calibration plate with holes having a hole at at least one hole position. Each hole position has a corresponding binary code according to whether there is a hole at the hole position, and each calibration plate with holes has a corresponding plate code according to the binary codes of the multiple hole positions, and the Hamming distance between the plate codes of any two calibration plates in the multiple calibration plates is greater than or equal to a preset threshold. The preset threshold can be set according to the number of hole positions. For example, for a 9-hole calibration plate, the preset threshold can be 3, and the Hamming distance between the plate codes of any two calibration plates is greater than or equal to 3.

[0079] Furthermore, the hole positions of the calibration plate include a first hole position and a second hole position, the hole located at the first hole position is the first calibration hole, and the hole located at the second hole position is the second calibration hole. The second hole position is located in the boundary area of ​​the calibration plate, and the first hole position is located in the middle area of ​​the calibration plate. There must be a hole at the second hole position, and there may or may not be a hole at the first hole position. The size of the second calibration hole is larger than the size of the first calibration hole, so that the second calibration hole can be determined first, and the position of the first calibration hole can be determined based on the positional relationship between the second calibration hole and the first calibration hole. Here, the positional relationship between the second calibration hole and the corner point of the calibration plate, the positional relationship between the second calibration hole and the first calibration hole, and the positional relationship between the first calibration hole and the second calibration hole can all be pre-stored, so that the position of the first calibration hole can be determined based on the position of the calibration plate corner point and / or the second calibration hole in the sensor data.

[0080] In some embodiments, when customizing a perforated calibration plate, a plate code library can be stored to determine the corresponding calibration plate style based on the plate codes in the library. In one example, if the calibration plate has n hole positions, there are 2n corresponding plate codes. A predetermined number of plate codes with a Hamming distance greater than or equal to a preset threshold are selected from these 2n plate codes to form the plate code library, and the corresponding calibration plate is produced. If the plate code of the calibration plate is identified from the sensor data, the identified plate code can be checked to see if it exists in the plate code library to verify whether the identification result is correct.

[0081] Different types of calibration plates can be used to calibrate different types of sensors. A single calibration plate can be used to calibrate multiple sensors simultaneously. If a calibration plate is placed within the field of view of two or more sensors to be calibrated, it can be reused to calibrate these two or more sensors. If a calibration plate is placed within the field of view of a specific sensor to be calibrated, it can be used exclusively for calibrating that sensor.

[0082] In specific applications, the placement and layout of various calibration plates in the calibration room can be flexibly adjusted based on actual needs. In practical applications, the calibration accuracy of the sensor to be calibrated can be improved by placing various calibration plates within the field of view (FOV) of the sensor to be calibrated.

[0083] In some embodiments, see Figure 2 Multiple calibration plates of the same type in the first, second, third, and / or fourth areas may have different distances, heights, and / or azimuth angles relative to the vehicle. When multiple calibration plates are provided, the different calibration plates may be distributed at different spatial locations, heights, distances, and angles within the field of view of the sensor to be calibrated. Figure 5The distances ( L1 , L2 ), heights ( h1 , h2 ), and orientation angles ( a1 , a2 ) of two calibration plates 171 , 172 of the same type relative to the vehicle 160 in the second region are shown.

[0084] In some embodiments, see Figure 2 , the arrangement of multiple calibration plates of the same type in the first area, the second area, the third area and / or the fourth area includes one or more of the following: 1) arranged in a single row or multiple rows in a plane that forms a predetermined angle with the direction of the vehicle's head; 2) arranged in a single column or multiple columns in a plane that forms a predetermined angle with the direction of the vehicle's head; 3) arranged in a curved shape in a plane that forms a predetermined angle with the direction of the vehicle's head; 4) arranged in a single row or multiple rows in multiple planes that form a predetermined angle with the direction of the vehicle's head, and the multiple planes may be parallel planes. Among them, the predetermined angle between the plane to which different types of calibration plates belong and the direction of the vehicle's head can range from 0° to 180°. The specific value of the predetermined angle is related to the type of calibration plate, the type of sensor to be calibrated, etc., and can be flexibly set according to actual needs. It can also be adaptively adjusted based on the calibration needs of specific application scenarios.

[0085] Below Figure 2 The following example illustrates the arrangement of various calibration plates in different areas. The spatial layout of multiple calibration plates can be symmetrically distributed around the central axis of the centering device, around the central axis of the calibration room, or around the parking position of the vehicle during calibration. The layout of one side can be used to determine the layout of the other side.

[0086] Multiple 3x3 calibration plates can be arranged in the first, second, and fourth areas, respectively. These 3x3 calibration plates are all located within the field of view of the fisheye camera, and the heights, distances from the vehicle, and / or orientation angles of the multiple 3x3 calibration plates in each area are different. In the first area, two rows of 3x3 calibration plates are arranged on a plane at a first predetermined angle to the vehicle's front direction, with four 3x3 calibration plates in each row, and adjacent 3x3 calibration plates are fixedly spaced. In the second area, two columns of 3x3 calibration plates are arranged on a plane at a second predetermined angle to the vehicle's front direction, with four 3x3 calibration plates in each column, and adjacent 3x3 calibration plates are fixedly spaced. In the fourth area, two columns of 3x3 calibration plates are arranged on a plane at a third predetermined angle to the vehicle's front direction, with four 3x3 calibration plates in each column, and adjacent 3x3 calibration plates are fixedly spaced. The specific values ​​of the first predetermined angle, the second predetermined angle, and the third predetermined angle can be flexibly set as needed and can also be adjusted based on the specific situation of the fisheye camera. For example, the first predetermined angle, the second predetermined angle, and the third predetermined angle point can each be an angle less than 90 degrees. The plane in which the 3x3 calibration plates are arranged in the second area and the plane in which the 3x3 calibration plates are arranged in the fourth area can be symmetrical with respect to the vehicle's centerline. In this way, multiple 3x3 calibration plates in each area are regularly arranged in a plane that is inclined at a certain angle to the plane where the calibration position is located, which can better meet the calibration requirements of sensors such as fisheye cameras.

[0087] The multiple single-marker plates in the first and third regions can be arranged in a single row or multiple rows in a plane perpendicular to the vehicle's frontal orientation. The multiple single-marker plates in the second and fourth regions can be arranged in a single row or multiple rows in a plane parallel to the vehicle's frontal orientation. The number of single-marker plates arranged in the first and third regions can be different, while the number of single-marker plates arranged in the second and fourth regions can be the same and symmetrically distributed about the vehicle's centerline.

[0088] Hybrid calibration plates can be placed in the first, second, third, and fourth areas. These hybrid calibration plates can be located within the field of view of the short-focus camera, the field of view of the lidar, or the common view of the lidar and the short-focus camera. The distance, height, and azimuth angle of the multiple hybrid calibration plates in each area relative to the vehicle are different. The multiple hybrid calibration plates in the second and fourth areas can be arranged in a single or multiple rows in multiple parallel planes at predetermined angles to the vehicle's frontal orientation. The predetermined angle can be flexibly set or adaptively adjusted as needed. Each row has three hybrid calibration plates. Because different planes have different heights and different distances from the vehicle, placing hybrid calibration plates in multiple planes is more conducive to sensors such as lidar capturing feature points in different depth directions, thereby improving the parameter calibration accuracy of such sensors. The multiple hybrid calibration plates in the first area can be arranged in a curved pattern in a plane perpendicular to the vehicle's frontal orientation to meet the calibration requirements of the short-focus camera and / or lidar. The multiple hybrid calibration plates in the third area can be arranged in an array in a plane perpendicular to the direction of the vehicle's front to meet the calibration requirements of short-focus cameras and / or lidars, and each array includes two rows and two columns of hybrid calibration plates.

[0089] Multiple 2x2 calibration plates can be arranged in the first area. These 2x2 plates can be located within the field of view of the short-focus camera, the field of view of the LiDAR, or the common field of view of the LiDAR and the short-focus camera. The plates can be positioned at varying distances and orientations relative to the vehicle. The 2x2 plates in the first area can be arranged in a curved pattern in a plane perpendicular to the vehicle's frontal orientation to accommodate calibration requirements for the short-focus camera and / or LiDAR.

[0090] The first area can be equipped with 2x2 calibration plates. These 2x2 plates can be located within the field of view of the short-focus camera, the field of view of the LiDAR, or the common field of view of the LiDAR and the short-focus camera. The plates can be positioned at varying distances and orientations relative to the vehicle. The 2x2 plates in the first area can be arranged in a curved pattern in a plane perpendicular to the vehicle's frontal orientation to accommodate calibration requirements for the short-focus camera and / or LiDAR.

[0091] Multiple porous calibration plates can be placed in the first area. These plates are located within the LiDAR's field of view and at varying distances and azimuths relative to the vehicle. The plates in the first area can be arranged in a single row in a plane at a predetermined angle relative to the vehicle's frontal orientation to accommodate LiDAR calibration requirements. This predetermined angle can be an acute angle less than 90 degrees and can be flexibly set or adaptively adjusted as needed.

[0092] The following examples illustrate exemplary specific implementations of deploying different types of calibration plates for different types of sensors to be calibrated.

[0093] Example 1

[0094] When the sensors to be calibrated include fisheye cameras, a 3x3 calibration plate can be placed within the fisheye camera's field of view around the vehicle to calibrate the fisheye camera's parameters, taking into account its ability to compensate for blind spots at close range. Furthermore, to ensure the accuracy of the extrinsic calibration results at long distances while ensuring accurate extraction of the first feature point within the fisheye camera's image, multiple single-marker plates can be placed at a greater distance from the vehicle to participate in the extrinsic calibration of the fisheye camera.

[0095] For example, Figure 2 The 3x3 calibration target and the single-marker target within the fisheye camera's field of view can be used for fisheye camera calibration. The target and the single-marker target are positioned at different distances, heights, and angles relative to the vehicle, providing sufficient constraints and effectively improving the accuracy of fisheye camera calibration.

[0096] Example 2

[0097] When the sensor to be calibrated includes a short-focus camera, 2x2 calibration plates, hybrid calibration plates, and single marker plates can be used to calibrate the short-focus camera's extrinsic parameters. Considering the short-focus camera's large field of view and long-range imaging characteristics, the 2x2 calibration plates, hybrid calibration plates, and single marker plates can be placed at different distances and heights within the short-focus camera's field of view to achieve short-focus camera calibration.

[0098] For example, Figure 2 A 2x2 calibration target, a hybrid calibration target, and a single marker target within the short-focus camera's field of view can be used for short-focus camera calibration. Placing multiple calibration targets at different heights and angles within the short-focus camera's field of view, or within the common view of the short-focus camera and lidar, provides sufficient constraints and effectively improves short-focus camera calibration accuracy.

[0099] Example 3

[0100] When the sensor to be calibrated includes a telephoto camera, considering the characteristics of the telephoto camera's small field of view and ultra-long-distance imaging, the hybrid calibration plate can be placed at different distances and heights within the telephoto camera's field of view to achieve telephoto camera parameter calibration.

[0101] For example, Figure 2A hybrid calibration target within the telephoto camera's field of view can be used for telephoto camera calibration. A hybrid calibration target within the telephoto camera's common view of the LiDAR can also be used for both. Placing multiple calibration targets at different heights and angles within the telephoto camera's field of view or within the common view of the telephoto camera and LiDAR provides sufficient constraints, effectively improving telephoto camera calibration accuracy.

[0102] Example 4

[0103] When the sensors to be calibrated include short / telephoto cameras and lidars, the hybrid calibration plate can be used to calibrate the parameters of the short / telephoto cameras and lidars, and the hybrid calibration plate and multi-hole calibration plate can be used to calibrate the parameters of the lidars.

[0104] For example, Figure 2 Hybrid calibration plates located within the common view of the short / telephoto camera and LiDAR can be used for calibration of these two cameras. Placing multiple hybrid calibration plates at different heights and angles within the common view of the short / telephoto camera and LiDAR provides sufficient constraints, effectively improving calibration accuracy for both cameras and LiDAR.

[0105] For example, hybrid and multi-hole calibration plates within the LiDAR field of view can be used for LiDAR parameter calibration. Placing multiple hybrid and multi-hole calibration plates at different heights and angles within the LiDAR field of view provides sufficient constraints, effectively improving LiDAR calibration accuracy.

[0106] In some embodiments, a centering device may be provided at the calibration position of the calibration room 100. The centering device may be used to at least align the vehicle's posture. In specific applications, the centering device may be installed at the calibration position. After the vehicle stops on the centering device, the centering device is activated to align the vehicle. Each time, the vehicle is aligned to a fixed position and posture by the centering device. In one example, the plane of the centering device in contact with the vehicle tires is a horizontal plane perpendicular to gravity. This allows the xy coordinate axis plane of the vehicle coordinate system to be horizontal. The constraints of this horizontal plane can be used to implement external parameter calibration between the combined navigation and calibration.

[0107] The centering device also ensures that the relationship between the calibration coordinate system and the vehicle coordinate system meets predetermined requirements. Specifically, the vehicle coordinate system's xy plane is perpendicular to gravity, with the heading angle aligned with the vehicle's forward direction. The xy plane of the calibration coordinate system is also perpendicular to gravity, with the heading angle aligned with the center axis of the centering device. The centering device ensures that the yaw angle error after the vehicle is aligned is less than 0.03°. Therefore, after the vehicle is centered on the centering device, its forward direction is aligned with the center axis of the centering device. Thus, the coordinate axes defined by the vehicle coordinate system and the calibration coordinate system are completely consistent, with only differences in xyz translation occurring, which can be directly measured.

[0108] In some embodiments, the calibration room 100 may further include a point coordinate measurement station for accommodating a point coordinate measurement device for measuring feature points on all calibration plates within the calibration room. For example, the point coordinate measurement station may be located in the fifth area. Point coordinate measurement devices include, but are not limited to, total stations, 3D laser scanners, and 3D surveying instruments.

[0109] The point coordinate measurement device can determine the coordinates of the calibration room's coordinate system for all feature points. The point coordinate measurement device's resection function allows it to measure the reference coordinate system coordinates of all feature points in the calibration room. The reference coordinate system and the calibration room coordinate system are convertible, and the feature points of the reference coordinate system can form a common view with the feature points of any sensor to be calibrated in all directions (for example, forward, sideways, and backward). Therefore, the total station can also be used to solve the external parameters from the sensor to the calibration room.

[0110] In some embodiments, the calibration room 100 further includes a lighting system installation location for installing a lighting system, which is used to illuminate and fill in the calibration room. For example, the lighting system installation location can be located at the top of the calibration room, in the upper space of the fifth area, so that the lighting system can cover all areas and evenly illuminate the calibration plates in each area.

[0111] The lighting system should preferably use a light source with good lighting conditions. In one example, the lighting system can use an array of distributed surface light sources to illuminate and fill in the calibration room, allowing the sensor to be calibrated to robustly and accurately capture the feature point information of the calibration plate in different weather and lighting conditions.

[0112] The calibration room of the embodiment of the present disclosure can be used to place calibration plates in the forward space, lateral space, and rearward space of the vehicle, which can meet the calibration requirements of various sensors and is suitable for parameter calibration of various sensors such as short-focus cameras, long-focus cameras, fisheye cameras, lidars, inertial navigation, combined navigation, etc. It can also be used for pairwise calibration and joint calibration between sensors, and has high versatility. In addition, the calibration room of the embodiment of the present disclosure can place calibration plates at different distances, heights, and azimuth angles in the forward space, lateral space, and rearward space of the vehicle, thereby meeting the calibration requirements of various sensor parameters and the spatial placement requirements of calibration plates in various application scenarios, and can meet the requirements of the calibration of autonomous driving vehicle production lines for robust, accurate, and efficient internal and external parameter calibration.

[0113] Figure 6 This is a flow chart of the calibration method provided by the embodiment of the present disclosure, which can be executed by the calibration device described below. Figure 6 , the method may include the following steps:

[0114] Step 601: obtaining data collected by the sensor to be calibrated in a calibration room, wherein a calibration plate is arranged in the calibration room, and the calibration plate includes characteristic points;

[0115] Step 602, using the data about the calibration room collected by the sensor to be calibrated, determine the coordinates of the sensor coordinate system to be calibrated of the feature points in the calibration room;

[0116] Step 603 : Determine at least one of the following based on the sensor coordinate system coordinates of each feature point in the calibration room and its calibration room coordinate system coordinates: a first transformation relationship between the sensor coordinate system and the calibration room coordinate system, and a second transformation relationship between each sensor coordinate system.

[0117] The calibration room involved in the calibration method of the embodiment of the present disclosure may be, but is not limited to, the calibration room described above.

[0118] The calibration method of the embodiment of the present disclosure is carried out in the aforementioned calibration room that can arrange multiple types of calibration plates and supports the arrangement of calibration plates in the forward, lateral and rearward directions. It can realize the calibration of parameters of various sensors to be calibrated, and can also support the calibration of parameters between sensors.

[0119] The data in step 601 depends on the type of sensor being calibrated. If the sensor being calibrated is a camera, the data about the calibration room's internal environment in step 601 is an image of the calibration room's internal environment captured by the camera, which includes all calibration plates within the camera's field of view. If the sensor being calibrated is a lidar, the data about the calibration room's internal environment in step 601 is a point cloud of the calibration room's internal environment collected by the lidar, which includes point cloud data for all calibration plates within the lidar's field of view.

[0120] In step 602, the feature points may be, but are not limited to, first feature points or second feature points. The sensor coordinate system coordinates of the feature points to be calibrated are associated with the sensor to be calibrated. If the sensor to be calibrated is a camera, the pixel coordinate system coordinates of all first feature points within the camera's field of view are obtained by processing images of the calibration room's internal environment captured by the camera. If the sensor to be calibrated is a lidar, step 602 may extract the lidar coordinate system coordinates of all second feature points within the lidar's field of view from the point cloud data captured by the lidar.

[0121] In step 602, if the camera intrinsic parameters are known, the camera coordinate system coordinates of the first feature point and / or the second feature point can also be calculated based on the camera projection model through the pixel coordinate system coordinates of the first feature point and / or the second feature point.

[0122] Previous article Figure 2 For example, the sensor coordinate system coordinates to be calibrated of the first feature point and / or the second feature point in the calibration room in step 602 may include:

[0123] 1) The 2D coordinates of the 4x9 marked corner points with IDs on each 3x3 calibration plate in the pixel coordinate system and the 3D coordinates in the camera coordinate system;

[0124] 2) The 2D coordinates of the 4x4 marked corner points with IDs on each 2x2 calibration plate in the pixel coordinate system and the 3D coordinates in the camera coordinate system;

[0125] 3) The two-dimensional coordinates of the mx4 marked corner points with ID identification of each hybrid calibration plate in the pixel coordinate system and the three-dimensional coordinates in the camera coordinate system, and the three-dimensional coordinates of the center points of the m calibration holes with ID identification in the camera coordinate system and the three-dimensional coordinates in the lidar coordinate system;

[0126] 4) The 2D coordinates of the 1x4 marker corner points with ID identification of each single marker plate in the pixel coordinate system and its 3D coordinates in the camera coordinate system;

[0127] 5) The three-dimensional coordinates of the center points of the n calibration holes with ID marks on each multi-hole calibration plate in the lidar coordinate system.

[0128] 6) The calibration plates within the common viewing area of ​​different sensors will also be reused.

[0129] Prior to step 603, the calibration method of the disclosed embodiment may further include determining the coordinates of a characteristic point in the calibration room in a calibration coordinate system, and associating and storing the coordinates of the characteristic point in the calibration coordinate system with its corresponding plate identifier and / or marking identifier. In this manner, the parameters of the sensor to be calibrated can be calibrated using the coordinates of the calibration coordinate system.

[0130] In one example, a calibration room coordinate system may be constructed, and the coordinates of the calibration room coordinate system of the feature points in the calibration room may be determined according to the attribute information of each calibration plate in the calibration room.

[0131] In one example, a point coordinate measurement device can be used to measure the inter-calibration coordinate system coordinates of feature points on each calibration plate, and the inter-calibration coordinate system coordinates of the feature points on each calibration plate can be associated with the plate identifier of the calibration plate and stored. It is understood that the method for obtaining the inter-calibration coordinate system coordinates of the feature points on each calibration plate is not limited to this method, and any other applicable method can be applied in the present disclosure.

[0132] Specifically, a point coordinate measurement device is used to perform point measurement on all feature points in the calibration room to obtain the three-dimensional coordinates of these feature points in the reference coordinate system. The reference coordinate system coordinates of the feature points in the calibration room measured by the total station are obtained. Based on the known conversion matrix from the reference coordinate system to the calibration room coordinate system, the reference coordinate system coordinates of all feature points in the calibration room are converted to the calibration room coordinate system to obtain the calibration room coordinates of the feature points in the calibration room. The xy coordinate axis plane of the reference coordinate system can be a horizontal plane. Using this plane constraint, it is easy to associate the reference coordinate system with the calibration room coordinate system, thereby obtaining the conversion matrix from the reference coordinate system to the calibration room coordinate system.

[0133] In addition to constructing the calibration room coordinate system, a calibration plate coordinate system and / or a marker coordinate system can also be constructed. According to the attribute information of each calibration plate in the calibration room, the calibration plate coordinate system can be constructed and the calibration plate coordinate system coordinates of each feature point on the calibration plate can be determined. The calibration plate coordinate system coordinates of each feature point on the calibration plate can be combined with the pixel coordinate system coordinates obtained in step 602 to determine the camera coordinate system coordinates of the feature point. In addition, the calibration plate coordinate system and the marker coordinate system can also be applied to the intrinsic parameter calibration of the camera, the extrinsic parameter calibration between the camera and the calibration room, and the extrinsic parameter calibration between the camera and the lidar. The marker coordinate system and the calibration plate coordinate system can be flexibly constructed according to actual needs, and their specific construction methods are not limited by the embodiments of the present disclosure.

[0134] In one implementation, the coordinates of the feature points on each calibration plate in the calibration room (the coordinates include but are not limited to one or more of the calibration room coordinate system coordinates, the calibration plate coordinate system coordinates, the marker coordinate system coordinates, the pixel coordinate system coordinates, the camera coordinate system coordinates, the lidar coordinate system coordinates, and the vehicle coordinate system coordinates) can be stored in a pre-agreed fixed order so as to directly read one or more coordinates of the required feature point or points. At the same time, the point correspondence of the feature points in different coordinate systems can be directly determined through sequential storage, so that it can be directly applied to the construction process of problems such as ICP and PnP, without having to find the point correspondence of different coordinate systems, effectively reducing the amount of calculation and reducing the calculation complexity, thereby improving processing efficiency and saving computing resources.

[0135] In one example, the coordinates of each first feature point in the marker (e.g., one or more of the coordinates of the calibration space coordinate system, the coordinates of the calibration plate coordinate system, the coordinates of the marker coordinate system, the coordinates of the pixel coordinate system, the coordinates of the camera coordinate system, and the coordinates of the lidar coordinate system) can be associated and stored with the corresponding marker identifier and plate identifier in order according to the position order of each marker in the calibration plate. For each marker, the coordinates of each first feature point on the marker can be associated and stored with the corresponding marker identifier and plate identifier according to the position order of the first feature point on the marker.

[0136] In one example, the coordinates of the second reference points of the calibration holes in the calibration plate (e.g., one or more of the coordinates of the calibration space coordinate system, the coordinates of the calibration plate coordinate system, the coordinates of the camera coordinate system, and the coordinates of the lidar coordinate system) can be stored in association with the corresponding plate identifiers according to the positional order of the calibration holes in the calibration plate. Specifically, the coordinates of the second reference points of the calibration holes in the calibration plate are stored in association with the corresponding plate identifiers, and the coordinates of the first reference points of the calibration plate on the calibration plate contour are stored in association with the corresponding plate identifiers according to the positional order of the first reference points on the calibration plate contour.

[0137] by Figure 3 For example, the four corner points of the mark are used as the first feature points, and the four corner points of the calibration plate 300 and the centers of the circular calibration holes on the calibration plate 300 are used as structural feature points. The coordinates of the first feature points and the structural feature points can be associated with the plate identifier, mark identifier, etc. in a clockwise order and stored.

[0138] For each mark on the calibration plate, the coordinates of the corner points of each mark can be stored in the clockwise order of "left, top, right, bottom". At the same time, for each mark, the coordinates of the four corner points on the mark are associated with the corresponding plate identifier and mark identifier in the clockwise order of "top left corner, top right corner, bottom right corner, bottom left corner".

[0139] For the four corner points on the calibration plate, the coordinates of the four corner points on the calibration plate can also be associated and stored with the corresponding plate identifiers in the clockwise order of "upper left corner, upper right corner, lower right corner, lower left corner".

[0140] For each calibration hole on the calibration plate, the coordinates of the center of each calibration hole can also be associated and stored with the corresponding plate identification in the clockwise order of "left, top, right, bottom".

[0141] In this implementation, there is no need to perform the point correspondence determination process, and the ICP problem or PnP problem can be directly constructed, which reduces the computational complexity, reduces the amount of operations, saves computing resources, and improves processing efficiency.

[0142] In step 602, the coordinates of the sensor coordinate system to be calibrated of the feature point can be stored in association with the plate identification, mark identification, etc. of the corresponding feature point, so that in step 803, the ICP problem or PnP problem can be directly constructed through the plate identification, mark identification and storage order of the feature point, without having to find the correspondence between the coordinates of the sensor coordinate system to be calibrated and the coordinates of the calibration coordinate system. This can greatly reduce the amount of calculation and reduce the calculation complexity, thereby improving the calibration efficiency and calibration accuracy.

[0143] In step 603, the two-dimensional coordinates and / or three-dimensional coordinates of the feature points with ID identification within the field of view extracted by the above-mentioned different sensors can be associated with the coordinates of the feature points with corresponding ID in the coordinate system between calibrations to construct ICP and PnP problems to solve the external parameters of the camera / lidar to the calibration room.

[0144] For example, the sensor to be calibrated includes a camera. If the extrinsic parameters between the camera and the calibration room need to be calibrated, the camera coordinate system coordinates of all feature points in the camera's field of view and the calibration room coordinate system coordinates can be used to construct an ICP problem to obtain the initial value of the extrinsic parameter. On the basis of the initial value, the pixel coordinate system coordinates of the first feature point in the camera's field of view and the calibration room coordinate system coordinates are used to construct a PnP problem and solve it, thereby obtaining the extrinsic parameters from the camera to the calibration room. The extrinsic parameters include the transformation matrix from the camera coordinate system to the calibration room coordinate system, and the transformation matrix includes the translation vector and rotation matrix from the camera coordinate system to the calibration room coordinate system.

[0145] For example, when the sensor to be calibrated is a lidar, if the external parameters between the lidar and the calibration room need to be calibrated, the coordinates of the lidar coordinate system of the second reference point in the lidar field of view and the coordinates of the calibration room coordinate system can be used to construct an ICP problem and solve it to obtain the external parameters from the lidar to the calibration room. The external parameters include the transformation matrix from the lidar coordinate system to the calibration room coordinate system, and the transformation matrix includes the translation vector and rotation matrix from the lidar coordinate system to the calibration room coordinate system.

[0146] For example, the sensors to be calibrated include lidar and cameras. If the external parameters of the camera and lidar need to be calibrated, the ICP problem can be constructed and solved using the lidar coordinate system coordinates and camera coordinate system coordinates of the second reference point on each calibration plate in the common view area of ​​the camera and lidar to obtain the external parameters from the camera to the lidar. The external parameters include the transformation matrix from the camera coordinate system to the lidar coordinate system, and the transformation matrix includes the translation vector and rotation matrix from the camera coordinate system to the lidar coordinate system.

[0147] For example, if the extrinsic parameters of the camera and lidar need to be calibrated, the extrinsic parameters between the camera and lidar can be obtained based on the extrinsic parameters from the camera coordinate system to the calibration coordinate system and the extrinsic parameters from the lidar coordinate system to the calibration coordinate system, with the calibration coordinate system as the transfer center.

[0148] In some embodiments, if the sensor to be calibrated includes a navigation device (such as inertial navigation or combined navigation), the above-mentioned calibration method may further include: determining the external parameters between the navigation device and the calibration, that is, determining the transformation relationship between the navigation coordinate system of the navigation device and the coordinate system between the calibration (including but not limited to the rotation matrix and the translation vector). Specifically, determining the external parameters between the inertial navigation / combined navigation and the calibration may include: first configuring the position and angular offset of the navigation device so that the coordinate system of the inertial navigation / combined navigation is consistent with the vehicle coordinate system, and then measuring the transformation matrix between the calibration coordinate system and the vehicle coordinate system. The transformation matrix between the calibration coordinate system and the vehicle coordinate system is the external parameter between the navigation device and the calibration. Among them, the translation vector in the transformation matrix between the calibration coordinate system and the vehicle coordinate system can be obtained by measurement, and the rotation matrix can be set to the unit matrix.

[0149] Due to installation errors and vehicle inequalities, the three-axis directions and origin positions of the navigation device's navigation coordinate system and the vehicle coordinate system may not be perfectly aligned. The xy plane of the navigation coordinate system may also not be perfectly flat, meaning it is not completely perpendicular to gravity. First, through navigation device self-calibration, the yaw angle output by the navigation device is aligned with the vehicle's heading, achieving yaw offset consistency between the navigation coordinate system and the vehicle coordinate system. Second, the roll / pitch offset is calculated using gravity decomposition and the navigation is configured so that the roll and pitch angles are output as zero on the centering device plane. This aligns the xy plane of the navigation coordinate system with gravity, aligning the xy plane with the vehicle coordinate system and achieving roll / pitch offset consistency between the navigation coordinate system and the vehicle coordinate system. Finally, the position offset of the navigation device is measured and configured to align its origin with the origin of the vehicle coordinate system, thus achieving consistent x / y / z offsets between the navigation coordinate system and the vehicle coordinate system.

[0150] The extrinsic parameters between any two sensors to be calibrated include the coordinate transformation relationship between the two sensors, such as a transformation matrix. Specifically, based on the extrinsic parameters from any sensor coordinate system to the inter-calibration coordinate system, the inter-calibration coordinate system serves as the transfer center to obtain the extrinsic parameters between any two sensors. These sensors can include cameras, lidars, inertial navigation systems, integrated navigation systems, and so on. This approach is particularly useful for calibrating parameters between cameras and lidars, cameras and cameras, or lidars and lidars without a common view area.

[0151] In some embodiments, the sensor extrinsic parameters obtained in step 603 may be verified using the coordinates of the calibration room coordinate system and a pre-set calibration verification threshold. Specifically, the sensor coordinate system coordinates of the feature points in the calibration room obtained in step 802 are the true coordinate values ​​of each feature point in the sensor coordinate system. The above calibration method may include the following steps for extrinsic parameter verification:

[0152] Step a1, determining the estimated coordinate value of the feature point in the sensor coordinate system based on the coordinates of the calibration coordinate system of the feature point and the external parameters of the sensor to be calibrated;

[0153] Step a2: verifying the second transformation relationship of the sensor to be calibrated based on the estimated coordinate value of the feature point in the sensor coordinate system, the true coordinate value, and a preset calibration verification threshold of the sensor to be calibrated.

[0154] Therefore, the calibration result obtained in step 603 can be verified and quantified in the calibration room to determine whether the calibration result meets expectations, thereby achieving the purpose of calibration verification.

[0155] In some embodiments, step a2 may include calculating the Euclidean distance between the estimated coordinate value and the true coordinate value of each feature point in the sensor coordinate system, summing and averaging the Euclidean distances of all feature points to obtain an average Euclidean distance value, and comparing the average Euclidean distance value with the calibration verification threshold of the sensor to be calibrated to determine whether the current extrinsic parameters of the sensor to be calibrated meet expectations. Specifically, if the average Euclidean distance value is greater than the calibration verification threshold of the sensor to be calibrated, the verification fails, and it can be determined that the current extrinsic parameters of the sensor to be calibrated do not meet expectations. If the average Euclidean distance value is less than or equal to the calibration verification threshold of the sensor to be calibrated, the verification passes, and it can be determined that the current extrinsic parameters of the sensor to be calibrated meet expectations.

[0156] In one example, if camera extrinsic parameters need to be verified, the implementation process may include: obtaining the pixel coordinate system coordinates of the first feature point in step 602, where the pixel coordinate system coordinates are the true pixel coordinate system coordinate values; determining an estimated pixel coordinate system coordinate value of the first feature point based on the calibration inter-coordinate system coordinates of the first feature point and the camera's extrinsic parameters; and verifying the camera's extrinsic parameters based on the estimated pixel coordinate system coordinate value of the first feature point, the true pixel coordinate system coordinate value of the first feature point, and a pre-set camera calibration verification threshold. This calibration result verification method is applicable to various cameras, including but not limited to fisheye cameras, short-focus cameras, and long-focus cameras.

[0157] In some embodiments, determining the pixel coordinate system coordinate estimate of the first feature point based on the inter-calibration coordinate system coordinates of the first feature point and the camera's extrinsic parameters may include: calculating the camera coordinate system coordinate estimate of the first feature point based on the camera's extrinsic parameters and the inter-calibration coordinate system coordinates of the first feature point, and projecting the camera coordinate system coordinate estimate of the first feature point onto the pixel coordinate system using a projection model of the current camera to obtain the pixel coordinate system coordinate estimate of the first feature point. For example, if the current camera is a short / telephoto camera, its camera projection model may be determined to be a pinhole model; if the current camera is a fisheye camera, its camera projection model may be determined to be a Scaramuzza model.

[0158] In an example, if the external parameters of the laser radar need to be verified, the implementation process may include: obtaining the laser radar coordinate system coordinates of the second reference point through step 602, the laser radar coordinate system coordinates are the true values ​​of the coordinates in the laser radar coordinate system, determining the coordinate estimation value of the second reference point in the laser radar coordinate system according to the calibration coordinate system coordinates of the second reference point and the external parameters of the laser radar, and verifying the external parameters of the laser radar according to the true value of the coordinates of the second reference point in the laser radar coordinate system, the coordinate estimation value and the pre-set calibration verification threshold of the laser radar.

[0159] The foregoing description of this specification describes specific embodiments. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in an order different from that described in the embodiments and still achieve the desired results. Furthermore, the processes depicted in the accompanying drawings do not necessarily require the specific order shown or the sequential order to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0160] The calibration method of the embodiment of the present disclosure has the following beneficial effects: 1. It can meet the requirements of robust, accurate and efficient internal and external parameter calibration for the calibration of autonomous driving vehicle production lines. 2. It is suitable for parameter calibration of various sensors such as short-focus cameras, long-focus cameras, fisheye cameras, lidars, inertial navigation, combined navigation, etc., and can also perform pairwise calibration and joint calibration between sensors, with high versatility. 3. It can meet the requirements of high efficiency, accuracy and robustness for feature point extraction and calibration required for sensor parameter calibration in vehicle production line calibration. After actual testing, it has been found that compared with other existing calibration methods, the calibration method of the embodiment of the present disclosure has more advantages in versatility, accuracy, robustness, calibration efficiency and other aspects.

[0161] Figure 7FIG. 7 is a schematic diagram of a calibration method 700 according to another embodiment of the present disclosure. As shown in the figure, the calibration method 700 includes steps S710 to S760.

[0162] In step 710, point cloud data collected by a point cloud collection device is acquired.

[0163] In step 720, target point cloud clusters that meet the size constraints of the calibration plate are identified from the point cloud data, wherein the size constraints of the calibration plate include but are not limited to the length constraint, width constraint, and aspect ratio constraint of the calibration plate.

[0164] In step 730 , a plurality of target hole positions are determined from the target point cloud cluster according to the attribute information of the calibration plate.

[0165] In step 740 , it is determined whether there is a hole at each target hole position based on the number of point cloud points in each target hole position.

[0166] In step 750 , a binary code of each target hole position is determined according to whether there is a hole at each target hole position.

[0167] In step 760 , the plate code corresponding to the target point cloud cluster is determined according to the binary code of each target hole position.

[0168] In one example, step 720 includes extracting edge feature points from the point cloud data and clustering the edge feature points to obtain at least one candidate point cloud cluster. Determining a minimum bounding box for each candidate point cloud cluster, that is, determining a minimum circumscribed geometric body, such as a minimum circumscribed sphere or minimum circumscribed cuboid, for each candidate point cloud cluster. Selecting a target point cloud cluster from the at least one candidate point cloud cluster whose minimum bounding box meets the size constraints of the calibration plate.

[0169] Here, the distance between each point cloud point in the multiple point cloud points and other point cloud points is calculated based on the point cloud data. If the distance is greater than a predetermined threshold, the point cloud point is an edge feature point. Afterwards, the coordinate values ​​of the point cloud points and the laser beam identification (i.e., ring information, line identification) are used to obtain the clustered point cloud cluster, or the angle value of the laser beam corresponding to each point cloud point is calculated based on the coordinate values ​​of each point cloud point, that is, the angle of the laser beam relative to the horizontal plane. Points with consistent laser beam identifications or similar angle values ​​are point cloud points corresponding to the same laser beam. Among them, the size constraints include but are not limited to the length of the minimum bounding box being greater than or equal to the length of the calibration plate, the width of the minimum bounding box being greater than or equal to the width of the calibration plate, and the aspect ratio of the minimum bounding box being less than or equal to the aspect ratio of the calibration plate. In addition, after determining the minimum bounding box, all point clouds within the minimum bounding box can be restored based on the size of the minimum bounding box and the original point cloud data of the frame to obtain the target point cloud cluster.

[0170] In one example, step 730 includes: determining at least one target vertex from the target point cloud cluster, and determining multiple target hole positions based on the attribute information of the calibration plate and the at least one target vertex. The target hole position can be considered as the maximum probability position of the first hole position of the calibration plate in the point cloud frame.

[0171] Here, the calibration device can also traverse the point cloud points in the target point cloud cluster, taking each traversed point cloud point as a candidate vertex, and determining a candidate area of ​​the same size as the calibration plate based on the candidate vertex. Based on the number of point cloud points within the candidate area corresponding to each candidate vertex, at least one target vertex is determined from the traversed candidate vertices. Optionally, the candidate area with the largest number of point cloud points can be determined as the calibration plate area, and the candidate vertex corresponding to the calibration plate area can be used as the target vertex.

[0172] Furthermore, the calibration device can also determine the reference point corresponding to the second hole position in the point cloud data based on the second positional relationship between the second hole position and the plate vertex in the calibration plate, as well as the target vertex. Given the calibration hole size of the calibration plate, the reference hole position corresponding to the second hole position in the point cloud data can be determined. Given the positional relationship between the first hole position and the second hole position, the target hole position corresponding to the first hole position in the point cloud data can be determined.

[0173] Furthermore, the calibration device can also determine an initial point based on the second positional relationship between the second hole position and the plate vertex, as well as the target vertex. This initial point can be considered the initial screening position for the reference point. Then, a first candidate region is determined based on the size information of the second hole position, with this initial point as the center. The candidate points within the first candidate region are traversed, and a second candidate region is determined based on the size information of the second hole position, with each traversed candidate point as the center. This yields second candidate regions corresponding to multiple candidate points. The reference point is determined from the traversed candidate points based on the number of point cloud points within each second candidate region. For example, the candidate point corresponding to the second candidate region containing the fewest point cloud points is used as the reference point.

[0174] In one implementation, the first candidate region is the minimum circumscribed polygon of the second hole position, and the second candidate region is the maximum inscribed polygon of the second hole position. Furthermore, the second calibration hole is a circular hole, and both the first candidate region and the second candidate region are square regions. The side length of the first candidate region is the side length of the minimum circumscribed square of the second calibration hole, and the side length of the second candidate region is the side length of the maximum inscribed square of the second calibration hole.

[0175] In addition, after identifying the target hole positions and reference hole positions from the point cloud data, the distance values ​​between multiple different hole positions can be calculated based on the attribute information of each hole position in the pre-stored calibration plate (such as position information, distance information, size information, etc.), such as the distance values ​​of different target hole positions, the distance values ​​of different reference hole positions, and the distance values ​​of different target hole positions and reference hole positions, and compared with the true value to determine whether the target point cloud cluster is an invalid point cloud cluster. For example, if the average value of the multiple distances differs from the true value by more than or equal to a predetermined threshold, the target point cloud cluster is determined to be an invalid point cloud cluster and is discarded.

[0176] In one example, step 740 includes: determining a maximum inscribed polygon corresponding to each target hole position, the maximum inscribed polygon including but not limited to a maximum inscribed square; and determining that a hole exists at the target hole position in response to the number of point cloud points within the maximum inscribed polygon being less than or equal to a predetermined threshold. Otherwise, determining that no hole exists at the target hole position.

[0177] Through the above steps, we can obtain the valid point cloud cluster in each frame of point cloud data, the calibration plate area corresponding to the calibration plate in the point cloud frame, the plate code corresponding to the calibration plate area, the reference hole position corresponding to the first hole position of the calibration plate in the point cloud frame, the reference point corresponding to the feature point of the first hole position in the point cloud frame, the target hole position corresponding to the second hole position in the point cloud frame, and the target point corresponding to the feature point of the second hole position in the point cloud frame.

[0178] The LiDAR coordinates of these feature points are extracted from each frame of point cloud data. Based on the pre-stored calibration plate coordinates of these points, the conversion relationship between the calibration plate coordinate system and the LiDAR coordinate system is calculated. Furthermore, the pixel coordinates of these feature points are extracted from the image data captured by the image acquisition device, and the conversion relationship between the calibration plate coordinate system and the camera coordinate system is determined based on the camera intrinsic parameter model. The conversion relationship between the camera coordinate system and the LiDAR coordinate system can be determined by transferring the calibration plate coordinate system.

[0179] The calibration device provided in the embodiment of the present disclosure may be implemented in the form of an electronic device. In this case, Figure 8 As shown in , the calibration device includes one or more processors 801 and a memory 802 for storing one or more programs, which are executed by the one or more processors 801 to implement the method flow in the above embodiments of the present disclosure and / or the program units corresponding to each unit in the device.

[0180] The various components are interconnected using different buses and can be installed on a common motherboard or in other ways as needed. The processor 801 can process instructions executed in the electronic device, including instructions stored in or on the memory to display graphical information of the user interface on an external input / output device (such as a display device coupled to the interface). In other embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. The processor 801 may include one or more single-core processors or multi-core processors. The processor 801 may include any combination of general-purpose processors or special-purpose processors (such as image processors, application processors, baseband processors, etc.). The memory 802 is a computer-readable storage medium provided by the present disclosure, which can be used to store non-transient software programs, non-transient computer executable programs and units, such as program instructions / units corresponding to the calibration method in the embodiment of the present disclosure. The processor 801 executes the above-mentioned method embodiments by running the non-transient software programs, instructions and units stored in the memory 802. Figure 6 and Figure 7 The procedures, instructions and units corresponding to the calibration method shown.

[0181] The calibration device may further include: an input device 803 and an output device 804. The processor 801, the memory 802, the input device 803 and the output device 804 may be connected via a bus or other means. Figure 8 The bus connection is taken as an example.

[0182] The input device 803 can receive input digital or character information, and generate signal input related to the user settings and function control of the calibration device, such as a touch screen, a keypad, a mouse, a trackpad, a touchpad, an indicator stick, one or more mouse buttons, a trackball, a joystick and other input devices. The output device 1004 may include a display device, an auxiliary lighting device (e.g., an LED) and a tactile feedback device (e.g., a vibration motor), etc. The display device may include, but is not limited to, a liquid crystal display (LCD), a light emitting diode (LED) display and a plasma display. In some embodiments, the display device may be a touch screen. The above-mentioned program (also referred to as software, software application, or code) includes machine instructions for a programmable processor, and these computer programs can be implemented using an object-oriented programming language, assembly or machine language.

[0183] As time goes by and technology develops, the meaning of medium becomes more and more extensive, and the propagation path of computer programs is no longer limited to tangible media, but can also be downloaded directly from the Internet, etc. Any combination of one or more computer-readable storage media can be used. Computer-readable storage media can adopt but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or components, or any combination of the above. More specific examples (non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memories (RAM), read-only memories (ROM), erasable programmable read-only memories (EPROM or flash memory), optical fibers, portable compact disk read-only memories (CD-ROMs), optical storage devices, magnetic storage devices, or any suitable combination of the above. In this document, computer-readable storage media can be any tangible medium that contains or stores a program that can be used by or in combination with an instruction execution system, device or device.

[0184] An embodiment of the present disclosure further provides a computer-readable storage medium having a computer program stored thereon, the program including instructions, which, when executed by one or more processors of a computing device, execute the steps of any one of the method embodiments described above.

[0185] The present disclosure also provides a calibration system comprising: a sensor to be calibrated mounted on a vehicle, a calibration room, a calibration plate located within the room, and the aforementioned calibration device. The calibration device is capable of communicating with the sensor to be calibrated, the vehicle is located at a calibration position within the calibration room, and the calibration plate is within the field of view of the sensor to be calibrated.

[0186] Figure 9 FIG. 9 shows a partial structure of a calibration system 900 according to an embodiment of the present disclosure. Figure 9 As shown, the sensors to be calibrated may include but are not limited to one or more of the following: camera, lidar, millimeter wave radar, inertial navigation, combined navigation, etc.

[0187] In some embodiments, the calibration system 900 may further include a centering device (not shown), located at the calibration position in the calibration room, for aligning the vehicle's posture. In some embodiments, the calibration system may further include a point coordinate measurement device, located at the point coordinate measurement position in the calibration room, for measuring the characteristic points of all calibration plates in the calibration room. In some embodiments, the calibration system may further include an illumination system, located at the lighting system layout position in the calibration room, for illuminating and supplementing the internal environment of the calibration room.

[0188] A1. A calibration room, wherein the calibration room comprises a first area, a second area, a third area, a fourth area and a fifth area; a calibration position is provided in the fifth area, the calibration position is used to park vehicles, and the vehicles are used to load sensors to be calibrated; the first area, the second area, the third area and the fourth area are located around the calibration position, and calibration plates are arranged in the first area, the second area, the third area and / or the fourth area, and the calibration plates are used to calibrate the parameters of the sensors to be calibrated; the calibration plates are located within the field of view of the sensors to be calibrated and are provided with marks and / or calibration holes. A7. A calibration room as described in A1, wherein a centering device is provided at the calibration position in the calibration room, and the centering device is used to align the vehicle posture. A8. A calibration room as described in A1, wherein each calibration plate has a unique plate identifier and each mark has a unique mark identifier. A9. A calibration room as described in A1, wherein the mark is a QR code. A10. A calibration room as described in A1, wherein the calibration plate includes characteristic points, and the characteristic points include first characteristic points about the mark and second characteristic points about the structural contour of the calibration plate, and the second characteristic points include at least one of the following: a first reference point about the outer contour of the calibration plate, and a second reference point about the calibration hole in the calibration plate. A11. A calibration room as described in A10, wherein the first characteristic points include corner points of the mark; and / or the first reference points are corner points of the outer contour of the calibration plate; and / or the second reference points are center points of the calibration holes. A12. A calibration room as described in A1, wherein the fifth area further includes: a point coordinate measurement position for placing a point coordinate measurement device, and the point coordinate measurement device is used to perform point measurement on characteristic points of all calibration plates in the calibration room.

[0189] B13. A calibration method, comprising: obtaining data about a calibration room collected by a sensor to be calibrated, wherein a calibration plate is arranged in the calibration room, and the calibration plate includes feature points; using the data about the calibration room collected by the sensor to determine the sensor coordinate system coordinates of each feature point in the calibration room; based on the sensor coordinate system coordinates of each feature point in the calibration room and its calibration room coordinate system coordinates, determining at least one of the following: a first transformation relationship between the sensor coordinate system and the calibration room coordinate system, and a second transformation relationship between each sensor coordinate system. B14. The calibration method according to B13, wherein the calibration room is a calibration room as described in any one of A1-A12. B15. The calibration method according to B13, further comprising: determining the calibration room coordinate system coordinates of the feature points of each calibration plate in the calibration room; and associating and saving the calibration room coordinate system coordinates of the feature points with their corresponding plate identifiers and / or marking identifiers. B16. The calibration method according to B15, wherein determining the inter-calibration coordinate system coordinates of the feature points in the calibration room comprises: obtaining the reference coordinate system coordinates of the feature points of each calibration plate in the calibration room measured by a point coordinate measuring device; and converting the reference coordinate system coordinates of each feature point in the calibration room to the inter-calibration coordinate system according to a known conversion matrix from the reference coordinate system to the inter-calibration coordinate system to obtain the inter-calibration coordinate system coordinates of each feature point in the calibration room. B17. The calibration method according to B13, wherein the sensor coordinate system coordinates of each feature point in the calibration room are the true coordinate values ​​of each feature point in the sensor coordinate system; the method further comprises: determining the coordinate estimate of the feature point in the sensor coordinate system based on the inter-calibration coordinate system coordinates of the feature point and the first transformation relationship; and verifying the second transformation relationship based on the coordinate estimate of the feature point in the sensor coordinate system, the true coordinate value, and a pre-set calibration verification threshold.

[0190] C18. A calibration device, comprising: one or more processors, and a memory storing a program, wherein the program includes instructions, and when the instructions are executed by the processor, the processor executes the calibration method according to any one of claims 13 to 17. C19. A calibration system, comprising: a calibration room as described in any one of A1 to A12; a sensor to be calibrated mounted on a vehicle, the vehicle being located at a calibration position in the calibration room; and a calibration device as described in C18, the calibration device being capable of communicating with the sensor to be calibrated. C20. The calibration system as described in C19 further comprises at least one of the following: a centering device, disposed at a calibration position in the calibration room, for aligning the vehicle's posture; a point coordinate measuring device, for performing point measurement on the characteristic points of each calibration plate in the calibration room; and a lighting system, for illuminating and supplementing the calibration room.

[0191] The technical solutions provided by the present disclosure are introduced in detail above. Specific examples are used herein to illustrate the principles and implementation methods of the present disclosure. The description of the above embodiments is only used to help understand the method and core ideas of the present disclosure. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scopes based on the ideas of the present disclosure. In summary, the content of this specification should not be understood as a limitation on the present disclosure. The above description is only a preferred embodiment of the present disclosure and is not intended to limit the present disclosure. Any modifications, equivalent substitutions, etc. made within the spirit and principles of the present disclosure should be included in the scope of protection of the present disclosure.

Claims

1. A calibration room, wherein: The calibration room includes a first area, a second area, a third area, a fourth area and a fifth area; A calibration position is provided in the fifth area, and the calibration position is used to park a vehicle, and the vehicle is used to load a sensor to be calibrated; The first area, the second area, the third area and the fourth area are located around the calibration position, and the first area, the second area, the third area and / or the fourth area are provided with calibration plates, and the calibration plates are used to calibrate the parameters of the sensor to be calibrated; The calibration plate is located within the field of view of the sensor to be calibrated and is provided with marks and / or calibration holes.

2. The calibration room according to claim 1, wherein: The calibration plates in the first area, the second area, the third area and / or the fourth area have different heights, and different distances and / or different orientation angles relative to the vehicle.

3. The calibration room according to claim 1, wherein: The first area is located in a first direction of the calibrated position, the second area is located in a second direction of the calibrated position, the third area is located in a third direction of the calibrated position, and the fourth area is located in a fourth direction of the calibrated position. The first direction is opposite to the third direction, and the second direction is opposite to the fourth direction.

4. The calibration room according to claim 1, wherein: The sensor to be calibrated includes one or more of a camera, a laser radar, an inertial navigation, and a combined navigation. The camera includes one or more of a fisheye camera, a short-focus camera, and a long-focus camera.

5. The calibration room according to any one of claims 1 to 4, wherein: The type of the calibration plate includes one or more of the following: a first calibration plate having a first number of markings; a second calibration plate having a second number of markings; a third calibration plate, provided with a third number of marks and a third number of calibration holes; a fourth calibration plate having a fourth number of marks; The fifth calibration plate is provided with a fifth number of calibration holes.

6. The calibration room according to claim 5, wherein: The sizes of the first calibration plate, the second calibration plate, the third calibration plate, the fourth calibration plate and the fifth calibration plate are different.

7. A calibration method comprising: Acquire data collected by the sensor to be calibrated about a calibration room, wherein the calibration room is provided with a calibration plate, and the calibration plate includes characteristic points; Determining the sensor coordinate system coordinates of each feature point in the calibration room using the data collected by the sensor about the calibration room; At least one of the following is determined based on the sensor coordinate system coordinates of each feature point in the calibration room and its calibration room coordinate system coordinates: a first transformation relationship between the sensor coordinate system and the calibration room coordinate system, and a second transformation relationship between each sensor coordinate system.

8. A calibration device, wherein: The device comprises: one or more processors, and A memory storing a program, wherein the program includes instructions, and when the instructions are executed by the processor, the processor executes the calibration method according to any one of claims 13 to 17.

9. A calibration system, wherein: include: The calibration room according to any one of claims 1 to 6; a sensor to be calibrated mounted on a vehicle, the vehicle being located at a calibration position in the calibration room; and The calibration device according to claim 8, wherein the calibration device is capable of communicating with the sensor to be calibrated. 10 . A computer-readable storage medium storing a program, the program comprising instructions, which, when executed by one or more processors of a computing device, cause the computing device to perform the calibration method according to claim 7 .

Images