Calibration board determination method, calibration board system, computing device and storage medium

Abstract

The invention provides a calibration board determination method, a calibration board system, computing equipment and a storage medium. The calibration board determination method comprises the following steps: acquiring point cloud data acquired by a point cloud acquisition device; identifying a target point cloud cluster conforming to the size constraint of a calibration plate from the point cloud data, wherein the calibration plate is provided with at least one calibration hole; determining a plurality of target hole sites from the target point cloud cluster according to attribute information of a calibration board; according to the number of the point cloud points in each target hole site, determining whether each target hole site has a hole; determining a binary code of each target hole site according to whether there is a hole at each target hole site; and determining a plate code corresponding to the target point cloud cluster according to the binary code of each target hole site.

Description

Technical Field

[0001] The present disclosure relates to the field of computer vision technology, and in particular to a calibration plate determination method, a calibration plate system, a computing device, and a storage medium. Background Art

[0002] Sensor calibration in the autonomous driving field involves calibrating the internal and external parameters of sensors such as cameras, Lidar (Light Detection and Ranging), millimeter-wave radar, and inertial / integrated navigation. This ensures unified coordinates for each sensor and provides reliable input for perception algorithms. Target-based calibration methods are among the calibration methods available for these sensors. In these methods, effective feature extraction and correlation of common features across the sensors to be calibrated are fundamental to optimizing the internal and external parameters.

[0003] The most common fiducial marking method for cameras and images is QR code marking. This plays a crucial role in tasks such as camera (multi-sensor) calibration, object tracking in cluttered or textureless environments, and vision-based simultaneous localization and mapping. LiDAR point cloud data is unstructured, significantly different from the structured data of camera images. Therefore, common fiducial marking methods and systems for cameras and images are not applicable to LiDAR point clouds. Summary of the Invention

[0004] In view of this, the present disclosure provides a calibration plate determination method, a calibration plate system, a computing device, and a storage medium to improve the robustness, accuracy, and versatility of sensor calibration.

[0005] According to a first aspect of the present disclosure, a calibration plate determination method is provided, comprising: acquiring point cloud data collected by a point cloud acquisition device; identifying a target point cloud cluster that meets the size constraints of a calibration plate from the point cloud data, the calibration plate having at least one hole; determining a plurality of target hole positions from the target point cloud cluster based on attribute information of the calibration plate; determining whether each target hole position has a hole based on the number of point cloud points in each target hole position; determining a binary code for each target hole position based on whether each target hole position has a hole; and determining a plate code corresponding to the target point cloud cluster based on the binary code for each target hole position.

[0006] According to a second aspect of the present disclosure, a calibration plate determination method is provided, comprising: obtaining the number of hole positions and the hole position distribution of the calibration plate, wherein each hole position has a corresponding binary code according to whether there is a hole at the hole position, and each calibration plate has a corresponding plate code according to the binary codes of the multiple hole positions; determining multiple plate codes according to the number of hole positions and the hole position distribution, wherein the Hamming distance between any two plate codes in the multiple plate codes is greater than or equal to 1; and generating a calibration plate pattern corresponding to each plate code in the multiple plate codes.

[0007] According to a third aspect of the present disclosure, a calibration plate system is provided, the calibration system comprising a plurality of calibration plates, each calibration plate having a plurality of hole positions, each calibration plate having a hole at at least one hole position, each hole position having a corresponding binary code according to whether there is a hole at the hole position, each calibration plate having a corresponding plate code according to the binary codes of the plurality of hole positions, and a Hamming distance between the plate codes of any two calibration plates among the plurality of calibration plates being greater than or equal to 1.

[0008] According to a fourth aspect of the present disclosure, a computing device is provided, comprising: one or more processors, and a memory for storing a program, wherein the program comprises instructions, and when the instructions are executed by the processor, the processor executes the method described in the first aspect and / or the second aspect above.

[0009] According to a fifth aspect of the present disclosure, a computer-readable storage medium storing a program is provided, wherein the program includes instructions, which, when executed by one or more processors of a computing device, cause the computing device to execute the method described in the first aspect and / or the second aspect above.

[0010] According to a sixth aspect of the present disclosure, a computer program product is provided, comprising a computer program, which implements the method described in the first aspect and / or the second aspect when executed by a processor.

[0011] According to the technical solution disclosed in the present invention, a reference marker that can be used for laser radar point cloud data is provided. Each calibration plate has multiple calibration hole positions, each calibration plate has a hole at at least one calibration hole position, each calibration hole position has a corresponding binary code according to whether there is a hole at the calibration hole position, and each calibration plate has a corresponding plate code according to the binary codes of the multiple calibration hole positions, so that each calibration plate has a unique identifier. A target point cloud cluster that meets the size constraints of the calibration plate can be identified from the point cloud data, and multiple target hole positions are determined from the target point cloud cluster based on the attribute information of the calibration plate. The number of point cloud points in each target hole position determines whether there is a hole at each target hole position; the binary code of each target hole position is determined based on whether there is a hole at each target hole position; and the plate code corresponding to the target point cloud cluster is determined based on the binary code of each target hole position. Therefore, the possible calibration plate and calibration plate code can be determined from a frame of point cloud data. The laser radar parameters can be calibrated based on the point cloud data and plate code of the calibration plate, thereby improving the versatility, accuracy and robustness of sensor calibration. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0013] Figure 1 Schematic diagram of a calibration plate according to an embodiment of the present disclosure;

[0014] Figure 2 A schematic diagram of a flow chart of a method for determining a calibration plate according to an embodiment of the present disclosure;

[0015] Figure 3 A schematic diagram of a flow chart of a method for determining a calibration plate according to an embodiment of the present disclosure;

[0016] Figure 4 An example diagram of determining a target point cloud cluster designed for an embodiment of the present disclosure;

[0017] Figure 5 This is an example diagram of determining reference points involved in the embodiments of the present disclosure;

[0018] Figure 6 Another schematic flow chart of the method for determining a calibration plate provided in an embodiment of the present disclosure;

[0019] Figure 7 A system architecture diagram applicable to the embodiments of the present disclosure;

[0020] Figure 8 A schematic diagram of a flow chart of a calibration method provided in an embodiment of the present disclosure;

[0021] Figure 9 A schematic diagram of a process for calibrating a camera and a lidar according to an embodiment of the present disclosure;

[0022] Figure 10 Schematic diagram of an exemplary implementation process for calibrating a camera and a lidar based on the initial values ​​of the camera's intrinsic parameters and extrinsic parameters in an embodiment of the present disclosure;

[0023] Figure 11 Another schematic diagram of a process for calibrating a camera and a lidar according to an embodiment of the present disclosure;

[0024] Figure 12 A schematic block diagram of a computing device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0025] The following will provide a clear and complete description of the technical solutions in the embodiments of the present disclosure, in conjunction with the accompanying drawings. It should be noted that the described embodiments represent only a portion of the embodiments of the present disclosure, and are not intended to be exhaustive. Based on the embodiments of the present disclosure, all other embodiments derived by persons of ordinary skill in the art without inventive effort are intended to fall within the scope of protection of the present disclosure. The terms used in the embodiments of the present disclosure are intended solely to describe specific embodiments and are not intended to limit the present disclosure. The singular forms "a," "the," and "the" used in the embodiments of the present disclosure and the appended claims are intended to include the plural forms, unless the context clearly indicates otherwise. It should be understood that the term "and / or" as used herein is merely a description of an association between associated objects, indicating that three possible relationships can exist. For example, "A and / or B" can represent: A exists alone, A and B exist simultaneously, or B exists alone. Furthermore, the character " / " as used herein generally indicates that the associated objects are in an "or" relationship. Depending on the context, the term "if" as used herein can be interpreted as meaning "when," "when," "in response to determining," or "in response to detecting." Similarly, depending on the context, the phrase "if determined" or "if (stated condition or event) is detected" may be interpreted as "when determined" or "in response to determining" or "when detecting (stated condition or event)" or "in response to detecting (stated condition or event)."

[0026] Figure 1A schematic diagram of a calibration plate system according to an embodiment of the present disclosure is shown. The calibration plate system includes multiple calibration plates 100, each of which has multiple hole positions, and each calibration plate 100 has a hole at at least one hole position. Each hole position has a corresponding binary code based on whether there is a hole at the hole position. Each calibration plate 100 has a corresponding plate code based on the binary codes of the multiple hole positions, and the Hamming distance between the plate codes of any two calibration plates 100 in the multiple calibration plates 100 is greater than or equal to 1. The hole positions of the multiple calibration plates 100 are the same, and the plate code of each calibration plate 100 is determined based on the positional order of the hole positions in the calibration plate and the binary codes of the calibration hole positions. Figure 1 Three calibration plates are shown as an example. In fact, the embodiment of the present disclosure does not limit the number of calibration plates included in the calibration plate system.

[0027] In some embodiments, the binary code includes 0 and 1, and the code is 1 if there is a hole and 0 if there is no hole, and vice versa. For example, for a 4×4 calibration plate, there are 16 hole positions in 4×4. The first calibration plate has holes at all 16 hole positions, and the plate code of the calibration plate is "111111111111". The second calibration plate has no hole at hole position 1 and is empty at the other 15 hole positions, and the plate code of the calibration plate is "0111111111111". The third calibration plate has no holes at hole positions 3, 5, 11, and 16, and the plate code of the calibration plate is "1101011111011110".

[0028] The Hamming distance represents the number of steps required for one set of binary data to become another set of data (it represents the number of different characters in corresponding positions of two binary strings of the same length). The smaller the Hamming distance, the higher the similarity between the two code values. For example, the Hamming distance of "1111 1111 1111 1111" and "1101 01111101 1110" is 4. The embodiment of the present disclosure uses the Hamming distance to represent the difference between two calibration plates. The Hamming distance of the plate codes of any two calibration plates is greater than or equal to a preset threshold, for example, greater than or equal to 1, that is, the structure of at least one well position in any two calibration plates is different. The preset threshold can be set according to the number of well positions. For example, for a 9-well calibration plate, the preset threshold can be 3, and the Hamming distance of the plate codes of any two calibration plates is greater than or equal to 3. For a 16-well calibration plate, the preset threshold can be 4. For a 25-well calibration plate, the preset threshold can be 5. Of course, these are only examples, and those skilled in the art can set the threshold of the Hamming distance as needed, and the present disclosure does not limit this.

[0029] In some embodiments, the hole positions of the calibration plate 100 include a first hole position and a second hole position. The hole located at the first hole position can be referred to as the first calibration hole 110, and the hole located at the second hole position can be referred to as the second calibration hole 130. The second hole position must have a hole, while the first hole position may or may not have a hole. The second hole position is located in the boundary area of ​​the calibration plate, and the first hole position is located in the middle area of ​​the calibration plate. The area between the middle area and the plate boundary of the calibration plate is the boundary area, and the second hole position is located in the boundary area of ​​the calibration plate.

[0030] For example, a second calibration hole is provided near a vertex of the calibration plate (i.e., within a predetermined range from the vertex). Furthermore, if the quadrilateral calibration plate has four vertex corners, a second calibration hole 130 may be provided near each vertex corner. The second calibration holes may be symmetrically distributed in the calibration plate.

[0031] For example, if the size of the first calibration hole 110 is smaller than the size of the second calibration hole 130, that is, the area occupied by the first calibration hole 110 is smaller than the area occupied by the second calibration hole 130. In this way, the second calibration hole 130 can be quickly found first, and then the first calibration hole 101 can be found based on the second calibration hole 130. Moreover, the second calibration hole 130 can be verified according to the pre-stored calibration plate attribute information first, and then the first calibration hole 110 can be found based on the second calibration hole 130. If the calibration hole is a circular hole, the diameter of the first calibration hole 110 is smaller than the diameter of the second calibration hole 130. Of course, the first calibration hole 110 and the second calibration hole 130 can also be holes of different shapes, for example, the first calibration hole 110 is a circular hole and the second calibration hole 130 is a polygonal hole, or vice versa, and the present disclosure does not limit this.

[0032] It should be understood that after the style of the calibration plate 100 is fixed, the various attribute information of the calibration plate 100 can be stored as a priori values ​​so that the calibration plate area in the image or point cloud data can be determined based on these attribute information. The attribute information of the calibration plate 100 includes but is not limited to the size information of the calibration plate 100, the sequence number (such as the vertex number) and position information of the characteristic points of the calibration plate, the position information of the first hole position, the position information of the second hole position, the size information of the first hole position, and the size information of the second hole position. Among them, the position information of each hole position includes but is not limited to the sequence number of each hole position (such as the hole position), the position relationship of the characteristic point of each hole position relative to a specific point (such as the characteristic point of other hole positions or the characteristic point of the calibration plate), and the characteristic point includes but is not limited to the center point, vertex, etc. The size information of the calibration plate 100 includes but is not limited to length, width, diameter, aspect ratio, etc. The size information of each hole position includes but is not limited to length, width, diameter, aspect ratio, and the distance value between each hole position and other hole positions.

[0033] The position information of the first hole position includes the positional relationship between the first hole position and the plate vertex of the calibration plate, and specifically may include the positional relationship between the center point of the first hole position and the plate vertex, and the positional relationship between each vertex of the first hole position and the plate vertex. The positional information of the second hole position includes but is not limited to the positional relationship between the second hole position and the first hole position, and the positional relationship between the second hole position and the plate vertex of the calibration plate. The positional relationship between the second hole position and the plate vertex of the calibration plate includes but is not limited to the positional relationship between the characteristic center point of the second hole position and the plate vertex. The positional relationship between the second hole position and the first hole position includes but is not limited to the positional relationship between the characteristic point of the second hole position and the characteristic point of the first hole position, and the positional relationship between the characteristic point of the second hole position and the characteristic point of the first hole position. Positional relationships include distance values, relative coordinate values, etc.

[0034] The characteristic points of each hole position include but are not limited to the center point, vertex, etc. of each hole position. The plate vertex includes one or more vertices of the calibration plate. In one implementation, the positional relationship of the characteristic points of each hole position relative to the same plate vertex can be uniformly recorded. In addition, the attribute information of the calibration plate can also include the coordinates of the calibration plate coordinate system of each characteristic point in the calibration plate (the vertex and center point of the calibration plate, the vertex and center point of the first hole position, the vertex and center point of the second hole position, etc.). The calibration plate coordinate system can take the vertex or center point of the calibration plate as the origin, the plane where the calibration plate is located as the xy plane, and the axis perpendicular to the plane where the calibration plate is located as the z axis.

[0035] Figure 2 The flowchart of the calibration plate determination method (also referred to as calibration plate pattern generation method) according to one embodiment of the present disclosure is shown. The method can be executed in a computing device and the pattern of the calibration plate to be produced can be determined according to the method. Figure 2 As shown, the calibration plate determination method includes:

[0036] Step 210: Obtain the number and distribution of the hole positions of the calibration plate, wherein each hole position has a corresponding binary code according to whether there is a hole at the hole position, and each calibration plate has a corresponding plate code according to the binary codes of the plurality of hole positions;

[0037] Step 230: determining a plurality of board codes according to the number of hole positions and the hole position distribution, wherein a Hamming distance between any two board codes in the plurality of board codes is greater than or equal to 1; and

[0038] Step 250: Generate a calibration plate pattern corresponding to each plate code in the plurality of plate codes.

[0039] In some embodiments, the plate code of each calibration plate is determined based on the positional sequence of each hole position and the binary code of each hole position. Each calibration plate has a hole at at least one hole position. The hole position distribution refers to the number of hole positions in each row and column. For example, a hole position distribution according to i×j includes i rows of holes and j columns of holes, for a total of i×j hole positions. A hole position distribution according to 4×5 includes 4 rows of holes and 5 columns of holes, for a total of 20 hole positions.

[0040] For a 16-hole calibration plate, there are 2 holes in total, depending on whether there is a hole at each hole position. 16 Plate codes, this order of magnitude is undoubtedly very large. In order to facilitate the detection of the produced calibration plates, the embodiment of the present disclosure limits the Hamming distance of the plate codes between any two calibration plates to be greater than or equal to a predetermined threshold, for example, greater than or equal to 1, which means that the position of at least one hole between any two calibration plates is different. The preset threshold can be set according to the number of hole positions. For example, for a 9-hole calibration plate, the predetermined threshold can be 3. For a 16-hole calibration plate, the predetermined threshold can be 4. For a 25-hole calibration plate, the predetermined threshold can be 5. Of course, these are only examples, and those skilled in the art can set the threshold of the Hamming distance as needed, and the present disclosure does not limit this.

[0041] According to the condition that "the Hamming distance of the plate codes between any two calibration plates is greater than or equal to the preset threshold", the 16 A predetermined number of board codes meeting the criteria are selected from the board codes. The predetermined number can be determined as needed; for example, 50 are selected. Each board code can be assigned a decimal number, with each board code corresponding to a board identifier, resulting in board codes 1, 2, ..., and 50. In some embodiments, considering that the board codes represent a set of binary values, the predetermined number of board codes can be selected in ascending order from the board codes meeting the criteria. These predetermined number of selected board codes can also be assigned decimal numbers in ascending order from the board codes themselves.

[0042] Afterwards, the corresponding calibration plate type can be determined based on the predetermined number of plate codes selected. Assuming that one of the plate codes is "110011111111", it can be determined that the middle area of ​​the calibration plate is a 16-hole calibration plate, with no holes at positions 3 and 4, and holes at the other 12 positions.

[0043] In some embodiments, the multiple plate codes determined in step 230 may be stored as a plate code library. This plate code library can be used as a reference value to identify calibration plates in images and / or point clouds. Because the calibration plates of the disclosed embodiments are manufactured based on this plate code library, if the plate code identified from the image and / or point cloud does not exist in the plate code library, the identified area is an invalid area. Furthermore, the computing device may associate and store the plate code (binary number) and corresponding plate identifier (decimal number) of each calibration plate.

[0044] Figure 3 A calibration plate determination method (also referred to as a calibration plate identification method) according to an embodiment of the present disclosure is shown, and the method can be executed in a computing device.

[0045] like Figure 3 As shown, in step 310, point cloud data collected by the point cloud collection device is obtained. The point cloud data can be stored in a k-dimensional tree structure, which can be used to accelerate subsequent query steps.

[0046] In step 320, target point cloud clusters that meet the size constraints of the calibration plate are identified from the point cloud data. The size constraints of the calibration plate include but are not limited to length constraints, width constraints, and aspect ratio constraints of the calibration plate.

[0047] In step 330 , a plurality of target hole positions are determined from the target point cloud cluster according to the attribute information of the calibration plate.

[0048] In step 340 , it is determined whether there is a hole at each target hole position based on the number of point cloud points in each target hole position.

[0049] In step 350 , a binary code of each target hole position is determined according to whether there is a hole at each target hole position.

[0050] In step 360 , the plate code corresponding to the target point cloud cluster is determined based on the binary code of each target hole position.

[0051] In some embodiments, as Figure 4 As shown, step 320 includes:

[0052] Step 321: Extract edge feature points from the point cloud data.

[0053] Step 322: Cluster the edge feature points to obtain at least one candidate point cloud cluster. The clustered point cloud clusters are obtained using the point cloud coordinates and laser beam identifiers (i.e., ring information, line identifiers, indicating which laser beam the point cloud point originated from). The clustering algorithm can be a commonly used existing clustering algorithm, and this disclosure does not impose any restrictions on this.

[0054] Step 323: Determine the minimum bounding box of each candidate point cloud cluster, that is, determine the minimum circumscribed geometric body of each candidate point cloud cluster, such as the minimum circumscribed sphere, the minimum circumscribed cuboid, etc.

[0055] Step 324: Select a target point cloud cluster from the at least one candidate point cloud cluster whose minimum bounding box meets the size constraints of the calibration plate. The size constraints include, but are not limited to, the length of the minimum bounding box being greater than or equal to the length of the calibration plate, the width of the minimum bounding box being greater than or equal to the width of the calibration plate, and the aspect ratio of the minimum bounding box being less than or equal to the aspect ratio of the calibration plate.

[0056] Considering that step 321 is clustered based on edge feature points, after determining the minimum bounding box that meets the size constraints of the calibration plate in step 324, all point clouds within the minimum bounding box can be restored based on the size of the minimum bounding box and the original point cloud data of the frame to obtain the target point cloud cluster. Of course, in step 320, it is also possible to directly cluster the point cloud points without extracting edge feature points to obtain the target point cloud cluster that meets the size constraints of the calibration plate.

[0057] In one implementation, step 321 includes:

[0058] Step 321A: Identify multiple point cloud points corresponding to the same laser beam from the point cloud data. These multiple point cloud points of the same laser beam can be referred to as a group of points corresponding to the laser beam. In some embodiments, the point cloud points corresponding to the same laser beam can be identified based on the laser beam identifier of each point cloud point. The angle value of the laser beam corresponding to each point cloud point (i.e., the angle between the laser beam used to scan the point cloud point and the horizontal plane) can also be calculated based on the coordinate value of each point cloud point. The point cloud points corresponding to the same laser beam are determined based on the angle value corresponding to each point cloud point. In addition, these two methods can be combined. When the point cloud point has a laser beam identifier, the laser beam identifier is used for judgment. When the point cloud point does not have a laser beam identifier, the angle value is used for judgment. When some points have a laser beam identifier and some points do not have a laser beam identifier, the point cloud points corresponding to the same laser beam are determined based on the laser beam angle value and / or laser beam identifier corresponding to each point. For example, if a point A has no laser beam mark, the first angle value of the laser beam corresponding to the point A is calculated, and l points within a predetermined range around the point are detected to see if they have laser beam marks. The second angle values ​​corresponding to the l points with laser beam marks are calculated, and the point with the angle value closest to point A and the difference between the two points is less than or equal to the preset threshold is determined. It is assumed to be point B, and it is considered that point A and point B correspond to the same laser beam.

[0059] Step 321B: Calculate the distance between each point in the multiple point cloud points and every other point cloud point based on the point cloud data. Specifically, assuming there are p point cloud points corresponding to the same laser beam, for any point cloud point C among the p point cloud points, calculate the mean distance between the point cloud point C and a predetermined number of adjacent point cloud points. The mean distance is the average value of the distances between point cloud point C and each of the other point cloud points. The predetermined number of adjacent point cloud points includes a predetermined number of point cloud points located to the left and right of point cloud point C, for example, five points to the left and right of point C, respectively.

[0060] Step 321C: In response to the distance being greater than or equal to a predetermined threshold, the corresponding point cloud point is determined to be an edge feature point. Further, in response to the distance mean being greater than or equal to a predetermined threshold, the corresponding point cloud point is determined to be an edge feature point. The present disclosure extracts edge feature points of spatially discontinuous edges in a point cloud. If a point is determined to be an edge feature point, then the q consecutive points adjacent to the point (for example, the four consecutive points adjacent to the point corresponding to the same laser beam) will no longer be determined to be edge feature points.

[0061] After step 320 , the target point cloud cluster in each frame of point cloud data is obtained. Each target point cloud cluster may correspond to a calibration plate. Then, in step 330 , the target hole positions that are most likely to correspond to each hole position of the calibration plate in the point cloud data can be determined.

[0062] In some embodiments, step 330 includes:

[0063] Step 331: Determine at least one target vertex from the target point cloud cluster.

[0064] In one implementation, the point cloud points in the target point cloud cluster can be traversed, and each traversed point cloud point is used as a candidate vertex. Based on the candidate vertex, a candidate region of the same size as the calibration plate is determined. Based on the number of point cloud points within the candidate region corresponding to each candidate vertex, at least one target vertex is determined from the traversed candidate vertices. Optionally, the candidate region with the largest number of point cloud points can be determined as the calibration plate region, and the candidate vertex corresponding to the calibration plate region can be used as the target vertex.

[0065] For example, take the traversed point cloud point D as a candidate vertex, and use this point D as the vertex to determine a candidate region DEFG based on the length and width of the calibration plate. Count the point cloud data within this candidate region. By traversing multiple points, multiple candidate regions can be obtained. The candidate region with the largest number of point cloud points can be considered to be the region in the point cloud data corresponding to the calibration plate, and the candidate vertex corresponding to this candidate region (i.e., the target vertex) may be the point with the highest probability corresponding to the plate vertex.

[0066] Here, the maximum probability coordinates of each vertex of the calibration plate in the point cloud frame can be determined. Of course, if the pre-stored attribute information contains the positional relationship of the feature points of each hole position relative to the same plate vertex, then it is only necessary to determine the possible position of the plate vertex in the point cloud frame. The starting point of the traversal is determined according to the orientation of the plate vertex in the calibration plate. For example, if the plate vertex is the upper left corner vertex of the calibration plate, then when traversing the point cloud cluster, the traversal starts from the upper left corner point cloud point of the point cloud cluster. Specifically, a point cloud point located in the upper left corner can be selected and traversed in a left-to-right and top-to-bottom traversal order.

[0067] In addition, before traversing the target point cloud cluster, a plane fitting algorithm such as RANSAC can be used to fit the target point cloud cluster to a plane. The target point cloud cluster can be transformed into a planar point cloud cluster based on the plane parameters obtained by fitting. For example, the target point cloud cluster can be transformed into a plane with a normal of [0, 0, 1]. In this way, the planar point cloud cluster can be traversed, for example, starting from the point at the upper left corner of the planar point cloud cluster.

[0068] Step 332: Determine multiple target hole positions based on the attribute information of the calibration plate and at least one target vertex. The target hole position can be considered as the maximum probability position of the first hole position of the calibration plate in the point cloud frame.

[0069] In one implementation, multiple target hole positions can be determined by referring to the positional relationship between the hole positions in the calibration plate and the plate vertices, as well as the coordinates of the target vertices. Specifically, the multiple target hole positions can be determined by referring to the positional relationship between the feature point of the first hole position and the plate vertices, the coordinates of the target vertices, and the size information of the first hole position.

[0070] In another implementation, if the calibration plate is provided with a second calibration hole at the second hole position, the maximum probability position of the second hole position in the point cloud frame can be found first, and then the target hole position can be determined based on the positional relationship between the second hole position and the first hole position.

[0071] Thus, in some embodiments, step 332 includes:

[0072] Step 332A: Determine a reference point for the second hole position in the point cloud data based on the second positional relationship between the second hole position and the plate vertex in the calibration plate, as well as the target vertex. Knowing the coordinates of the reference point and the dimension information of the second hole position in the pre-stored attribute information can determine the reference hole position from the point cloud data. Here, the reference point can be considered the point with the highest probability of the feature point of the second hole position in the calibration plate in the point cloud frame, and the target vertex can be considered the point with the highest probability of the plate vertex in the calibration plate in the point cloud frame. The position with the highest probability of the first hole position in the calibration plate in the point cloud frame can be referred to as the target hole position, and the position with the highest probability of the second hole position in the calibration plate in the point cloud frame can be referred to as the reference hole position. If the calibration plate has multiple second hole positions, each with a reference point, multiple reference points can be determined from the calibration plate region (e.g., the center points of the four circular holes at the four corners). In one implementation, the point clouds at the multiple corners of the target point cloud cluster can be traversed separately to independently determine the reference hole position and reference point at each corner.

[0073] In some embodiments, an anti-false detection operation can also be performed in this step. For example, the distance values ​​between each pair of reference hole positions in a plurality of reference hole positions are calculated respectively, and by comparing the distance values ​​between each pair of reference hole positions with the true value, it is determined whether the target point cloud cluster is an invalid point cloud cluster, and the invalid point cloud cluster is discarded. The true value is based on the actual distance value of each pair of second hole positions in the attribute information of the calibration plate. Furthermore, the attribute information also includes the mean of the distance between each pair of second hole positions. In this way, the mean of the distance between each pair of reference hole positions is calculated from the point cloud frame. If the difference between the two means is greater than a predetermined threshold, the target point cloud cluster is determined to be an invalid point cloud cluster.

[0074] Step 332B: Determine multiple target hole positions based on the first positional relationship between the first hole position and the second hole position in the calibration plate, the reference point, and the size information of the first hole position. For example, based on the positional relationship between the feature points of each first hole position and the feature points of the second hole position in the pre-stored attribute information and the determined reference point, the corresponding point of the feature point of the first hole position in the point cloud frame (which can be called the target point of the target hole position) can be determined. Combined with the size information of the first hole position, the multiple target hole positions can be determined.

[0075] Moreover, in step 332B, the target hole position can also be determined by integrating the coordinates of multiple reference points determined in step 332A, that is, the target hole position is determined based on the positional relationship between the first hole position in the calibration plate and the characteristic points of each second hole position located at the vertex.

[0076] In some embodiments, step 332A includes determining an initial point based on the second positional relationship between the second hole location and the plate vertex, as well as the target vertex. This initial point can be considered the initial screening position for the reference point. A first region is then determined based on the size information of the second hole location, centered on the initial point. The candidate points within the first region are traversed, and a second region is determined based on the size information of the second hole location, centered on each candidate point. This yields second regions corresponding to multiple candidate points. The reference point is determined from the candidate points traversed based on the number of point cloud points within each second region. For example, the candidate point corresponding to the second region containing the fewest point cloud points is used as the reference point.

[0077] In one implementation, the first region is the minimum circumscribed polygon of the second hole position, and the second region is the maximum inscribed polygon of the second hole position. Furthermore, the second calibration hole is a circular hole, and both the first region and the second region are square regions. The side length of the first region is the side length of the minimum circumscribed square of the second calibration hole, and the side length of the second region is the side length of the maximum inscribed square of the second calibration hole.

[0078] In one implementation, the computing device may also determine the calibration plate area from the point cloud data, and divide the calibration plate area into a two-dimensional grid, that is, into a grid with multiple rows and columns, for example, a grid with the number of rows and columns being greater than or equal to 1000, but of course not limited thereto. The method for determining the calibration plate area has been disclosed in the description based on step 331 and will not be repeated here. The candidate points in 332A are the grid vertices within the first area. In this way, the grid vertices within the first area can be traversed, and each traversed grid vertex can be used as the center to determine the second area according to the size information of the second hole position, thereby obtaining the second areas corresponding to the multiple grid vertices, and the reference point can be determined from the traversed grid vertices according to the number of point cloud points within each second area.

[0079] Based on this, determining the reference point from the traversed candidate points based on the number of point cloud points within each second region includes: for each mesh vertex, generating a label for the mesh vertex based on whether a point cloud point exists within a predetermined range of the mesh vertex, where the label includes a first label and a second label; determining the number of mesh vertices labeled with the first label within each second region; and determining the second center point from the traversed mesh vertices based on the number of mesh vertices corresponding to each second region. In one example, the first label is 1, indicating the presence of a point cloud point, and the second label is 0, indicating the absence of a point cloud point; of course, the reverse is also possible, and this disclosure is not limited to this.

[0080] After the calibration plate area is divided into a grid, multiple grid vertices will exist. In some embodiments, the determination is made based on the grid vertices. For each grid vertex, if a point cloud point exists within a predetermined range of the grid vertex, the grid vertex is marked as a first marker; if no point cloud point exists, the grid vertex is marked as a second marker. In other embodiments, the determination is made based on the point cloud points. For each point cloud point, the grid vertex closest to each point cloud point is determined, and the determined grid vertex is marked as the first marker.

[0081] In one implementation, mesh vertices that have already been marked with the first mark will not be marked again, while mesh vertices that have not been marked before or have already been marked with the second mark may be marked with the first mark in subsequent judgments. In another implementation, the number of times each mesh vertex is marked with the first mark can be counted, and the number of times it is marked with the first mark is increased by 1 each time it is marked with the first mark. This can provide the mesh vertices marked with the first mark in the calibration plate area and the number of times the mesh vertex is marked with the first mark.

[0082] By judging each point cloud point, it is possible to determine which grid vertices are marked as the first mark and which are marked as the second mark. When the number of grids divided into the calibration plate area is large enough, the disclosed embodiment can even achieve a one-to-one correspondence between a point cloud point and a grid vertex, that is, each point cloud point can be uniquely associated with a grid vertex.

[0083] Afterwards, the number of mesh vertices marked with the first marker in each second region can be counted, and the reference point can be determined from the traversed mesh vertices based on this count. For example, the mesh vertex corresponding to the second region with the least number of markers can be determined as the reference point. Furthermore, if a mesh vertex is marked with the first marker multiple times, the count value can be simply 1 or the actual number of times it has been marked, and this disclosure does not impose any restrictions on this.

[0084] Figure 5 FIG. 1 shows a schematic diagram of determining a reference point according to an embodiment of the present disclosure. Figure 5 As shown, it is assumed that four target vertices D, E, F, and G are identified from the point cloud data, and the identified calibration plate area is DEFG. The calibration plate area is divided into a grid of 12 rows and 12 columns (i.e., 12×12). According to the attribute information of the calibration plate and the position of the target vertex, an initial point P is determined, and a square area is constructed with the point P as the center and the diameter of the calibration hole as the side length, i.e., the first area 510. There are 4 grid vertices in the first area, represented by triangle points, and each grid vertex has its own label value. Then, a square area is constructed with each grid vertex as the center point and the side length of the inscribed square of the calibration hole as the side length, i.e., the second area 530. Figure 5The figure shows an example of a second region 530 constructed with the first grid point Q as the center. The number of point cloud points in each second region is counted. If the number of point cloud points in the second region corresponding to the first grid point Q is the smallest, the first grid point Q is determined as a reference point, which may be the corresponding point of the center point of the calibration hole in the point cloud data.

[0085] In one implementation, the number of mesh vertices marked with the first mark in each second area may be counted, and the mesh vertex corresponding to the second area with the smallest number value may be determined as the reference point.

[0086] Afterwards, multiple target hole positions in the calibration plate area can be determined based on the position of the reference point and the attribute information of the calibration plate, and in step 340 , it is determined whether there is a hole at each target hole position.

[0087] In some embodiments, step 340 includes:

[0088] Step 341: Determine the maximum inscribed polygon corresponding to each target hole position, where the maximum inscribed polygon includes but is not limited to the maximum inscribed square;

[0089] Step 342: In response to the number of point cloud points within the largest inscribed polygon being less than or equal to a predetermined threshold, it is determined that there is a hole at the target hole position. Otherwise, it is determined that there is no hole at the target hole position.

[0090] In some embodiments, the calibration plate determination / identification method may further include a verification step to improve detection accuracy. Specifically, the calibration plate determination / identification method may further include:

[0091] Step 370: Calculate the distance between each pair of target hole locations in the plurality of target hole locations, and compare the distance between each pair of target hole locations with the true value to determine whether the target point cloud cluster is an invalid point cloud cluster, and discard the invalid point cloud cluster. The true value is the actual distance between each pair of target hole locations in the attribute information of the calibration plate.

[0092] Step 380: Calculate the distance between each pair of reference holes in the plurality of reference holes, and compare the distance between each pair of reference holes with the true value to determine whether the target point cloud cluster is an invalid point cloud cluster, and discard the invalid point cloud cluster. The true value is based on the actual distance between each pair of reference holes in the attribute information of the calibration plate.

[0093] Step 390: Determine whether the target point cloud cluster is an invalid point cloud cluster based on whether the determined plate code exists in a pre-stored plate code library, and discard the invalid point cloud cluster. The plate code library includes the plate code of the calibration plate. In other words, the plate code library includes multiple plate codes determined based on the calibration plate pattern generation method. The calibration plate of the disclosed embodiment is manufactured based on these multiple plate codes. Therefore, the plate code library stores the plate codes corresponding to each calibration plate in the calibration plate system.

[0094] If a target point cloud cluster is judged to be an invalid point cloud cluster, the calculation of the target point cloud cluster can be terminated, and the calculation results based on the target point cloud cluster, such as reference hole positions, target hole positions, reference points, target points, etc., are discarded, and the next target point cloud cluster is judged and calculated.

[0095] Through the above steps, we can obtain the valid point cloud cluster in each frame of point cloud data, the calibration plate area corresponding to the calibration plate in the point cloud frame, the plate code corresponding to the calibration plate area, the reference hole position corresponding to the first hole position of the calibration plate in the point cloud frame, the reference point corresponding to the feature point of the first hole position in the point cloud frame, the target hole position corresponding to the second hole position in the point cloud frame, and the target point corresponding to the feature point of the second hole position in the point cloud frame.

[0096] The LiDAR coordinates of these feature points are extracted from each frame of point cloud data. Based on the pre-stored calibration plate coordinates of these points, the conversion relationship between the calibration plate coordinate system and the LiDAR coordinate system is calculated. Furthermore, the pixel coordinates of these feature points are extracted from the image data captured by the image acquisition device, and the conversion relationship between the calibration plate coordinate system and the camera coordinate system is determined based on the camera intrinsic parameter model. The conversion relationship between the camera coordinate system and the LiDAR coordinate system can be determined by transferring the calibration plate coordinate system.

[0097] Figure 6 A flowchart of a calibration plate determination method (or calibration plate identification method) according to another embodiment of the present disclosure is shown. The method can be executed on a computing device.

[0098] like Figure 6 As shown, in step 601, the point cloud acquisition device acquires each frame of point cloud data. Each frame of point cloud can also be called each point cloud frame. A point cloud frame may acquire data of multiple calibration plates, so the calibration plate areas corresponding to different calibration plates can be extracted accordingly.

[0099] In step 602, edge feature points are extracted from each point cloud frame. Specifically, the edge feature points can be determined based on the distance values ​​between each point cloud point and multiple left and right points in the same laser beam.

[0100] In step 603 , edge feature points are clustered according to the coordinate data of the point cloud points and the laser beam identification to obtain a plurality of clustered point cloud clusters.

[0101] In step 604 , the minimum bounding box of each point cloud cluster is calculated, and the minimum bounding box that meets the size constraints of the calibration plate is determined based on the attribute information of the calibration plate, such as the length, width, and aspect ratio.

[0102] In step 605 , based on the determined minimum bounding box size that meets the size constraint of the calibration plate and the original point cloud data, all point cloud points within the minimum bounding box are restored to obtain a target point cloud cluster.

[0103] In step 606, for each target point cloud cluster in each point cloud frame, the point cloud points of each target point cloud cluster are traversed in turn to determine the target vertex, that is, to determine the maximum probability position of the plate vertex of the calibration plate in the point cloud frame.

[0104] In step 607, the reference hole position (the maximum probability position of the second calibration hole in the point cloud frame) and the reference point (the maximum probability position of the feature point of the second calibration hole in the point cloud frame) are determined according to the attribute information of the calibration plate and the coordinates of the target vertex.

[0105] In step 608, the distance values ​​between each pair of reference hole positions in the plurality of reference hole positions are calculated respectively, and by comparing the distance values ​​between each pair of reference hole positions with the true value, it is determined whether the target point cloud cluster is an invalid point cloud cluster, and the invalid point cloud cluster is discarded.

[0106] In step S609, based on the attribute information of the calibration plate and the coordinates of the reference point, multiple target hole positions (the maximum probability position of the first calibration hole in the point cloud frame) and the target point of each target hole position (the maximum probability position of the feature point of the first calibration hole in the point cloud frame) are determined.

[0107] In step 610, whether there is a hole at each target hole position is determined based on the number of point cloud points at the target hole position, and a code for each target hole position is generated, thereby obtaining the plate code of the calibration plate. In one implementation, whether there is a hole at each target hole position can be determined based on the number of point cloud points within the largest inscribed square of each target hole position.

[0108] In step 611 , if the plate code of the calibration plate does not exist in the pre-stored plate code library, the target point cloud cluster is determined to be an invalid point cloud cluster, and the invalid point cloud cluster and the result calculated based on the invalid point cloud cluster are discarded.

[0109] In step 612, based on the coordinates of the feature points of the calibration plate and the plate encoding, tasks such as multi-sensor calibration and sensor positioning are performed. For example, based on the calibration plate coordinate system coordinates and the lidar coordinate system coordinates of each feature point of the calibration hole, the transformation relationship between the calibration plate coordinate system and the lidar coordinate system is calculated (each transformation relationship in the embodiment of the present disclosure may include a translation vector and a transformation matrix). For example, if the target vertex in the upper left corner is determined based on the upper left corner of the target point cloud cluster, and the lidar coordinate system coordinates of the center point of the second calibration hole located in the upper left corner are determined based on the target vertex and the attribute information of the calibration plate, and the calibration plate coordinate system coordinates of the center point are already pre-stored in the structure object, the lidar coordinates and calibration plate coordinate system coordinates of the same center point can be matched. Similarly, the lidar coordinates and calibration plate coordinate system coordinates of other feature points of the calibration plate can be determined accordingly.

[0110] In addition, if the calibration plate is placed in the calibration room, the calibration room has a corresponding calibration room coordinate system. Similarly, the coordinates of each feature point of each calibration plate in the calibration room coordinate system can be pre-stored. Then, the position of the calibration plate in the calibration room can be obtained based on the transformation relationship between the calibration plate coordinate system coordinates and the calibration room coordinate system coordinates of each feature point. Moreover, through the transfer of the calibration plate coordinate system, the transformation relationship of various other coordinate systems can be obtained. For example, based on the transformation relationship between the calibration plate coordinate system and the camera coordinate system, and the transformation relationship between the calibration plate coordinate system and the lidar coordinate system, the transformation relationship between the camera coordinate system and the lidar coordinate system can be calculated.

[0111] Through the above process, we can obtain the valid point cloud clusters in each frame of point cloud, as well as the feature point coordinates of each valid point cloud cluster, the position and pose of the calibration plate, and the plate identification of the calibration plate.

[0112] In one implementation, before step 606, the process further includes the following steps: performing plane fitting on each target point cloud cluster, and transforming the point clouds in the target point cloud cluster to a plane with a normal of [0, 0, 1] based on the plane parameters obtained by fitting. Before step 609, the process further includes the following steps: transforming the coordinates of the reference point from the plane with a normal of [0, 0, 1] back to the original plane to obtain the coordinates of the LiDAR system.

[0113] It should be understood that the present disclosure mentions a variety of predetermined thresholds corresponding to a variety of parameters, and those skilled in the art can determine the predetermined threshold corresponding to each parameter as needed, and the present disclosure does not limit this.

[0114] The disclosed embodiments provide a universal, real-time fiducial marker format for LiDAR calibration. This method is compatible with calibration plates of various sizes and styles, and is suitable for plate point cloud extraction, plate feature point extraction, plate pose solution, and plate encoding and decoding for various types of LiDAR. This method operates robustly in dark environments and those with varying brightness. Furthermore, this method requires only the LiDAR's spatial information related to x, y, and z data and laser beam identification, and does not rely on laser intensity data, resulting in high accuracy and wide applicability.

[0115] The disclosed embodiment also designs a matching plate code library, which serves as a dictionary library to guide the generation of calibration plate styles and can also be used to query and judge plate codes. By presetting the minimum Hamming distance of the plate codes of each calibration plate, the distinctiveness between the calibration plates is improved, the accuracy of calibration plate decoding is improved, and thus the accuracy of multi-sensor calibration and positioning is improved.

[0116] Furthermore, in some embodiments of the present disclosure, in addition to the first and second calibration holes, one or more marks may be provided on the calibration plate, each with a corresponding marking identifier. To better capture valid features during calibration, the marks on the calibration plate may be symmetrically arranged, the second calibration holes on the calibration plate may also be symmetrically arranged, and the first holes may also be symmetrically arranged.

[0117] The mark can be printed, drawn, or affixed to the calibration plate. The mark can be a regular shape such as a quadrilateral, triangle, circle, or other shape. The mark can also be an irregular shape. In some embodiments of the present disclosure, the mark can be, but is not limited to, a QR code, an image, etc., and each mark has a unique mark identifier. For example, the mark can be, but is not limited to, an ArUco mark, i.e., an ArUco QR code.

[0118] Different types of calibration plates can be designed as needed to calibrate different sensors in different scenarios. To facilitate feature capture and camera calibration calculations, different calibration plates, different markings on the plates, and different calibration holes on the plates can be of the same or different sizes, and this disclosure does not impose any restrictions on this.

[0119] The mark of the calibration plate has first feature points, which can be pre-selected as needed. To more accurately capture the features of the mark, the first feature points of the mark can be feature points on the outline of the mark, such as the four vertices of a quadrilateral mark.

[0120] The structural contour of the calibration plate has second characteristic points, and these second characteristic points can also be selected according to actual needs. In one example, the second characteristic points of the structural contour of the calibration plate may include at least one or more of the following: characteristic points of the outer contour of the calibration plate, characteristic points of the calibration holes in the calibration plate, and high reflection points in the calibration plate (high reflectivity materials are set at specific positions of the calibration plate to form high reflection points of the calibration plate, that is, points where the laser intensity information is higher than a predetermined threshold). Among them, the characteristic points of the outer contour of the calibration plate include but are not limited to the four edge vertices of the calibration plate. Among them, the characteristic points of the calibration holes include the characteristic points of the first calibration hole mentioned above, the characteristic points of the second calibration hole, etc., such as the vertices and center points of the calibration holes.

[0121] Furthermore, the attribute information of the calibration plate may also include, but is not limited to, the plate identification of the calibration plate, the number of marks, the mark identification of each mark, the mark size and mark position, the number of first calibration holes, the number of second calibration holes, the position information of the first calibration holes, the position information of the second calibration holes, etc. The position information of each calibration hole may also include information such as the distance from each calibration hole to the adjacent edge of the calibration plate, the distance between the calibration hole and the adjacent mark, and the mark position may include information such as the distance from the mark to the adjacent edge of the calibration plate, the distance between the mark and the adjacent calibration hole, and the distance between the mark and the adjacent mark.

[0122] In one implementation, various attribute information of the calibration plate can be associated and stored through plate identification, mark identification, plate code, plate identification, etc., so that the attribute information of the required calibration plate, mark and calibration hole can be queried efficiently, quickly and accurately.

[0123] In some embodiments of the present disclosure, the above-mentioned calibration method may further include: establishing a calibration plate coordinate system and a structure object of the calibration plate, and storing the calibration plate coordinate system coordinates of each feature point on the calibration plate in the structure object according to the attribute information of the calibration plate, so that the calibration plate coordinate system coordinates of the required feature points can be directly read during calibration, thereby speeding up the calibration process, improving calibration efficiency, and reducing computational complexity.

[0124] Specifically, for each calibration plate, a calibration plate coordinate system of the calibration plate can be constructed first, and then the calibration plate coordinates of all feature points on the calibration plate are calculated based on the attribute information of the calibration plate and written into the structure object of the calibration plate.

[0125] In one implementation, the calibration plate coordinate system coordinates of each feature point on the calibration plate may be stored in a pre-agreed fixed order, so that the calibration plate coordinate system coordinates of one or more required feature points can be directly read.

[0126] In one example, the calibration plate coordinates of each first feature point on the mark can be stored in association with the corresponding mark identifier and plate identifier in order according to the positional order of the marks on the calibration plate. For each mark, the calibration plate coordinates of each first feature point on the mark can be stored in association with the corresponding mark identifier and plate identifier in order according to the positional order of the first feature point on the mark.

[0127] In one example, the calibration plate coordinate system coordinates of the characteristic points of the first calibration holes in the calibration plate (such as the center points of the circular holes) can be stored in association with the corresponding plate identifiers in sequence according to the position sequence of the first calibration holes in the calibration plate. The calibration plate coordinate system coordinates of the characteristic points of the first calibration holes in the calibration plate can be stored in association with the corresponding plate identifiers in sequence according to the position sequence of the second calibration holes. The calibration plate coordinate system coordinates of the characteristic points of the first calibration holes in the calibration plate (such as the center points of the circular holes) can be stored in association with the corresponding plate identifiers in sequence according to the position sequence of the vertices of the calibration plate.

[0128] For each mark on the calibration plate, the calibration plate coordinate system coordinates of the vertices on each mark can be stored in sequence in the clockwise position order of "left, top, right, bottom". At the same time, for each mark, the calibration plate coordinate system coordinates of the four vertices on the mark are associated with the corresponding plate identifier, the sequence number of the mark, and the mark identifier in sequence in the clockwise position order of "top left corner, top right corner, bottom right corner, bottom left corner". For the four vertices on the calibration plate, the calibration plate coordinate system coordinates of the four vertices on the calibration plate, the sequence number of the vertices, and the corresponding plate identifier can also be associated with the storage in sequence in the clockwise position order of "top left corner, top right corner, bottom right corner, bottom left corner".

[0129] For each calibration hole on the calibration plate, the calibration plate coordinate system coordinates of the center point of each calibration hole, the sequence number of the calibration hole and the corresponding plate identifier can also be associated and stored in the clockwise position order of "left, top, right, bottom". Among them, the first calibration hole and the second calibration hole can be stored separately, that is, the center points of the first calibration plate are associated and stored according to the calibration plate coordinate system coordinates, the sequence number of the first calibration hole and the corresponding plate identifier, and the vertices of the first calibration plate can also be associated and stored according to the calibration plate coordinate system coordinates, the sequence number of the first calibration hole and the corresponding plate identifier. The center points of the second calibration plate are associated and stored according to the calibration plate coordinate system coordinates, the sequence number of the first calibration hole and the corresponding plate identifier, and the vertices of the second calibration plate can also be associated and stored according to the calibration plate coordinate system coordinates, the sequence number of the second calibration hole and the corresponding plate identifier.

[0130] In one example, the calibration board structure object may be, but is not limited to, a Board structure object. Using the Board structure object, the calibration board coordinate system coordinates of each feature point can be conveniently associated and stored with the board identifier, marker identifier, and the like. It should be understood that the calibration board structure object may also employ any other type of structure applicable to the present disclosure, and the present disclosure does not impose any limitation thereto.

[0131] By adopting the above storage method, the calibration plate coordinate system coordinates of specific feature points on a calibration plate can be easily and efficiently read through the plate code, plate identification and marking identification of the calibration plate. Then, the sensor calibration can be performed based on the calibration plate coordinate system coordinates and sensor coordinate system coordinates of these points.

[0132] The following describes a method for calibrating a sensor using the calibration plate according to an embodiment of the present disclosure.

[0133] Figure 7 An exemplary system architecture diagram to which the present disclosure can be applied is shown in FIG. Figure 7 The system primarily includes sensors such as cameras, lidar, millimeter-wave radar, inertial navigation, and integrated navigation, as well as a calibration device. Each sensor communicates with the calibration device. The camera is used to capture images of the environment within its field of view. The camera can be, but is not limited to, a short-focus or long-focus camera using a pinhole model, a fisheye camera using a Scaramuzza model, or a fisheye camera using a double sphere model. The calibration device, as a computing device, extracts feature points of a calibration plate based on the camera's image data and the lidar's point cloud data, and then performs sensor calibration based on the extracted feature points.

[0134] The above-mentioned system is mounted on different platforms according to the needs of different scenarios to realize the calibration of sensors. For example, the above-mentioned system can be set on a mobile platform such as a vehicle or a robot, and the mobile platform is stopped at a specified position in a calibration scene such as a calibration room (for example, the center position of the calibration room, etc.) and aligned so that the calibration plates placed in the calibration room are in the common viewing area of ​​the sensor, so as to calibrate the sensor. For another example, the above-mentioned system can be set on a mobile platform such as a vehicle or a robot, and the mobile platform is stopped at any position in any scene, and the calibration plate is arranged in the common viewing area of ​​the sensor, and the calibration of the sensor is realized by moving the calibration plate or switching the posture in the common viewing area.

[0135] As a possible implementation, the calibration device can be installed on a computer terminal with strong computing power deployed on the sensor platform, or on a server deployed independently of the sensor platform. The calibration device can communicate with the sensors on the platform through various wireless communication methods, or can be interconnected with each sensor through a physical interface. The calibration device calibrates the intrinsic and / or extrinsic parameters of each sensor by acquiring, analyzing, and processing the sensor data collected by each sensor.

[0136] It should be understood that Figure 7 The number of calibration devices and sensors in the figure is only for illustration. Any number and type of sensors and calibration devices may be used as required.

[0137] Figure 8 This is a flow chart of a calibration method provided in an embodiment of the present disclosure. The method can be performed by Figure 7 The calibration device in the system shown is performed. Figure 8 , the method may include the following steps:

[0138] Step 801: acquiring images and point cloud data of a calibration plate collected by a camera and a lidar, respectively, wherein the calibration plate is provided with a mark, and the calibration plate includes first feature points of the mark and second feature points of a structural contour of the calibration plate;

[0139] Step 803: Determine the pixel coordinate system coordinates of the first feature point according to the image;

[0140] Step 805: Determine the laser radar coordinate system coordinates of the second feature point based on the point cloud data;

[0141] Step 807: Determine a first transformation relationship between the calibration plate coordinate system and the laser radar coordinate system according to the calibration plate coordinate system coordinates and the laser radar coordinate system coordinates of the second feature point;

[0142] Step 809: Determine the LiDAR coordinates of the first feature point based on the calibration plate coordinates and the first transformation relationship. The calibration plate coordinates of the first feature point can be read from the calibration plate structure object based on the plate identifier of the calibration plate. The calibration plate coordinates of the first feature point can be converted to the LiDAR coordinate system based on the first transformation relationship obtained in step 807. This will yield the LiDAR coordinates of the first feature point.

[0143] Step 811: Calibrate the camera and lidar parameters based on the pixel coordinate system coordinates and the lidar coordinate system coordinates of the first feature point. Parameter calibration includes but is not limited to calibration of camera intrinsic parameters, calibration of camera extrinsic parameters, and joint calibration of the camera and lidar.

[0144] The present disclosure obtains images and point cloud data collected by cameras and lidars, and obtains the spatial transformation relationship (i.e., the first transformation relationship) between the calibration plate coordinate system and the lidar coordinate system based on the lidar coordinate system coordinates of the second feature point about the calibration plate structure outline extracted from the point cloud data and the calibration plate coordinate system coordinates of the second feature point. Then, the lidar coordinate system coordinates of the first feature point of each mark on the calibration plate are determined by the calibration plate coordinate system coordinates and the spatial transformation relationship. Combined with the pixel coordinate system coordinates of the first feature point extracted from the image, the feature points of each mark on the calibration plate can be applied to the calibration of the camera and lidar, thereby realizing rapid calibration of the internal and external parameters of the camera and lidar without manual participation and without the need to add prior data such as maps. The versatility, accuracy and robustness of the calibration are all improved.

[0145] In some embodiments, step 801 may include: acquiring an image sequence captured by a camera, performing target detection on each frame of the image sequence, and determining at least one image frame that contains a calibration plate and has a number of markers on the calibration plate that is greater than or equal to a predetermined threshold. The predetermined threshold can be set based on the detection requirements of a specific scene. For example, the predetermined threshold can be set to the number of markers required for the minimum detection requirement in the current scene. Thus, if a frame of image does not contain a calibration plate or the number of markers detected on the calibration plate in the frame of image is less than the minimum detection requirement, the frame of image can be discarded, and only images with a number of markers greater than or equal to the minimum detection requirement are retained to participate in the calibration process.

[0146] In step 803, feature point detection and object detection can be performed on each frame of the image acquired in step 801 to obtain the pixel coordinate system coordinates and corresponding marker identifiers of the first feature points of all calibration plates in the image. The marker identifier can be used to query the corresponding plate identifier from, for example, the structure object of the calibration plate. The pixel coordinate system coordinates of each first feature point are associated with the corresponding plate identifier and marker identifier in the order of the first feature point's position on the marker. In this way, the pixel coordinate system coordinates of a specific first feature point can be directly read in step 811, thereby reducing the complexity of constructing the PnP problem, reducing the amount of calculation, saving computing resources, and improving calibration efficiency.

[0147] There are many ways to extract the laser radar coordinate system coordinates of the second feature point from the point cloud data in step 805. In one implementation, an exemplary implementation process for extracting the laser radar coordinate system coordinates of the second feature point may include the following steps S11 to S13:

[0148] Step S11, identifying point cloud points corresponding to the same laser beam from the point cloud data, and the point cloud points corresponding to the same laser beam constitute a point cloud group;

[0149] Step S12, identifying edge feature points from the point cloud points;

[0150] Specifically, the points where the distance in the neighborhood of each point cloud group suddenly changes are calculated. These points are the edge feature points of the calibration plate. The edge feature points of the calibration plate can include the feature points of the outer contour of the calibration plate itself and the outer contour feature points of each calibration hole. The first reference point of the calibration plate must be included in the outer contour feature points of the calibration plate itself, and the second reference point of the calibration plate can be determined based on the outer contour feature points of the calibration hole. It is known that the point cloud point has a distance value, so the distance in the field suddenly changes, for example, the difference between the distance value of the point and the distance value of the surrounding points within a predetermined distance range can be greater than or equal to a fourth threshold.

[0151] Step S13: determining the second feature point of the calibration plate according to the edge feature point.

[0152] In some embodiments, in step 807, an association can be established between the calibration plate coordinate system coordinates and the lidar coordinate system coordinates based on the coordinates of the second feature point, and an iterative closest point (ICP) problem can be constructed to solve the first transformation relationship between the calibration plate coordinate system and the lidar coordinate system. The ICP problem is a point matching algorithm. Given the coordinates and corresponding relationships of a 3D point in two three-dimensional spatial coordinate systems, the ICP can be used to solve the transformation relationship between the two three-dimensional spatial coordinate systems.

[0153] In the embodiment of the present disclosure, the coordinates of each plate vertex in different coordinate systems can be read in sequence according to the plate identification and the serial number of each plate vertex, the coordinates of the characteristic points of each hole position in different coordinate systems can be read in sequence according to the plate identification and the serial number of each hole position, and the coordinates of the characteristic points of each mark in different coordinate systems can be read in sequence according to the plate identification and the mark identification of each mark, so that the point correspondence between the different coordinate systems can be directly obtained without executing the point correspondence determination process, which can greatly simplify the construction process of the ICP problem, reduce the computational complexity and amount of calculation, reduce the use of computing resources, reduce hardware costs and improve processing efficiency.

[0154] In an example, when solving the transformation relationship from the lidar coordinate system to the camera coordinate system, the point cloud data of the calibration plate collected by the lidar is first used to obtain the lidar coordinate system coordinates of the structural feature points on the calibration plate, and the lidar coordinate system coordinates of the second feature point on the calibration plate are stored in the order of the position of the second feature point and associated with the plate identifier of the calibration plate; then, the camera coordinate system coordinates and lidar coordinate system coordinates of all the second feature points of each calibration plate are read sequentially, and the ICP problem is directly constructed with the lidar coordinate system coordinates of all the second feature points of each calibration plate as the source point cloud and the camera coordinate system coordinates of all the second feature points of each calibration plate as the target point cloud, so as to solve the rotation transformation relationship and translation transformation relationship from the lidar coordinate system to the camera coordinate system. The rotation transformation relationship and translation transformation relationship from the lidar coordinate system to the camera coordinate system are the transformation relationship from the lidar coordinate system to the camera coordinate system.

[0155] In one example, the specific implementation process of step 807 may include the following steps S21 to S23:

[0156] Step S21, reading the coordinates of the first feature point of the calibration plate in the calibration plate coordinate system according to the plate identification of the calibration plate;

[0157] Step S22, using the calibration plate coordinate system coordinates of the first feature point as source point data, using the laser radar coordinate system coordinates of the first feature point as target point data, using the closest distance as a matching condition to determine the point correspondence and construct an ICP problem to obtain the current value of the first transformation relationship;

[0158] Specifically, the coordinates of the calibration plate coordinate system of the first feature point can be used as source data, the coordinates of the laser radar coordinate system of the first feature point can be used as target data, the point correspondence between the source data and the target data can be determined with the closest distance as the matching condition, the projection data of the source data in the laser radar coordinate system is calculated based on the preset initial value of the first transformation relationship, the residual sum between the target data and the projection data is calculated, and the least squares method is used to minimize the residual sum to optimize the first transformation relationship and thereby obtain the current value of the first transformation relationship.

[0159] In step S23, a predetermined convergence condition is determined. If so, the current value of the first transformation relationship is used as the final value. Otherwise, the process returns to step S22 and continues iterating until the convergence condition is met. The convergence condition can be pre-set as needed. The convergence condition includes, but is not limited to, reaching an upper limit on the number of iterations.

[0160] In one implementation, if the camera intrinsic parameters are unknown or the camera intrinsic parameters are not accurate enough, the calibration in step 811 can be performed by Figure 9 The process shown is implemented as follows, see Figure 9 , the process includes the following steps:

[0161] Step 901: construct a PnP (Perspective-n-Point) problem based on the laser radar coordinate system and pixel coordinate system coordinates of the first feature point, and obtain the initial values ​​of the camera's intrinsic parameters and extrinsic parameters by optimizing and solving the PnP problem;

[0162] The PnP problem is a method for solving the motion of three-dimensional to two-dimensional point pairs. When the three-dimensional coordinates of multiple feature points in the world coordinate system and the pixel coordinates of these points are known, the pose of the camera in the world coordinate system can be estimated through PnP. If the world coordinate system is selected as the LiDAR coordinate system, PnP can be used to solve the pose of the camera in the LiDAR coordinate system, that is, the pose of the camera coordinate system relative to the LiDAR coordinate system. This pose is the transformation relationship between the camera coordinate system and the LiDAR coordinate system. In the embodiment of the present disclosure, the coordinates of the marked feature points or structural feature points in different coordinate systems are sequentially read according to the information such as the plate identification and the marking identification, so that the point correspondence in different coordinate systems can be directly obtained. There is no need to perform the point correspondence determination process, which can greatly simplify the construction process of the PnP problem, reduce the computational complexity and amount of calculation, reduce the use of computing resources, reduce hardware costs, and improve processing efficiency.

[0163] The pixel coordinates of each first feature point can be read from the calibration plate's structure object in the order in which they are stored, along with the LiDAR coordinates of the first feature point, to form point pairs. Each point pair contains both the LiDAR coordinates and the pixel coordinates of the same first feature point. Point-to-point (PnP) problems can then be directly constructed using these point pairs. This significantly reduces the complexity and computational effort of constructing the PnP problem.

[0164] In one implementation, before constructing the PnP problem, the corresponding camera intrinsic parameter model and PnP problem cost function can be determined based on the camera type. For example, a first PnP problem cost function for long / short focal length cameras and a second PnP problem cost function for fisheye cameras can be preconfigured. This allows for different PnP cost function interfaces to be provided for different camera types, enabling the calibration method of the disclosed embodiments to calibrate different types of LiDARs and cameras commonly used in scenarios such as autonomous driving.

[0165] In step 901, a corresponding camera intrinsic parameter model can be selected based on the current camera type to construct the projection process from the lidar coordinate system to the pixel coordinate system. A corresponding PnP cost function can also be selected based on the current camera type to construct a corresponding PnP cost function. For example, a long / short focal length camera uses the first PnP cost function, while a fisheye camera uses the second PnP cost function.

[0166] In step 901, the theoretical value of the intrinsic parameter and the manually measured value of the extrinsic parameter can be used as initial values ​​to optimize and solve the PnP problem, and obtain the initial values ​​of the camera intrinsic parameter and the camera extrinsic parameter.

[0167] If m frames of image and m frames of point cloud data are obtained in step 801, where m is an integer greater than 1, m sets of pixel coordinate system coordinates of the first feature point can be obtained in step 803, and m sets of lidar coordinate system coordinates of the first feature point can be obtained in step 809. In this case, the theoretical value of the intrinsic parameter and the manually measured value of the extrinsic parameter can be used as the initial value, and a PnP problem can be constructed based on the m sets of pixel coordinate system coordinates and the m sets of lidar coordinate system coordinates of the first feature point, respectively. By optimizing and solving the PnP problem, m first intrinsic parameters and m first extrinsic parameters of the camera are obtained. Then, the second intrinsic parameter of the camera is determined (e.g., by taking an average or weighted average) based on the m first intrinsic parameters, and the second intrinsic parameter is used as the initial value of the camera intrinsic parameter, while the m first extrinsic parameters are used as the initial value of the camera extrinsic parameter.

[0168] Step 903: calibrate the camera and the lidar based on the initial values ​​of the camera's internal parameters and external parameters.

[0169] In one specific implementation, after step 901 or during step 903, the method may further include optimizing the initial camera intrinsic parameter values ​​using the coordinates of the first feature point in the calibration plate coordinate system and the pixel coordinate system. Specifically, a Point-to-Point (PnP) problem may be constructed based on the coordinates of the first feature point in the calibration plate coordinate system and the pixel coordinate system, and the PnP problem may be optimized and solved based on the initial camera intrinsic parameter values ​​to obtain optimized camera intrinsic parameter values. In one example, the initial camera intrinsic parameter values ​​may be the second intrinsic parameter described above.

[0170] Constructing a PnP problem based on the calibration plate coordinate system coordinates and the pixel coordinate system coordinates of the first feature point may include: constructing m PnP problems based on m groups of pixel coordinate system coordinates of the first feature point and the calibration plate coordinate system coordinates of the first feature point, optimizing and solving the m PnP problems based on the initial values ​​of the camera's intrinsic parameters to obtain m first intrinsic parameter optimization values ​​of the camera, and then determining the second intrinsic parameter optimization value of the camera by averaging or weighted averaging the m first intrinsic parameter optimization values.

[0171] For each set of pixel coordinate system coordinates, the PnP problem can be constructed in the following way: according to the plate identification and marking identification of the calibration plate, the calibration plate coordinate system coordinates and the pixel coordinate system coordinates of each first feature point are read in sequence according to the position order of the first feature point, and the calibration plate coordinate system coordinates and the pixel coordinate system coordinates corresponding to the same first feature point are combined to form a point pair. Based on the point pair, the camera intrinsic parameter model corresponding to the current camera and the PnP problem cost function corresponding to the current camera, the PnP problem is constructed according to the existing formula.

[0172] In one example, the exemplary implementation process of calibrating the camera and the lidar based on the initial values ​​of the camera's internal parameters and external parameters in step 903 is shown in Figure 10 , which may include the following steps:

[0173] Step 1001: Construct a PnP problem based on the coordinates of the calibration plate coordinate system and the pixel coordinate system of the first feature point, and optimize and solve the PnP problem based on the initial value of the camera's intrinsic parameter to obtain a second transformation relationship between the calibration plate coordinate system and the camera coordinate system. Here, the initial value of the camera's intrinsic parameter can be, but is not limited to, the theoretical value of the camera's intrinsic parameter, the second intrinsic parameter mentioned above, or others.

[0174] Step 1003, determining the camera coordinate system coordinates of the second feature point according to the calibration plate coordinate system coordinates of the second feature point and the second transformation relationship;

[0175] Step 1005: construct an ICP problem based on the camera coordinate system coordinates and the lidar coordinate system coordinates of the second feature point, optimize and solve the ICP problem based on the initial values ​​of the camera's extrinsic parameters, and obtain a third transformation relationship between the camera coordinate system and the lidar coordinate system.

[0176] For each set of camera coordinate system coordinates of the second feature point, the ICP problem can be constructed in the following way: the lidar coordinate system coordinates of each second feature point are read in sequence according to the plate identification and marking identification of the calibration plate in the order of the position of the second feature point, the lidar coordinate system coordinates and the camera coordinate system coordinates corresponding to the same first feature point are combined to form a point pair, and the ICP problem is constructed based on the point pair and the initial value of the camera extrinsic parameter obtained in step 901 and optimized and solved to obtain the third transformation relationship between the camera coordinate system and the lidar coordinate system.

[0177] In one implementation, the calibration of step 811 can be performed by Figure 11 The process shown is implemented. Figure 11 As shown, the process may include the following steps:

[0178] Step 1101, obtaining a second transformation relationship between the calibration plate coordinate system and the camera coordinate system according to the calibration plate coordinate system coordinates and the pixel coordinate system coordinates of the first feature point;

[0179] A Point-n-Pair (PnP) problem is constructed based on the coordinates of the calibration plate and pixel coordinates of the first feature point. Based on the camera intrinsic parameters, which remain unchanged, the PnP problem is optimized to obtain the second transformation relationship between the calibration plate and camera coordinate systems. Since the camera intrinsic parameters are known and relatively accurate, they can be kept unchanged during the PnP problem solution, and only the second transformation relationship between the calibration plate and camera coordinate systems can be optimized until convergence conditions are met.

[0180] Step 1103, obtaining the camera coordinate system coordinates of the second feature point according to the calibration plate coordinate system coordinates of the second feature point and the second transformation relationship;

[0181] Step 1105 , obtaining a third transformation relationship between the camera coordinate system and the lidar coordinate system according to the camera coordinate system coordinates and the lidar coordinate system coordinates of the second feature point.

[0182] If m groups of camera coordinate system coordinates and m groups of lidar coordinate system coordinates of the second feature point are obtained before, an ICP problem can be constructed based on each group of camera coordinate system coordinates of the second feature point and its corresponding group of lidar coordinate system coordinates, and the ICP problem is optimized and solved to obtain the third transformation relationship between the camera coordinate system and the lidar coordinate system. If the third transformation matrix between m camera coordinate systems and the lidar coordinate system is obtained, its average value, weighted average value or other value can be taken as the final value of the third transformation matrix between the camera coordinate system and the lidar coordinate system.

[0183] Therefore, the external parameter calibration of the camera and lidar can be achieved by using the coordinates of the calibration plate coordinate system.

[0184] The foregoing description of this specification describes specific embodiments. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in an order different from that described in the embodiments and still achieve the desired results. Furthermore, the processes depicted in the accompanying drawings do not necessarily require the specific order shown or the sequential order to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0185] Each embodiment in this specification is described in a progressive manner. The same or similar parts between the embodiments can be referred to each other. Each embodiment focuses on the differences from other embodiments. In particular, for system or system embodiments, since they are basically similar to method embodiments, the description is relatively simple. For relevant parts, refer to the partial description of the method embodiment. The system and system embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. A person of ordinary skill in the art can understand and implement it without expending creative work.

[0186] In addition, an embodiment of the present disclosure further provides a computer-readable storage medium on which a computer program is stored. The program includes instructions, and when the instructions are executed by one or more processors of a computing device, the steps of any one of the method embodiments described above are executed.

[0187] The calibration apparatus provided in the embodiment of the present disclosure may be implemented in the form of a computing device. Figure 12 FIG. 1 shows a block diagram of a computing device according to an embodiment of the present disclosure. Figure 12 As shown in , the computing device includes one or more processors 1201, and also includes a memory 1202 for storing one or more programs, which are executed by the one or more processors 1201 to implement the method flow shown in the above embodiments of the present disclosure and / or the program units corresponding to each unit in the device.

[0188] The various components are interconnected using different buses and can be installed on a common motherboard or in other ways as needed. The processor 1201 can process instructions executed within the electronic device, including instructions stored in or on a memory to display graphical information of a user interface on an external input / output device (such as a display device coupled to an interface). In other embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. The processor 1201 may include one or more single-core processors or multi-core processors. The processor 1201 may include a combination of any general-purpose processor or a dedicated processor (such as an image processor, an application processor, a baseband processor, etc.).

[0189] The memory 1202 is a computer-readable storage medium provided by the present disclosure, which can be used to store non-transient software programs, non-transient computer executable programs and units, such as program instructions / units corresponding to the above-mentioned methods in the embodiments of the present disclosure. The processor 1201 executes the non-transient software programs, instructions and units stored in the memory 1202, thereby executing the above-mentioned method embodiments. Figure 2 The program, instructions and units corresponding to the calibration method shown. The computing device may also include: an input device 1203 and an output device 1204. The processor 1201, the memory 1202, the input device 1203 and the output device 1204 may be connected via a bus or other means. Figure 12The example of the connection via bus is taken. The input device 1203 can receive input digital or character information, and generate signal input related to the user settings and function control of the calibration device, such as a touch screen, a keypad, a mouse, a trackpad, a touchpad, an indicator stick, one or more mouse buttons, a trackball, a joystick and other input devices. The output device 1204 may include a display device, an auxiliary lighting device (e.g., an LED) and a tactile feedback device (e.g., a vibration motor), etc. The display device may include, but is not limited to, a liquid crystal display (LCD), a light emitting diode (LED) display and a plasma display. In some embodiments, the display device may be a touch screen.

[0190] The above-mentioned program (also referred to as software, software application, or code) includes machine instructions for a programmable processor, and these computing programs can be implemented using object-oriented programming languages, assembly or machine languages. As time goes by and technology develops, the meaning of media becomes more and more extensive, and the dissemination path of computer programs is no longer limited to tangible media, but can also be downloaded directly from the Internet. Any combination of one or more computer-readable storage media can be used. Computer-readable storage media can be used, but not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or components, or any combination of the above. More specific examples (non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memories (RAM), read-only memories (ROM), erasable programmable read-only memories (EPROM or flash memory), optical fibers, portable compact disk read-only memories (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the above. In this document, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by an instruction execution system, device or device or used in combination with it.

[0191] A1. A calibration plate determination method, comprising: acquiring point cloud data collected by a point cloud acquisition device; identifying a target point cloud cluster that meets the size constraints of a calibration plate from the point cloud data, the calibration plate having at least one hole; determining multiple target hole positions from the target point cloud cluster based on attribute information of the calibration plate; determining whether each target hole position has a hole based on the number of point cloud points within each target hole position; determining a binary code for each target hole position based on whether each target hole position has a hole; and determining the plate code corresponding to the target point cloud cluster based on the binary code for each target hole position. A4. The method as described in A1, wherein identifying a target point cloud cluster that meets the size constraints of the calibration plate from the point cloud data comprises: extracting edge feature points from the point cloud data; obtaining at least one candidate point cloud cluster by clustering the edge feature points; determining a minimum bounding box for each candidate point cloud cluster; and selecting a target point cloud cluster from the at least one candidate point cloud cluster whose minimum bounding box meets the size constraints of the calibration plate. A5. The method as described in A4, wherein extracting edge feature points from point cloud data comprises: identifying multiple point cloud points corresponding to the same laser beam from the point cloud data; calculating the distance between each point cloud point in the multiple point cloud points and the other point cloud points based on the point cloud data; and determining the corresponding point cloud point as an edge feature point in response to the distance being greater than or equal to a predetermined threshold. A6. The method as described in A5, wherein calculating the distance between each point cloud point in the multiple point cloud points and the other point cloud points based on the point cloud data comprises: for any point cloud point in the multiple point cloud points, calculating the average of the distances between the point cloud point and a predetermined number of adjacent point cloud points. A7. The method as described in A5, wherein identifying point cloud points corresponding to the same laser beam from the point cloud data comprises at least one of the following: identifying point cloud points corresponding to the same laser beam based on the laser beam identifier of each point cloud point; calculating the angle value of the laser beam corresponding to each point cloud point based on the coordinate values ​​of each point cloud point, and determining the point cloud points corresponding to the same laser beam based on the angle value corresponding to each point cloud point. A8. The method as described in A1, wherein determining multiple target hole positions from the target point cloud cluster based on the attribute information of the calibration plate includes: determining at least one target vertex from the target point cloud cluster; and determining multiple target hole positions based on the attribute information of the calibration plate and the at least one target vertex.

[0192] A9. The method as described in A8, wherein determining at least one target vertex from the target point cloud cluster includes: traversing the point cloud points in the target point cloud cluster, taking each traversed point cloud point as a candidate vertex, and determining a candidate area of ​​the same size as the calibration plate based on the candidate vertex; and determining at least one target vertex from the traversed candidate vertices based on the number of point cloud points within the candidate area corresponding to each candidate vertex. A10. The method as described in A9, wherein determining at least one target vertex from the traversed candidate vertices based on the number of point cloud points within the candidate area corresponding to each candidate vertex includes: determining the candidate area with the largest number of point cloud points as the calibration plate area; and taking the candidate vertex corresponding to the calibration plate area as the target vertex. A11. A method as described in A8, wherein the attribute information includes a first positional relationship between a first hole position and a second hole position in the calibration plate, a second positional relationship between the second hole position and a plate vertex, and size information of the first hole position, and determining the multiple target hole positions based on the attribute information and the at least one target vertex includes: determining a reference point of the second hole position in the point cloud data based on the second positional relationship and the target vertex; and determining the multiple target hole positions based on the first positional relationship, the reference point, and the size information of the first hole position.

[0193] A12. The method as described in A11, wherein the attribute information further includes size information of the second hole location, and determining the reference point of the second hole location in the point cloud data based on the second positional relationship and the target vertex comprises: determining an initial point based on the second positional relationship and the target vertex; determining a first region based on the size information of the second hole location with the initial point as the center; traversing candidate points within the first region, determining a second region based on the size information of the second hole location with each traversed candidate point as the center; and determining the reference point from the traversed candidate points based on the number of point cloud points within each second region. A13. The method as described in A12, wherein the first region is the minimum circumscribed polygon of the first hole location, and the second region is the maximum inscribed polygon of the first hole location. A14. The method as described in A12, wherein determining the reference point from the traversed candidate points based on the number of point cloud points within each second region comprises: selecting the candidate point corresponding to the second region containing the least number of point cloud points as the reference point.

[0194] A15. The method as described in A12 further includes: determining a calibration plate area from the point cloud data; dividing the calibration plate area into a two-dimensional grid; wherein the candidate points are grid vertices within the first area. A16. The method as described in A15, wherein determining the reference point from the traversed candidate points based on the number of point cloud points within each second area includes: for each grid vertex, generating a label for the grid vertex based on whether there is a point cloud point within a predetermined range of the grid vertex, wherein the label includes a first label and a second label; determining the number of grid vertices marked with the first label within each second area; and determining the reference point from the traversed grid vertices based on the number of grid vertices corresponding to each second area. A17. The method as described in A1, wherein determining whether there is a hole at each target hole position based on the number of point cloud points within each target hole position includes: determining the maximum inscribed polygon corresponding to the target hole position; and determining that there is a hole at the target hole position in response to the number of point cloud points within the maximum inscribed polygon being less than or equal to a threshold.

[0195] B18. A method for determining a calibration plate, comprising: obtaining the number and distribution of hole positions of the calibration plate, wherein each hole position has a corresponding binary code based on whether there is a hole at the hole position, and each calibration plate has a corresponding plate code based on the binary codes of the multiple hole positions; determining multiple plate codes based on the number and distribution of hole positions, wherein the Hamming distance between any two plate codes in the multiple plate codes is greater than or equal to 1; and generating a calibration plate pattern corresponding to each plate code in the multiple plate codes. B19. The method as described in B18, wherein the plate code of each calibration plate is determined according to the position sequence of each hole position and the binary code of each hole position, each calibration plate has a hole at at least one hole position, and the hole position distribution includes the distribution of multiple hole positions in the calibration plate. B20. The method as described in B19, further comprising: storing the multiple plate codes as a plate code library.

[0196] C21: A calibration plate system comprising multiple calibration plates, each calibration plate having multiple hole positions, each calibration plate having a hole at at least one hole position, each hole position having a corresponding binary code based on whether the hole position has a hole, each calibration plate having a corresponding plate code based on the binary codes of the multiple hole positions, and the Hamming distance between the plate codes of any two calibration plates in the multiple calibration plates being greater than or equal to 1. C22. The system as described in C21, wherein the hole positions of the calibration plates are identical, and the plate code of each calibration plate is determined based on the positional sequence of the hole positions in the calibration plate and the binary codes of the hole positions.

[0197] The technical solutions provided by the present disclosure are introduced in detail above. Specific examples are used herein to illustrate the principles and implementation methods of the present disclosure. The description of the above embodiments is only used to help understand the method and core ideas of the present disclosure. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scopes based on the ideas of the present disclosure. In summary, the content of this specification should not be understood as a limitation on the present disclosure. The above description is only a preferred embodiment of the present disclosure and is not intended to limit the present disclosure. Any modifications, equivalent substitutions, etc. made within the spirit and principles of the present disclosure should be included in the scope of protection of the present disclosure.

Claims

1. A method for determining a calibration plate, comprising: Acquiring point cloud data collected by a point cloud collection device; identifying a target point cloud cluster from the point cloud data that meets the size constraints of a calibration plate, the calibration plate having at least one hole; determining a plurality of target hole positions from the target point cloud cluster according to the attribute information of the calibration plate; Determine whether there is a hole at each target hole position based on the number of point cloud points in each target hole position; Determine the binary code of each target hole position according to whether there is a hole at each target hole position; as well as The plate code corresponding to the target point cloud cluster is determined according to the binary code of each target hole position.

2. The method of claim 1, further comprising: Whether the target point cloud cluster is an invalid point cloud cluster is determined according to whether the plate code exists in a pre-stored plate code library, and the plate code library includes the plate code of the calibration plate.

3. The method of claim 1 , further comprising: Calculating the distance between any two of the plurality of target hole positions; By comparing the distance value between each two target hole positions with the true value, it is determined whether the target point cloud cluster is an invalid point cloud cluster.

4. The method according to claim 1, wherein Identifying a target point cloud cluster that meets the size constraint of the calibration plate from the point cloud data includes: Extracting edge feature points from the point cloud data; By clustering the edge feature points, at least one candidate point cloud cluster is obtained; Determine the minimum bounding box for each candidate point cloud cluster; and A target point cloud cluster whose minimum bounding box meets the size constraint of the calibration plate is selected from the at least one candidate point cloud cluster.

5. The method according to claim 4, wherein: Extracting edge feature points from the point cloud data includes: identifying, from the point cloud data, a plurality of point cloud points corresponding to a same laser beam; Calculating the distance between each point cloud point and other point cloud points in the plurality of point cloud points according to the point cloud data; In response to the distance being greater than or equal to a predetermined threshold, the corresponding point cloud point is determined to be an edge feature point.

6. A method for determining a calibration plate, comprising: Obtain the number and distribution of well positions of the calibration plate, wherein each well position has a corresponding binary code according to whether there is a hole at the well position, and each calibration plate has a corresponding plate code according to the binary codes of the plurality of well positions; Determining a plurality of plate codes according to the hole number and hole distribution, wherein a Hamming distance between any two plate codes in the plurality of plate codes is greater than or equal to 1; and A calibration plate pattern corresponding to each plate code of the plurality of plate codes is generated.

7. A calibration plate system, the calibration system comprising multiple calibration plates, each calibration plate having multiple hole positions, each calibration plate having a hole at at least one hole position, each hole position having a corresponding binary code according to whether there is a hole at the hole position, each calibration plate having a corresponding plate code according to the binary codes of the multiple hole positions, and the Hamming distance between the plate codes of any two calibration plates among the multiple calibration plates is greater than or equal to 1.

8. A computing device comprising: one or more processors, and A memory storing a program, the program comprising instructions which, when executed by the processor, cause the processor to perform the method according to any one of claims 1 to 6.

9. A computer-readable storage medium storing a program, the program comprising instructions which, when executed by one or more processors of a computing device, cause the computing device to perform the method according to any one of claims 1 to 6.

10. A computer program product comprising a computer program, wherein when the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images