Calibration plate position acquisition method, joint calibration method, calibration plate and equipment

Abstract

The application discloses a method for acquiring a calibration board position, a joint calibration method, a calibration board and equipment. The method comprises the following steps: acquiring scene point cloud data; wherein the scene point cloud data is obtained by scanning a calibration scene provided with a calibration board by an optical scanning device, and at least one hollow part is arranged on the calibration board; based on the scene point cloud data, the position information of each hollow part is determined; and the target point cloud coordinates corresponding to the target position in the calibration board are determined by using the position information of each hollow part. In the foregoing manner, the point cloud coordinates corresponding to the position in the calibration board can be accurately determined.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of calibration, in particular to a method for acquiring a position of a calibration board, a joint calibration method, a calibration board and equipment. BACKGROUND

[0002] At present, in the fields of mobile robots, autonomous driving, assisted driving and environment perception, a single sensor cannot meet the perception requirements of complex environments, and multi-sensor fusion algorithms have become the mainstream algorithms, which fuse information collected by multiple sensors to complement each other. Camera and optical scanning device play an increasingly important role in these fields, and the premise of their fusion is to jointly calibrate them.

[0003] Among them, the joint calibration of the two is mainly realized by using a calibration board. However, the determination of the point cloud coordinates of the points on the calibration board mainly depends on the point cloud shape of the calibration board itself, and the edge of the calibration board may be distorted, which may result in low accuracy and insufficient precision of the point cloud coordinates of the points on the calibration board. SUMMARY

[0004] The technical problem solved by the present application is to provide a method for acquiring a position of a calibration board, a joint calibration method, a calibration board and equipment, which can accurately determine the point cloud coordinates corresponding to the positions in the calibration board.

[0005] To solve the above technical problems, one technical solution adopted by the present application is to provide a method for acquiring a position of a calibration board, which comprises: acquiring scene point cloud data; wherein the scene point cloud data is obtained by scanning a calibration scene provided with a calibration board by an optical scanning device, and the calibration board is provided with at least one hollow part; determining the position information of each hollow part based on the scene point cloud data; and determining the target point cloud coordinates corresponding to a target position in the calibration board by using the position information of each hollow part.

[0006] To solve the above technical problems, another technical solution adopted by the present application is to provide a joint calibration method, which comprises: acquiring scene image and scene point cloud data; wherein the scene image and the scene point cloud data are respectively acquired by a camera and an optical scanning device based on a calibration scene comprising a calibration board, and the calibration board is provided with a positioning pattern and a hollow part; obtaining target image coordinates corresponding to a plurality of target positions based on the positioning pattern of the calibration board in the scene image, wherein the plurality of target positions are positions on the calibration board; and obtaining target point cloud coordinates corresponding to the plurality of target positions based on the scene point cloud data and the hollow part of the calibration board; and combining the target image coordinates and the target point cloud coordinates corresponding to the plurality of target positions to obtain calibration parameters.

[0007] To solve the above technical problems, another technical scheme adopted by the present application is to provide a calibration board, the calibration board is provided with at least one positioning pattern and at least one hollow part, wherein the positioning pattern is used to determine the image coordinates of the target position of the calibration board in the camera coordinate system, and the hollow part is used to determine the point cloud coordinates of the target position in the scanning coordinate system of the optical scanning device.

[0008] To solve the above technical problems, another technical scheme adopted by the present application is to provide an electronic device, the electronic device comprises a memory and a processor, the memory stores program instructions, and the processor is used to execute the program instructions to realize the above-mentioned calibration board position acquisition method or the above-mentioned joint calibration method.

[0009] To solve the above technical problems, another technical scheme adopted by the present application is to provide a computer readable storage medium, the computer readable storage medium is used to store program instructions, the program instructions can be executed to realize the above-mentioned calibration board position acquisition method or the above-mentioned joint calibration method.

[0010] The above technical scheme uses the position information of the hollow part to determine the target point cloud data corresponding to the target position in the calibration board. Therefore, the target point cloud coordinates corresponding to the target position in the calibration board are determined by using the hollow part provided on the calibration board. Since the hollow part is hollow on the calibration board, the position of the hollow part can be accurately determined. Since the hollow part and the target position in the calibration board are relatively fixed, the target point cloud coordinates corresponding to the target position in the calibration board can be accurately determined based on the hollow part on the calibration board. BRIEF DESCRIPTION OF DRAWINGS

[0011] Figure 1 is a structural schematic diagram of an embodiment of the calibration board provided by the present application;

[0012] Figure 2 is a flowchart of an embodiment of the calibration board position acquisition method provided by the present application;

[0013] Figure 3 is a schematic diagram of an embodiment of the calibration scene provided by the present application;

[0014] Figure 4 is Figure 2 is a flowchart of an embodiment of the step S22 shown in the figure;

[0015] Figure 5 is Figure 4 is a flowchart of an embodiment of the step S41 shown in the figure;

[0016] Figure 6 is a schematic diagram of an embodiment of the initial point cloud data of the calibration board provided by the present application;

[0017] Figure 7 is a flowchart of an embodiment of step S42 shown in FIG. 4; Figure 4

[0018] Figure 8 is a flowchart of an embodiment of step S71 shown in FIG. 7; Figure 7

[0019] Figure 9 is a schematic diagram of an embodiment of the calibration coordinate system provided by the present application;

[0020] Figure 10 is a flowchart of an embodiment of step S72 shown in FIG. 7; Figure 7

[0021] Figure 11 is a schematic diagram of another embodiment of the calibration coordinate system provided by the present application;

[0022] Figure 12 is a flowchart of an embodiment of step S23 shown in FIG. 2; Figure 2

[0023] Figure 13 is a flowchart of an embodiment of the joint calibration method provided by the present application;

[0024] Figure 14 is a flowchart of an embodiment of step S1302 shown in FIG. 13; Figure 13

[0025] is a schematic diagram of an embodiment of the calibration region provided by the present application; Figure 15

[0026] Figure 16 is a flowchart of an embodiment of step S1401 shown in FIG. 14; Figure 14

[0027] Figure 17 is a schematic diagram of an embodiment of the feature code provided by the present application;

[0028] Figure 18 is a flowchart of an embodiment of step S1402 shown in FIG. 14; Figure 14

[0029] Figure 19 is a schematic diagram of an embodiment of the chessboard corner point provided by the present application;

[0030] Figure 20 is a structural schematic diagram of an embodiment of the electronic device provided by the present application;

[0031] Figure 21 is a structural schematic diagram of an embodiment of the computer readable storage medium provided by the present application. DETAILED DESCRIPTION ​​​​​​​

[0032] The scheme of the embodiments of the present application will be described in detail below with reference to the accompanying drawings of the specification.

[0033] In the following description, for the purpose of explanation and not limitation, specific details are set forth, such as particular system configurations, interfaces, techniques, in order to provide a thorough understanding of the present application.

[0034] The term "and / or" herein merely describes an associated relationship between associated objects, which means that there can be three relationships, for example, A and / or B can mean that A exists alone, A and B exist together, and B exists alone. In addition, the character " / " herein generally represents an "or" relationship between the front and rear associated objects. In addition, "multiple" herein means two or more. In addition, the term "at least one" herein means any one of a plurality or any combination of at least two of a plurality, for example, including at least one of A, B, and C can mean including any one or more elements selected from the set consisting of A, B, and C.

[0035] Please refer to Figure 1 , Figure 1 is a structural schematic diagram of an embodiment of the calibration plate provided by the present application. The present application provides a calibration plate 100, which is provided with at least one positioning pattern 10 and at least one hollow part 20, wherein the positioning pattern 10 is used to determine the image coordinates of the target position of the calibration plate 100 in the camera coordinate system, and the hollow part 20 is used to determine the point cloud coordinates of the target position in the scanning coordinate system of the optical scanning device. In the fields of mobile robots, autonomous driving, assisted driving, and environmental perception, a single sensor is difficult to meet the perception needs of complex environments, and multi-sensor fusion algorithms have become the mainstream algorithms, and multi-sensor fusion algorithms are used to complement each other. Among them, the camera and the optical scanning device play an increasingly important role in these fields, and the premise of their fusion use is to jointly calibrate them. Since the positioning pattern 10 provided on the calibration plate 100 can be used to determine the image coordinates of the target position of the calibration plate 100 in the camera coordinate system and the hollow part 20 provided on the calibration plate 100 can be used to determine the point cloud coordinates of the target position in the scanning coordinate system of the optical scanning device, the calibration plate 100 provided by the present application can be used to realize the joint calibration of the camera and the optical scanning device, that is, the calibration plate 100 provided by the present application can be used to calculate the conversion parameters between the camera coordinate system corresponding to the camera and the scanning coordinate system corresponding to the optical scanning device, establish the corresponding relationship between the point cloud scanned by the optical scanning device and the image photographed by the camera, and realize the coordinate unification in space.

[0036] The shape, size, number of the hollow parts 20 provided on the calibration plate 100 and the specific position of the hollow parts 20 provided on the calibration plate 100 are not limited, and can be specifically set according to actual use needs. For example, the shape of the hollow parts 20 provided on the calibration plate 100 is circular, square or rectangular, etc.; the number of the hollow parts 20 is one, two or three, etc.; and the hollow parts 20 are provided on the diagonal lines of the calibration plate 100. In addition, the type, number, size of the positioning pattern 10 provided on the calibration plate 100 and the specific position of the positioning pattern 10 provided on the calibration plate 100 are not limited, and can be specifically set according to actual use needs. For example, the positioning pattern 10 is a feature code or a chessboard, or the positioning pattern 10 simultaneously includes a feature code and a chessboard; when the positioning pattern is a feature code, the feature code includes two and is provided on the diagonal lines of the calibration plate 100, or when the positioning pattern 10 is a chessboard, the chessboard is one and is centrally provided on the calibration plate 100.

[0037] Please continue to refer to Figure 1 In a specific embodiment, the calibration plate 100 includes two hollow circles, and the positioning pattern 10 is a feature code and a chessboard. Among them, the chessboard is 4x4 in size, and the center point of the chessboard is located at the center point of the calibration plate 100; the two hollow circles are located at the upper right corner and the lower left corner of the calibration plate 100, and are symmetrically distributed about the center of the calibration plate 100, and the midpoint of the line connecting the centers of the two hollow circles is the center point of the calibration plate 100; the two feature codes are located at the upper left corner and the lower right corner of the calibration plate 100, and each feature code of the calibration plate 100 is different and has a unique feature ID number.

[0038] Please refer to Figure 2 , Figure 2 is a flowchart of an embodiment of a method for acquiring a position of a calibration plate provided by the present application. It should be noted that the present embodiment is not limited to the order of the flowchart shown in Figure 2 . As shown in Figure 2 , the present embodiment includes:

[0039] Step S21: acquiring scene point cloud data.

[0040] The method of the present embodiment is used to determine the target point cloud coordinates corresponding to the target position in the calibration plate, which can improve the accuracy of determining the target point cloud coordinates corresponding to the target position.

[0041] In an embodiment, the scene point cloud data can be obtained from local storage or cloud storage. In other embodiments, the scene point cloud data can also be obtained by real-time scanning of the calibration scene by an optical scanning device.

[0042] As shown in Figure 3 , Figure 3is a schematic diagram of an embodiment of a calibration scene provided by the present application, and the scene point cloud data described herein is obtained by scanning a calibration scene provided with a calibration board by an optical scanning device. The type of the optical scanning device is not limited, and can be specifically set according to actual use needs. For example, the optical scanning device can be a laser radar, a depth camera, a binocular camera, or the like.

[0043] As shown in Figure 1 , the calibration board is provided with at least one hollow part. By providing the hollow part on the calibration board, the position of the hollow part can be accurately determined subsequently by using the hollowing feature of the hollow part on the calibration board. Since the hollow part and the target position in the calibration board are relatively fixed, the point cloud coordinates corresponding to the target position in the calibration board can be accurately determined based on the hollow part of the calibration board subsequently, without relying on the point cloud shape of the calibration board itself. The point cloud coordinate calculation error caused by the edge distortion of the calibration board can be avoided, and the accuracy of the determined point cloud coordinates corresponding to the target position is improved. The number of hollow parts provided on the calibration board, the shape of the hollow part, and the specific setting position of the hollow part on the calibration board are not limited, and can be specifically set according to actual use needs. For example, as shown in Figure 1 , two hollow parts are provided on the calibration board, the shape of the hollow part is circular, and the two hollow parts are provided on the diagonal line of the calibration board, that is, the two hollow parts are diagonally arranged on the calibration board.

[0044] Step S22: determining the position information of each hollow part based on the scene point cloud data.

[0045] In this embodiment, the position information of each hollow part is determined based on the scene point cloud data. That is, the position of each hollow part on the calibration board is determined through the obtained scene point cloud data. Since the hollow part is hollowed on the calibration board, the accuracy of the determined position information of each hollow part is high. Since the hollow part and the target position in the calibration board are relatively fixed, the point cloud coordinates corresponding to the target position in the calibration board can be accurately determined based on the hollow part of the calibration board subsequently.

[0046] In an embodiment, the position information of each hollow part can be directly determined based on the scene point cloud data. In order to avoid the influence of other factors existing in the calibration scene on the determination of the position information of each hollow part and improve the accuracy of the determined position information of each hollow part, in other embodiments, the calibration board point cloud data corresponding to the calibration board is extracted from the scene point cloud data, and then the position information of each hollow part is determined based on the calibration board point cloud data corresponding to the calibration board.

[0047] In an embodiment, the position information of each hollow part can be determined by the distribution of each point in the point cloud data in the calibration coordinate system. Of course, in other embodiments, the position information of the hollow part can also be determined based on the scene point cloud data by other ways.

[0048] Step S23: Determine the target point cloud coordinate corresponding to the target position in the calibration plate by using the position information of each hollow part.

[0049] In this embodiment, the position information of each hollow part is used to determine the target point cloud coordinate corresponding to the target position in the calibration plate, that is, the target point cloud data corresponding to the target position in the calibration plate is determined by using the hollow part provided on the calibration plate. Compared with the way of determining the target point cloud coordinate corresponding to the target position in the calibration plate by using the shape of the calibration plate itself which may exist edge distortion, the present application determines the target point cloud coordinate corresponding to the target position in the calibration plate by using the hollow part provided on the calibration plate. Since the hollow part is hollow on the calibration plate, the position of the hollow part can be accurately determined. Since the hollow part and the target position in the calibration plate are relatively fixed, the target point cloud coordinate corresponding to the target position in the calibration plate can be accurately determined based on the hollow part on the calibration plate.

[0050] In an embodiment, the target position is the center position, that is, the target point cloud coordinate corresponding to the center of the calibration plate is determined by using the position information of each hollow part. Of course, in other embodiments, the target position can also be any position other than the center position in the calibration plate, which is not limited here.

[0051] Since the size of the calibration plate is fixed and the position of the hollow part on the calibration plate is also fixed, the actual position relationship between the target position in the calibration plate and each hollow part on the calibration plate can be known. Therefore, in an embodiment, the target point cloud data corresponding to the target position in the calibration plate can be determined by using the position information of the hollow part and the actual position relationship between the target position in the calibration plate and each hollow part on the calibration plate. Of course, in other embodiments, the target point cloud data corresponding to the target position in the calibration plate can also be determined based on the position information of the hollow part by other ways, which is not limited here.

[0052] In the above embodiment, the position information of the hollow part is used to determine the target point cloud data corresponding to the target position in the calibration plate. Therefore, the target point cloud coordinate corresponding to the target position in the calibration plate is determined by using the hollow part provided on the calibration plate. Since the hollow part is hollow on the calibration plate, the position of the hollow part can be accurately determined. Since the hollow part and the target position in the calibration plate are relatively fixed, the target point cloud coordinate corresponding to the target position in the calibration plate can be accurately determined based on the hollow part on the calibration plate.

[0053] Please refer to Figure 4 , Figure 4 is Figure 2 the flowchart of an embodiment of step S22 shown in FIG. 8. It should be noted that this embodiment does not necessarily have the same result as the above embodiment.Figure 4 The flow sequence shown is limited. As Figure 4 In the embodiment, the position of the hollow part is determined based on the distribution of each point in the point cloud data corresponding to the calibration plate in the calibration coordinate system, and specifically includes:

[0054] Step S41: Extract the calibration plate point cloud data corresponding to the calibration plate from the scene point cloud data.

[0055] In the embodiment, the calibration plate point cloud data corresponding to the calibration plate is extracted from the scene point cloud data, so that the position of each hollow part is determined based on the calibration plate point cloud data corresponding to the calibration plate in the subsequent step. On the one hand, the amount of point cloud data is reduced, and the efficiency of determining the position information of the hollow part is improved. On the other hand, the influence of other factors existing in the calibration scene on the determination of the position information of each hollow part is reduced, and the accuracy of the determined position information of each hollow part is improved.

[0056] In an embodiment, as Figure 5 shown, Figure 5 is Figure 4 The flowchart of one embodiment of step S41 shown is a plane fitting of the point cloud data corresponding to the calibration plate to obtain the calibration plate point cloud data corresponding to the calibration plate, so as to accurately extract the calibration plate point cloud data corresponding to the calibration plate. Specifically, the following sub-steps are included:

[0057] Step S51: Determine the estimated point cloud coordinates of the target position of the calibration plate based on the positional relationship between the calibration plate and the optical scanning device.

[0058] In the embodiment, the estimated point cloud coordinates of the target position of the calibration plate are determined based on the positional relationship between the calibration plate and the optical scanning device. Specifically, as Figure 3 shown, since the positional relationship between the calibration plate and the optical scanning device can be obtained when the calibration scene is arranged, and the position of the calibration plate relative to the optical scanning device will not change after the calibration scene is arranged, the point cloud coordinates of the target position of the calibration plate can be approximately determined according to the positional relationship between the calibration plate and the optical scanning device, i.e. the estimated point cloud coordinates of the target position of the calibration plate.

[0059] Step S52: Extract the point cloud data within a preset range centered at the estimated point cloud coordinates of the calibration plate from the scene point cloud data as the initial point cloud data corresponding to the calibration plate.

[0060] In this embodiment, the point cloud data in a preset range centered on the estimated point cloud coordinates corresponding to the calibration board is extracted from the scene point cloud data as the initial point cloud data corresponding to the calibration board. In an embodiment, the point cloud data in a preset range centered on the estimated point cloud coordinates corresponding to the calibration board can be extracted from the scene point cloud data by using a pass-through filtering algorithm. Of course, in other embodiments, the point cloud data in a preset range centered on the estimated point cloud coordinates corresponding to the calibration board can also be extracted from the scene point cloud data by using other related algorithms or models, which are not limited here.

[0061] wherein, since the actual size of the calibration board is known and will not change, the preset range can be determined according to the actual size of the calibration board and the range of the point cloud data corresponding to the calibration board to be extracted. For example, the target position is the center of the calibration board, the point cloud coordinates of the center of the calibration board are (x, y, z), and the actual size of the calibration board is a square of 5*5. As shown in FIG. 6, Figure 6 Figure 6 is a schematic diagram of an embodiment of the initial point cloud data of the calibration board provided by the present application. If there is no other object around the calibration board in the calibration scene and the initial point cloud data of the entire calibration board needs to be extracted, in order to ensure that the initial point cloud data of the entire calibration board can be extracted, the points in the range of ±2.7 centered on the estimated point cloud coordinates (x, y, z) of the center of the calibration board are taken as the initial point cloud data corresponding to the calibration board. If the distortion problem at the edge of the calibration board is considered, the point cloud data smaller than the actual area size of the calibration board is extracted as the initial point cloud data corresponding to the calibration board, for example, the points in the range of ±2.3 centered on the estimated point cloud coordinates (x, y, z) of the center of the calibration board are taken as the initial point cloud data corresponding to the calibration board.

[0062] Step S53: performing plane fitting on the initial point cloud data, and projecting the fitting result onto a plane to obtain calibration board point cloud data.

[0063] In this embodiment, the initial point cloud data is subjected to plane fitting, and the fitting result is projected onto a plane to obtain calibration board point cloud data. It should be noted that the calibration board point cloud data is located on the same plane, and the small plane formed by the calibration board point cloud data on the plane is approximately the calibration board.

[0064] In an embodiment, the RANdom SAmple Consensus (RANSAC) algorithm can be used to perform plane fitting on the initial point cloud data corresponding to the calibration board. Of course, in other embodiments, the plane fitting algorithm, the Iterative Closest Point (ICP) algorithm, etc. can also be used to perform plane fitting on the initial point cloud data corresponding to the calibration board, which is not limited here. ​

[0065] Step S42: Determine the first calibration coordinates of the preset positions of the hollow parts in the calibration coordinate system according to the distribution of the points in the calibration plate point cloud data in the calibration coordinate system.

[0066] In this embodiment, the first calibration coordinates of the preset positions of the hollow parts in the calibration coordinate system are determined according to the distribution of the points in the calibration plate point cloud data in the calibration coordinate system. That is, the first calibration coordinates of the preset positions of the hollow parts in the calibration coordinate system are taken as the position information of the corresponding hollow parts.

[0067] The preset positions of the hollow parts are not limited, and can be specifically set according to actual use needs. For example, the preset positions of the hollow parts are the center points of the hollow parts, etc.

[0068] Since the hollow parts are hollow on the calibration plate, that is, there is no object at the specific positions corresponding to the hollow parts on the calibration plate, the optical scanning device cannot scan the specific point information at the hollow parts of the calibration plate when performing optical scanning on the calibration plate. Therefore, in an embodiment, as shown in Figure 7 , Figure 7 is Figure 4 The flowchart of step S42 in one embodiment is shown in

[0069] Step S71: Obtain the calibration coordinates of the points in the calibration plate point cloud data in the calibration coordinate system.

[0070] In this embodiment, the calibration coordinates of the points in the calibration plate point cloud data in the calibration coordinate system are obtained. In an embodiment, the calibration coordinates of the points in the calibration plate point cloud data in the calibration coordinate system can be obtained from local storage or cloud storage. In other embodiments, the calibration coordinates of the points in the calibration plate point cloud data in the calibration coordinate system can also be generated in real time.

[0071] In an embodiment, as shown in Figure 8 , Figure 8 is Figure 7 The flowchart of step S71 in one embodiment is shown in

[0072] Step S81: Determine the point cloud coordinates of the reference positions in the calibration plate by using the calibration plate point cloud data.

[0073] In this embodiment, the point cloud coordinates of a reference position on the calibration board are determined using the point cloud data of the calibration board. The reference position on the calibration board can be any position on the calibration board; for example, the reference position on the calibration board can be the center position of the calibration board.

[0074] In one embodiment, the reference position is the center position of the calibration board. In this case, the average coordinates of all points in the calibration board's point cloud data are obtained and used as the point cloud coordinates of the reference position in the calibration board. That is, all points in the calibration board's point cloud data are traversed, and the average coordinates of all points in the calibration board's point cloud data are calculated to obtain the average coordinates. These average coordinates approximate the center coordinates of the calibration board, so they are used as the point cloud coordinates of the center position of the calibration board.

[0075] Step S82: Using the point cloud coordinates of the reference position, establish a calibration coordinate system and determine the transformation relationship between the scanning coordinate system and the calibration coordinate system corresponding to the optical scanning device.

[0076] In this embodiment, a calibration coordinate system is established using the point cloud coordinates of the reference position, and the transformation relationship between the scanning coordinate system and the calibration coordinate system corresponding to the optical scanning device is determined.

[0077] In one embodiment, a calibration coordinate system is established with the point cloud coordinates of the reference position in the calibration plate as the origin. Of course, in other embodiments, another position can be determined based on the reference position in the calibration plate, and a calibration coordinate system can be established with that other position as the origin; no specific limitation is made here.

[0078] In one specific implementation, such as Figure 9 As shown, Figure 9 This is a schematic diagram of an embodiment of the calibration coordinate system provided in this application. The reference position is the center position of the calibration plate, specifically... Then, a calibration coordinate system is established with the center of the calibration plate as the origin, the horizontal direction as the positive x-axis, the vertical direction as the positive y-axis, and the z-axis perpendicular to the XOY plane.

[0079] In one embodiment, the coordinate transformation matrix between the scanning coordinate system and the calibration coordinate system is determined using the point cloud coordinates of the reference position on the calibration plate and the plane normal vector of the plane corresponding to the reference position. The determined coordinate transformation matrix is ​​the transformation relationship between the scanning coordinate system and the calibration coordinate system corresponding to the optical scanning device. Of course, in other embodiments, other methods can be used to determine the transformation relationship between the scanning coordinate system and the calibration coordinate system corresponding to the optical scanning device, and no specific limitation is made here.

[0080] Step S83: Using the transformation relationship between the scanning coordinate system and the calibration coordinate system, obtain the calibration coordinates of each point in the calibration board point cloud data in the calibration coordinate system.

[0081] In this embodiment, the conversion relationship between the scanning coordinate system and the calibration coordinate system is used to obtain the calibration coordinates of each point in the calibration coordinate system in the calibration plate point cloud data. Specifically, as shown in Figure 9 The scanning coordinate system of the optical scanning device is different from the calibration coordinate system, so it is necessary to convert the point cloud coordinates of each point in the calibration plate point cloud data to the calibration coordinate system for representation, so as to obtain the calibration coordinates of each point in the calibration coordinate system in the calibration plate point cloud data.

[0082] For example, taking the calibration coordinate system as shown in Figure 9 ; the coordinate conversion matrix between the scanning coordinate system and the calibration coordinate system is determined as Tl2b, then the calibration coordinates of each point (X 1i ,Y 1i ,Z 1i ) in the calibration plate point cloud data in the calibration coordinate system are (X 2i ,Y 2i ,0).

[0083] Step S72: determining the corresponding search area for each hollow part.

[0084] In this embodiment, the corresponding search area is determined for each hollow part. Since the positions of the hollow parts on the calibration plate are different, the corresponding search area is determined for each hollow part to accurately determine the first calibration coordinates of the preset positions of the hollow parts in the calibration coordinate system.

[0085] In an embodiment, as shown in Figure 10 , the step S72 of determining the corresponding search area for each hollow part specifically includes the following sub-steps: Figure 10 Figure 7 As shown in the flowchart of one embodiment of the step S72, the step of determining the corresponding search area for each hollow part specifically includes the following sub-steps:

[0086] Step S1001: obtaining the second calibration coordinates of the preset positions of the hollow parts by using the calibration coordinates of the reference positions in the calibration coordinate system and the positional relationship between the reference positions and the preset positions of the hollow parts.

[0087] ​In this embodiment, the second calibration coordinates of the preset positions of the hollow parts are obtained by using the calibration coordinates of the reference positions in the calibration board in the calibration coordinate system and the positional relationship between the reference positions and the preset positions of the hollow parts. Since the calibration coordinate system is constructed based on the plane formed by the calibration board point cloud data corresponding to the calibration board, and the plane formed by the calibration board point cloud data corresponding to the calibration board is approximately the plane of the calibration board, the positional relationship between the preset positions of the hollow parts and the reference positions in the calibration board on the calibration board is equivalent to the positional relationship between the second calibration coordinates of the preset positions of the hollow parts and the calibration coordinates of the reference positions in the calibration coordinate system. Therefore, after obtaining the positional relationship between the preset positions of the hollow parts and the target positions in the calibration board and the calibration coordinates of the reference positions in the calibration coordinate system, the second calibration coordinates of the preset positions of the hollow parts in the calibration coordinate system can be calculated.

[0088] It should be noted that the second calibration coordinates of the preset positions of the hollow parts can be regarded as the estimated calibration coordinates obtained by approximately estimating the positional relationship between the reference positions and the preset positions of the hollow parts on the calibration board.

[0089] For example, as shown in FIG. 6, the calibration coordinate system is established with the center of the calibration board as the reference position, the calibration board includes two hollow parts, and the preset positions of the hollow parts are the centers of the hollow parts. Figure 11 Figure 11 As shown in FIG. 6, the calibration coordinate system is established with the center of the calibration board as the reference position, the calibration board includes two hollow parts, and the preset positions of the hollow parts are the centers of the hollow parts.Since the calibration board is fixed, the relative positions of the centers of the two hollow parts and the center of the calibration board are unchanged, and since the calibration coordinate system is constructed based on the plane formed by the calibration board point cloud data corresponding to the calibration board, and the plane formed by the calibration board point cloud data corresponding to the calibration board is approximately the plane of the calibration board, the positional relationship between the preset positions of the hollow parts and the reference positions in the calibration board on the calibration board is equivalent to the positional relationship between the second calibration coordinates of the preset positions of the hollow parts and the calibration coordinates of the reference positions in the calibration coordinate system. Therefore, after determining the relative positional relationship between the centers of the two hollow parts and the center of the calibration board on the calibration board, the second calibration coordinates of the centers of the two hollow parts can be calculated. For example, the second calibration coordinates of the centers of the two hollow parts are respectively

[0090] In addition, as shown in FIG. 6, the points corresponding to the calibration board are all located in the boundary, and there is no point corresponding to the calibration board outside the boundary. Figure 11

[0091] Step S1002: For each second calibration coordinate, the area of the preset shape containing the second calibration coordinate is taken as the search area corresponding to the hollow part.

[0092] ​In this embodiment, for each second calibration coordinate, a preset shape region containing the second calibration coordinate is used as the search region corresponding to the hollow part. In one embodiment, as shown in Figure 11 FIG. 6, a preset shape region centered on the second calibration coordinate is used as the search region corresponding to the hollow part. Of course, in other embodiments, a preset shape region centered on another calibration coordinate can be used as the search region corresponding to the hollow part, which is not limited here.

[0093] The preset shape is not limited, and can be set according to actual needs. For example, the preset shape can be a square, a rectangle, a circle, etc.

[0094] For example, as shown in Figure 11 FIG. 6, it is assumed that the calibration plate includes two hollow parts, and the preset shape is a square; a square region centered on the second calibration coordinate corresponding to the hollow part A is used as the search region A corresponding to the hollow part A, and a square region centered on the second calibration coordinate corresponding to the hollow part B is used as the search region B corresponding to the hollow part B.

[0095] Step S73: For each hollow part, the calibration coordinates of each point in the calibration coordinate system are used to determine each search point corresponding to the hollow part and obtain the number of associated points of each search point.

[0096] In this embodiment, for each hollow part, the calibration coordinates of each point in the calibration coordinate system are used to determine each search point corresponding to the hollow part and obtain the number of associated points of each search point; the search point is a point located in the search region, and the number of associated points of the search point is the number of points contained in a set region defined by the search point, the shape and size of the set region are the same as those of the hollow part, and the search point is located at a preset position in the set region. That is, for each hollow part, the points in the search region are respectively used as search points, and the number of points in the set region corresponding to each search point is determined, which is the number of associated points of the search point corresponding to the set region.

[0097] For example, as shown in Figure 11 FIG. 6, it is assumed that the hollow part is a hollow circle; for the search point a1 corresponding to the hollow part A, the number of points contained in the set region a corresponding to the search point a1 is 4, so the number of associated points of the search point a1 is 4; for the search point b1 corresponding to the hollow part B, the number of points contained in the set region β corresponding to the search point b1 is 3, so the number of associated points of the search point b1 is 3.

[0098] Step S74: Based on the number of associated points of each search point corresponding to the hollow part, the calibration coordinate of a search point is selected as the first calibration coordinate of the preset position of the hollow part.

[0099] In this embodiment, the first calibration coordinate of the preset position of the hollow part is selected as the calibration coordinate of a search point based on the number of associated points of each search point corresponding to the hollow part. Since the hollow part is hollow on the calibration board, i.e., there is no object at the specific position on the calibration board corresponding to the hollow part, the optical scanning device is used to perform optical scanning on the calibration board, and the specific point information cannot be scanned at the hollow part of the calibration board. Therefore, in an embodiment, the calibration coordinate of the search point with zero associated points is selected as the first calibration coordinate of the preset position of the hollow part. Since there may be errors in the actual confirmation of the number of associated points of each search point, in other embodiments, the calibration coordinate of the search point with the least number of associated points is selected as the first calibration coordinate of the preset position of the hollow part.

[0100] Referring to Figure 12 , Figure 12 is Figure 2 a flowchart of an embodiment of step S23. It should be noted that the sequence of the flowchart shown in Figure 12 is not limited to the embodiment. As shown in Figure 12 , the embodiment includes:

[0101] Step S1201: obtaining the target calibration coordinate of the target position in the calibration coordinate system of the calibration board based on the first calibration coordinate of the preset position of each hollow part in the calibration coordinate system and the positional relationship between the preset position of each hollow part and the target position in the calibration board.

[0102] In this embodiment, the target calibration coordinate of the target position in the calibration coordinate system of the calibration board is obtained based on the first calibration coordinate of the preset position of each hollow part in the calibration coordinate system and the positional relationship between the preset position of each hollow part and the target position in the calibration board. That is, the first calibration coordinate of the preset position of each hollow part in the calibration coordinate system is used as the positional information of each hollow part to determine the target point cloud coordinate corresponding to the target position in the calibration board. Since the calibration coordinate system is constructed based on the plane formed by the calibration board point cloud data corresponding to the calibration board, and the plane formed by the calibration board point cloud data corresponding to the calibration board is approximately the plane of the calibration board, the positional relationship between the preset position of each hollow part and the target position in the calibration board is equivalent to the positional relationship between the first calibration coordinate of the preset position of each hollow part and the target calibration coordinate of the target position in the calibration coordinate system; therefore, after the first calibration coordinate of the preset position of each hollow part in the calibration coordinate system and the positional relationship between the preset position of each hollow part and the target position in the calibration board are determined, the target calibration coordinate of the target position in the calibration coordinate system of the calibration board can be calculated.

[0103] For example, taking the case that the calibration plate includes two hollow circles, the two hollow circles are arranged at the upper left corner and the lower right corner of the calibration plate and are symmetrically distributed about the center of the calibration plate, the midpoint of the line connecting the centers of the two hollow circles is the center of the calibration plate, and the preset positions of the hollow circles are the centers of the hollow circles, and the target position is the center of the calibration plate: since the target position of the calibration plate is the center of the calibration plate and the midpoint of the line connecting the centers of the two hollow circles is the center of the calibration plate, the positional relationship between the preset positions of the hollow parts and the target position in the calibration plate is that the midpoint of the line connecting the preset positions of the two hollow parts is the target position in the calibration plate; therefore, equivalently in the calibration coordinate system, the midpoint of the line connecting the first calibration coordinates corresponding to the preset positions of the two hollow parts is the target calibration coordinate of the target position of the calibration plate in the calibration coordinate system, that is, the calibration coordinate corresponding to the midpoint of the line connecting the centers of the two hollow circles is the target calibration coordinate of the center of the calibration plate in the calibration coordinate system; therefore, the calibration coordinate corresponding to the midpoint of the line connecting the centers of the two hollow circles can be calculated based on the first calibration coordinates corresponding to the centers of the two hollow circles, and the calibration coordinate corresponding to the midpoint of the line connecting the centers of the two hollow circles is taken as the target calibration coordinate of the center of the calibration plate in the calibration coordinate system, for example, the calibration coordinate corresponding to the midpoint of the line connecting the centers of the two hollow circles is Therefore, the target calibration coordinate of the center of the calibration plate in the calibration coordinate system is

[0104] For another example, taking the case that the calibration plate includes one hollow circle, the target position of the calibration plate is position A in the calibration plate, and the preset position of the hollow part is the center of the hollow part: since the positional relationship between the preset position of the hollow part and the target position in the calibration plate is that position A and the center of the hollow part are both on the diagonal line of the calibration plate (position A is closer to the center of the calibration plate) and position A is 0.3 cm away from the center of the hollow part; therefore, equivalently in the calibration coordinate system, the first calibration coordinates corresponding to the preset position of the hollow part are both located on the 45° angle line in the calibration coordinate system, and the calibration coordinate corresponding to the point 0.3 cm away from the first calibration coordinates corresponding to the preset position of the hollow part is the target calibration coordinate of position A in the calibration coordinate system.

[0105] Step S1202: obtaining the target point cloud coordinate corresponding to the target position in the calibration plate by using the conversion relationship between the calibration coordinate system and the scanning coordinate system of the optical scanning device and the target calibration coordinate.

[0106] In this embodiment, the target point cloud coordinates corresponding to the target position in the calibration plate are obtained by utilizing the transformation relationship between the calibration coordinate system and the scanning coordinate system of the optical scanning device, as well as the target calibration coordinates. Since the above steps yield the point cloud coordinates of the target position in the calibration plate within the calibration coordinate system, and the calibration coordinate system is an auxiliary coordinate system designed to facilitate the calculation of the target point cloud coordinates corresponding to the target position in the calibration plate, the point cloud coordinates of the target position in this coordinate system cannot be directly used as the target point cloud coordinates of the target position. Therefore, it is necessary to utilize the transformation relationship between the calibration coordinate system and the scanning coordinate system of the optical scanning device to transform the target calibration coordinates of the target position in the calibration coordinate system to obtain the target point cloud coordinates corresponding to the target position in the calibration plate.

[0107] For example, suppose the transformation relationship between the calibration coordinate system and the scanning coordinate system of the optical scanning device is represented by the coordinate transformation matrix T. l2b And the target position in the calibration plate, in the calibration coordinate system, is the target calibration coordinate. Therefore, it is possible to utilize the coordinate transformation matrix T l2b The target position on the calibration plate is located in the target calibration coordinate system. Transforming to the scanning coordinate system, we obtain the target point cloud coordinates corresponding to the target position in the calibration plate.

[0108] The specific process of obtaining the transformation relationship between the calibration coordinate system and the scanning coordinate system of the optical scanning device, as well as the construction of the calibration coordinate system, can be found in steps S81-S82 above, and will not be repeated here.

[0109] Please see Figure 13 , Figure 13 This is a schematic flowchart of an embodiment of the joint calibration method provided in this application. It should be noted that if substantially the same result is obtained, this embodiment does not necessarily reflect that result. Figure 13 The illustrated process sequence is limited. For example... Figure 13 As shown, this embodiment includes:

[0110] Step S1301: Acquire scene images and scene point cloud data.

[0111] The optical scanning device has the advantage of accurately reflecting the spatial three-dimensional information of the environment, but it is deficient in detail description. The camera cannot reflect the spatial three-dimensional information of the environment, but it has outstanding effects in detail and color description. Therefore, the data of the optical scanning device and the camera are usually fused to ensure that the spatial three-dimensional information of the environment can be reflected while the details and colors can also be reflected. The premise of fusing the optical scanning device and the camera is to jointly calibrate the two to realize the coordinate unification in space. It should be noted that the joint calibration of the two mainly represents the establishment of the corresponding relationship between each point in the point cloud scanned by the optical scanning device and each pixel in the image photographed by the camera to obtain the spatial conversion relationship of the coordinate systems of the two, that is, to determine the calibration parameters. Therefore, the method of the embodiment is used to determine the calibration parameters between the camera and the optical scanning device to realize the joint calibration of the optical scanning device and the camera.

[0112] In the embodiment, scene image and scene point cloud data are acquired. The scene image and the scene point cloud data are respectively acquired by the camera and the optical scanning device based on a calibration scene including a calibration board.

[0113] In an embodiment, the calibration scene includes a plurality of calibration boards, and the scene image and the scene point cloud data are respectively acquired by the camera and the optical scanning device based on the calibration scene including the plurality of calibration boards. Each calibration board is located in the field of view of the camera and the scanning range of the optical scanning device, and each calibration board does not block each other in the field of view and the scanning range. Of course, in other embodiments, the calibration board scene can also include only one calibration board, which is not limited here.

[0114] In a specific embodiment, as shown in Figure 3 The calibration scene includes nine calibration boards. The calibration boards are placed in the field of view of the camera and the scanning range of the optical scanning device, and are placed at three different distances from the camera and the optical scanning device, i.e., far, middle and near. Three calibration boards are placed at each distance, and each calibration board is kept a certain distance apart so that the calibration boards are respectively located at the left edge, the center and the right edge of the image on the camera image. The calibration boards at the three distances are different in height and do not block each other on the camera image and the scanning range of the optical scanning device. In addition, it should be noted that, in order to improve the accuracy of the joint calibration of the camera and the optical scanning device, no other objects are arranged in a certain area around each calibration board in the calibration scene.

[0115] In addition, the calibration board is provided with a positioning pattern and a hollow part. For the related description of the calibration board, please refer to the above embodiments, which will not be repeated here.

[0116] In an embodiment, the scene image and the scene point cloud data can be obtained from local storage or cloud storage. In other embodiments, the scene point cloud data can be obtained by scanning the calibration scene in real time by an optical scanning device, and the scene image can be obtained by capturing the calibration scene in real time by a camera.

[0117] Step S1302: Obtain target image coordinates corresponding to the target positions based on the positioning pattern of the calibration board in the scene image.

[0118] In the embodiment, the target image coordinates corresponding to the target positions are obtained based on the positioning pattern of the calibration board in the scene image, wherein the target positions are positions on the calibration board. That is, the positioning pattern on the calibration board is set to determine the target image coordinates of the target positions on the calibration board, that is, the target image coordinates of the target positions on the calibration board can be determined by using the positioning pattern on the calibration board.

[0119] In an embodiment, the calibration scene includes only one calibration board, and the target positions are all located on the calibration board. In other embodiments, the calibration scene includes multiple calibration boards, and the target positions are positions on the multiple calibration boards. In a specific embodiment, the number of target positions is at least eight, and the number of calibration boards is the same as the number of target positions. The target positions are not limited, and can be set according to actual use needs. For example, the target position of the calibration board is the center position of the calibration board.

[0120] For example, the calibration scene includes nine calibration boards, and each target position is the center point of a calibration board. Based on the positioning pattern of the calibration board in the scene image, the target image coordinates corresponding to the nine target positions are (μ1, v1)~(μ9, v9), respectively.

[0121] In an embodiment, the positioning pattern of the calibration board includes a first positioning pattern and a second positioning pattern. The target image coordinates corresponding to the target positions can be obtained in combination with the first positioning pattern and the second positioning pattern. Of course, in other embodiments, the positioning pattern of the calibration board can only include the first positioning pattern or the second positioning pattern. In this case, the target image coordinates corresponding to the target positions can be obtained only according to the first positioning pattern or the second positioning pattern, which is not limited herein.

[0122] Since the scene image is obtained by a camera, the scene image may have distortion problems. Therefore, in order to improve the accuracy of the subsequent determined calibration parameters, in an embodiment, before obtaining the target image coordinates corresponding to the target positions based on the positioning pattern of the calibration board in the scene image, the scene image is processed for distortion by using the intrinsic parameters and distortion parameters of the camera to correct the distortion problems of the scene image.

[0123] Step S1303: obtaining target point cloud coordinates corresponding to the target positions based on the scene point cloud data and the hollow part of the calibration board.

[0124] In this embodiment, target point cloud coordinates corresponding to the target positions are obtained based on the scene point cloud data and the hollow part of the calibration board. In an embodiment, the target point cloud coordinates corresponding to the target positions can be obtained based on the scene point cloud data and the hollow part of the calibration board by using the related method mentioned in the method for obtaining the calibration board provided in this application, which will not be described here.

[0125] Step S1304: obtaining the calibration parameters by combining the target image coordinates and the target point cloud coordinates corresponding to the target positions.

[0126] In this embodiment, the calibration parameters are obtained by combining the target image coordinates and the target point cloud coordinates corresponding to the target positions. Due to the hollowing feature of the hollow part on the calibration board, the position of the hollow part can be accurately determined, and since the hollow part and the target positions in the calibration board are relatively fixed, the target point cloud coordinates corresponding to the target positions in the calibration board can be accurately determined based on the hollow part on the calibration board. Since the target point cloud coordinates of the target positions in the joint calibration method provided in this application are determined based on the hollow part on the calibration board, the accuracy of the target point cloud coordinates of the target positions determined is high, and thus the accuracy of the calibration parameters obtained by combining the target image coordinates and the target point cloud coordinates corresponding to the target positions is high, which improves the accuracy and precision of the joint calibration. In addition, in the actual joint calibration process, the calibration board does not need to be moved, and the degree of automation is high, the calibration process is convenient, the calibration efficiency is high, and the requirement for the calibration scene is low.

[0127] Specifically, taking a calibration board including 9 target positions, target image coordinates corresponding to each target position being (μ1, v1)~(μ9, v9) respectively, and target point cloud coordinates corresponding to each target position being (X1, Y1, Z1)~(X9, Y9, Z9) respectively as an example. The image coordinates and the point cloud coordinates of the points on the calibration board have the following relationship:

[0128] (u, v) = K(R(X, Y, Z) + t)

[0129] wherein K is the camera intrinsic parameter, which is a known quantity; R is a 3x3 rotation matrix, and t is a 3x1 rotation vector, and R and t are the calibration parameters.

[0130] Further, the target image coordinates and the target point cloud coordinates of the 9 target positions are substituted into the above relationship to obtain the calibration parameters.

[0131] ​In the above embodiments, due to the hollowing feature of the hollow part on the calibration board, the position of the hollow part can be accurately determined, and since the hollow part and the target position in the calibration board are relatively fixed, the target position in the calibration board corresponding to the target point cloud coordinate can be accurately determined based on the hollow part on the calibration board. Since the target point cloud coordinates of the plurality of target positions in the joint calibration method provided by the application are determined based on the hollow part on the calibration board, the accuracy of the target point cloud coordinates corresponding to the plurality of target positions is high, so that the accuracy of the calibration parameters obtained by combining the target image coordinates and the target point cloud coordinates corresponding to the plurality of target positions is high, thereby improving the accuracy and precision of the joint calibration. In addition, in the actual joint calibration process, the calibration board does not need to be moved, and the degree of automation is high, the calibration process is convenient, the calibration efficiency is high, and the requirement for the calibration scene is low.

[0132] Please refer to Figure 14 , Figure 14 is Figure 13 a flowchart of an embodiment of step S1302. It should be noted that if there is substantially the same result, the present embodiment is not limited to the order of the flowchart shown in Figure 14 . As shown in Figure 14 , in the present embodiment, the positioning pattern of the calibration board includes a first positioning pattern and a second positioning pattern, and the target image coordinates corresponding to a plurality of target positions are obtained in combination with the first positioning pattern and the second positioning pattern, specifically including:

[0133] The calibration scene includes at least one calibration board, and the positioning pattern of each calibration board includes a first positioning pattern and a second positioning pattern. The set position of each second positioning pattern corresponds to a target position, and the second positioning pattern is located within the preset range area of the first positioning pattern in the calibration board. Wherein, the number and specific position of the first positioning pattern and the second positioning pattern included in each calibration board are not limited; for example, the number of first positioning patterns is two and is distributed on the diagonal line of the calibration board, and the second positioning pattern and the hollow part are located within the quadrilateral area defined by the two first positioning patterns in the calibration board, and each first positioning pattern has one corner as the corner of the quadrilateral area.

[0134] In an embodiment, the first positioning pattern is a feature code; wherein the type of the feature code is not limited, for example, the feature code is an Aruco code. Of course, in other embodiments, the first positioning pattern can also be a chessboard or other types of patterns, etc., which are not limited here. In an embodiment, the second positioning pattern is a chessboard.

[0135] Step S1401: based on the image position of the first positioning pattern in the scene image, extracting the calibration region corresponding to each calibration board from the scene image.

[0136] For the case that the calibration scene only includes one calibration board, in order to reduce the influence of other factors existing outside the calibration board area on determining the target image coordinates corresponding to the target position, or for the case that the calibration scene includes two or more calibration boards, in order to accurately determine the target point cloud coordinates corresponding to the target position on each calibration board, in the embodiment, as shown in Figure 15 Figure 15 is a schematic diagram of an embodiment of the calibration area provided by the present application, based on the image position of the first positioning pattern in the scene image, the calibration area corresponding to each calibration board is extracted from the scene image; that is, in the calibration area, only the image area corresponding to one calibration board is included.

[0137] In an embodiment, as shown in Figure 16 Figure 16 is a flowchart of an embodiment of the step S1401 shown in Figure 14 The step S1601: detecting the first positioning pattern on the scene image to obtain the detection result of each first positioning pattern.

[0138] In the embodiment, the first positioning pattern is detected on the scene image to obtain the detection result of each first positioning pattern. Wherein, the algorithm for detecting the first positioning pattern is not limited, and can be specifically set according to actual use needs.

[0139] Wherein, the detection result of the first positioning pattern includes the identification and the corner point coordinates of the first positioning pattern.

[0140] In an embodiment, as shown in

[0141] Figure 17 is a schematic diagram of an embodiment of the feature code provided by the present application, the first positioning pattern is a feature code, at this time, the feature code detection is performed by using the Opencv related API to obtain the ID number (identification) and the corner point coordinates of the four corner points corresponding to each feature code. Figure 17 The step S1602: determining at least two first positioning patterns located on the same calibration board by using the identification, and obtaining the calibration area corresponding to the calibration board by using the corner point coordinates of the at least two first positioning patterns located on the same calibration board.

[0142]

[0143] ​​​​In the embodiment, the at least two first positioning patterns located on the same calibration board are determined by the identification, and the calibration region corresponding to the calibration board is obtained by using the corner point coordinates of the at least two first positioning patterns located on the same calibration board. Specifically, since the identification of each first positioning pattern is unique, it can be determined by the identification of the first positioning pattern which block of the calibration board it belongs to; in addition, the region between the minimum horizontal and vertical index μ min , the minimum vertical coordinate v min and the maximum horizontal and vertical index μ max , the maximum vertical coordinate v max is taken as the calibration region corresponding to each calibration board, that is, the region between (μ min , v min ) and (μ max , v max ) is taken as the calibration region corresponding to the calibration board.

[0144] Step S1402: For each calibration region, the image coordinates of the set position of the second positioning pattern in the calibration region are obtained as the target image coordinates of the corresponding target position.

[0145] In the embodiment, for each calibration region, the image coordinates of the set position of the second positioning pattern in the calibration region are obtained as the target image coordinates of the corresponding target position. In an embodiment, the set position of the second positioning pattern is the center position of the second positioning pattern, and at this time, for each calibration region, the image coordinates of the center position of the second positioning pattern in the calibration region are obtained as the target image coordinates of the corresponding target position.

[0146] In an embodiment, as shown in Figure 18 , Figure 18 is Figure 14 the flowchart of an embodiment of step S1402, and obtaining the image coordinates of the set position of the second positioning pattern in the calibration region as the target image coordinates of the corresponding target position specifically includes the following sub-steps:

[0147] Step S1801: For each calibration region corresponding to the calibration board, corner point detection of the second positioning pattern is performed on the calibration region to obtain at least one corner point coordinate of the second positioning pattern of the calibration board.

[0148] In the embodiment, for each calibration region corresponding to the calibration board, corner point detection of the second positioning pattern is performed on the calibration region to obtain at least one corner point coordinate of the second positioning pattern of the calibration board. The algorithm for detecting the second positioning pattern is not limited, and can be specifically set according to actual use needs.

[0149] In an embodiment, the second positioning pattern is a chessboard, and the chessboard corner point detection is performed by using the Opencv related API to obtain the corner point coordinates of each corner point of the chessboard.

[0150] For example, as shown in Figure 19 , Figure 19 is a schematic diagram of an embodiment of the chessboard corner point provided by the present application, taking the second positioning pattern on the calibration board as a 4x4 chessboard for example; the chessboard corner point detection is performed by using the Opencv related API to obtain the corner point coordinates of the nine corner points of the chessboard.

[0151] Step S1802: obtaining the image coordinates of the set position of the second positioning pattern by using at least one corner point coordinate of the second positioning pattern of the calibration board and the position relationship between the set position of the second positioning pattern and the corner point, and taking the image coordinates of the set position of the second positioning pattern as the target image coordinates of the corresponding target position.

[0152] In the embodiment, the image coordinates of the set position of the second positioning pattern are obtained by using at least one corner point coordinate of the second positioning pattern of the calibration board and the position relationship between the set position of the second positioning pattern and the corner point, and taking the image coordinates of the set position of the second positioning pattern as the target image coordinates of the corresponding target position.

[0153] In an embodiment, the set position of the second positioning pattern is the center position of the second positioning pattern, and in this case, the corner point coordinate corresponding to the center position of the calibration board among the corner point coordinates of the second positioning pattern of the calibration board is taken as the target image coordinate of the corresponding target position. In a specific embodiment, as shown in Figure 19 , the second positioning pattern is arranged centrally relative to the calibration board, and in this case, the corner point coordinates of the corner point located in the middle position are sorted, and the corner point coordinates of the corner point located in the middle position are taken as the target image coordinates of the corresponding target position.

[0154] It should be noted that for the calibration region corresponding to each calibration board, steps S1801-S1802 need to be performed respectively.

[0155] Please refer to Figure 20 , Figure 20 is a structural schematic diagram of an embodiment of the electronic device provided by the present application. The electronic device 200 includes a memory 2001 and a processor 2002 coupled to each other, and the processor 2002 is used to execute the program instructions stored in the memory 2001 to implement the steps of any of the above-mentioned calibration board position acquisition method or joint calibration method embodiments. In a specific implementation scenario, the electronic device 200 can include but is not limited to: a microcomputer, a server, in addition, the electronic device 200 can also include a notebook computer, a tablet computer and other mobile devices, which are not limited here.

[0156] Specifically, the processor 2002 is configured to control itself and the memory 2001 to implement the steps of any of the above-described calibration board position acquisition method or joint calibration method embodiments. The processor 2002 can also be referred to as a CPU (Central Processing Unit). The processor 2002 can be an integrated circuit chip having a processing capability of signals. The processor 2002 can also be a general processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. The general processor can be a microprocessor or the processor can also be any conventional processor or the like. In addition, the processor 2002 can be jointly implemented by integrated circuit chips.

[0157] Please refer to Figure 21 , Figure 21 is a structural schematic diagram of an embodiment of the computer readable storage medium provided by the present application. The computer readable storage medium 210 of the embodiment of the present application stores program instructions 2101, which, when executed, implement the method provided by any embodiment of the calibration board position acquisition method or joint calibration method and any non-conflicting combination. Among them, the program instructions 2101 can form a program file and be stored in the above computer readable storage medium 210 in the form of a software product, so that a computer device (which can be a personal computer, a server, or a network device, etc.) executes all or part of the steps of the method of each embodiment of the present application. The aforementioned computer readable storage medium 210 includes: a U disk, a mobile hard disk, a ROM (Read-Only Memory), a RAM (Random Access Memory), a magnetic disk or an optical disk, and various media that can store program codes, or a terminal device such as a computer, a server, a mobile phone, a tablet computer.

[0158] If the technical solutions of the present application involve personal information, the product applying the technical solutions of the present application has been explicitly informed of the personal information processing rules before processing the personal information, and has obtained the personal independent consent. If the technical solutions of the present application involve sensitive personal information, the product applying the technical solutions of the present application has obtained the personal independent consent before processing the sensitive personal information, and at the same time meets the requirement of "explicit consent". For example, at the personal information collection device such as camera, a clear and prominent mark is set to inform that it has entered the personal information collection range and will collect personal information. If the individual voluntarily enters the collection range, it is considered to agree to collect personal information. Or, on the device for processing personal information, the personal information processing rules are informed by using obvious marks / information, and the personal authorization is obtained by means of pop-up information or asking the individual to upload his / her personal information. The personal information processing rules can include personal information processor, personal information processing purpose, processing method and personal information type, etc.

[0159] The above is only an embodiment of the present application, and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation using the content of the present application specification and drawings, or direct or indirect application in other related technical fields, is also included in the patent protection scope of the present application.

Claims

1. A method for obtaining the position of a calibration plate, characterized in that, include: Acquire scene point cloud data; wherein, the scene point cloud data is obtained by scanning a calibration scene with a calibration plate by an optical scanning device, and the calibration plate has at least one hollow part; Based on the scene point cloud data, determine the position information of each of the hollowed-out parts; Using the position information of each of the hollowed-out portions, the target point cloud coordinates corresponding to the target position in the calibration plate are determined; The step of determining the position information of each of the hollowed-out portions based on the scene point cloud data includes: Extract the calibration board point cloud data corresponding to the calibration board from the scene point cloud data; Obtain the calibration coordinates of each point in the calibration board point cloud data in the calibration coordinate system; Determine the corresponding search area for each of the aforementioned hollowed-out portions; For each of the hollowed-out portions, using the calibration coordinates of each point in the calibration coordinate system, the corresponding search points of the hollowed-out portions are determined and the number of associated points of each search point is obtained. The search point is a point located within the search area, and the number of associated points of the search point is the number of points contained in the set area defined by the search point. The shape and size of the set area are the same as those of the hollowed-out portions, and the search point is located at a preset position within the set area. Based on the number of associated points of each search point corresponding to the hollow part, the calibration coordinates of one of the search points are selected as the first calibration coordinates of the preset position of the hollow part.

2. The method according to claim 1, characterized in that, The step of obtaining the calibration coordinates of each point in the calibration board point cloud data in the calibration coordinate system includes: The point cloud coordinates of the reference position in the calibration board are determined using the point cloud data of the calibration board. Using the point cloud coordinates of the reference position, a calibration coordinate system is established, and the transformation relationship between the scanning coordinate system corresponding to the optical scanning device and the calibration coordinate system is determined. By utilizing the transformation relationship between the scanning coordinate system and the calibration coordinate system, the calibration coordinates of each point in the calibration board point cloud data in the calibration coordinate system are obtained; The step of determining the corresponding search area for each of the hollowed-out portions includes: By using the calibration coordinates of the reference position in the calibration plate in the calibration coordinate system, and the positional relationship between the reference position and the preset position of each of the hollow parts, the second calibration coordinates of the preset position of each of the hollow parts are obtained; For each of the second calibration coordinates, the region containing the preset shape of the second calibration coordinate is taken as the search region corresponding to the hollow part of the second calibration coordinate; The step of selecting the calibration coordinates of one of the search points as the first calibration coordinates of the preset position of the hollow part based on the number of associated points of each search point corresponding to the hollow part includes: The calibration coordinates of the search point with the fewest associated points are used as the first calibration coordinates of the preset position of the hollow part.

3. The method according to claim 2, characterized in that, The step of using the region containing the preset shape of the second calibration coordinate as the search region corresponding to the hollow part corresponding to the second calibration coordinate includes: The area of ​​a preset shape centered on the second calibration coordinates is used as the search area corresponding to the hollow part, and the preset shape is a square. And / or, the reference position is the center position of the calibration board, and the step of determining the point cloud coordinates of the reference position in the calibration board using the point cloud data of the calibration board includes: The average coordinates of all points in the calibration board point cloud data are obtained and used as the point cloud coordinates of the reference position in the calibration board. The step of establishing a calibration coordinate system using the point cloud coordinates of the reference position includes: The calibration coordinate system is established using the reference position as the origin. And / or, determining the transformation relationship between the scanning coordinate system corresponding to the optical scanning device and the calibration coordinate system includes: Using the point cloud coordinates of the reference position and the plane normal vector of the plane corresponding to the reference position, the coordinate transformation matrix between the scanning coordinate system and the calibration coordinate system is determined.

4. The method according to claim 1, characterized in that, The step of extracting the calibration board point cloud data corresponding to the calibration board from the scene point cloud data includes: Based on the positional relationship between the calibration plate and the optical scanning device, the estimated point cloud coordinates of the target position of the calibration plate are determined; From the scene point cloud data, extract point cloud data within a preset range centered on the estimated point cloud coordinates corresponding to the calibration board, and use it as the initial point cloud data corresponding to the calibration board. The initial point cloud data is fitted to a plane, and the fitting result is projected onto a plane to obtain the calibration board point cloud data.

5. The method according to claim 1, characterized in that, The step of determining the target point cloud coordinates corresponding to the target position in the calibration plate using the position information of each of the hollowed-out portions includes: By using the first calibration coordinates of the preset positions of each of the hollow parts in the calibration coordinate system, and the positional relationship between the preset positions of each of the hollow parts and the target position in the calibration plate, the target calibration coordinates of the target position in the calibration plate in the calibration coordinate system are obtained. By utilizing the transformation relationship between the calibration coordinate system and the scanning coordinate system of the optical scanning device, as well as the target calibration coordinates, the target point cloud coordinates corresponding to the target position in the calibration plate are obtained.

6. The method according to claim 1, characterized in that, The hollowed-out portion has at least one of the following characteristics: it is circular in shape and there are two of them; And / or, the target location is the center location.

7. A joint calibration method, characterized in that, The method includes: Acquire scene images and scene point cloud data; wherein, the scene images and the scene point cloud data are respectively acquired by the camera and the optical scanning device based on a calibration scene including a calibration plate, and the calibration plate is provided with positioning patterns and cutouts; Based on the positioning pattern of the calibration board in the scene image, target image coordinates corresponding to several target positions are obtained, wherein the several target positions are the positions on the calibration board; and, Based on the scene point cloud data and the cutout portion of the calibration plate, the target point cloud coordinates corresponding to the plurality of target positions are obtained; wherein, the target point cloud coordinates corresponding to the target positions are obtained using the method described in any one of claims 1 to 6; By combining the target image coordinates and target point cloud coordinates corresponding to the aforementioned target locations, calibration parameters are obtained.

8. The method according to claim 7, characterized in that, The number of calibration plates is at least one, and the positioning pattern of each calibration plate includes a first positioning pattern and a second positioning pattern. The set position of each second positioning pattern corresponds to a target position, and the second positioning pattern is located within a preset range area of ​​the first positioning pattern in the calibration plate. The method of obtaining target image coordinates corresponding to several target positions based on the positioning pattern of the calibration board in the scene image includes: Based on the image position of the first positioning pattern in the scene image, the calibration area corresponding to each calibration plate is extracted from the scene image; For each calibration region, the image coordinates of the set position of the second positioning pattern in the calibration region are obtained as the target image coordinates of the corresponding target position.

9. The method according to claim 8, characterized in that, The number of the first positioning patterns is at least two, and the step of extracting the calibration region corresponding to each calibration plate from the scene image based on the image position of the first positioning patterns in the scene image includes: The scene image is subjected to detection of the first positioning pattern to obtain the detection results of each first positioning pattern; wherein, the detection results include the identifier and corner coordinates of the first positioning pattern; The identification is used to determine at least two first positioning patterns located on the same calibration plate, and the corner coordinates of the at least two first positioning patterns located on the same calibration plate are used to obtain the calibration area corresponding to the calibration plate; And / or, obtaining the image coordinates of the set position of the second positioning pattern in the calibration area as the target image coordinates of the corresponding target position includes: For each calibration plate corresponding to a calibration area, corner point detection of the second positioning pattern is performed on the calibration area to obtain the coordinates of at least one corner point of the second positioning pattern of the calibration plate. Using the coordinates of at least one corner point of the second positioning pattern on the calibration plate, and the positional relationship between the set position of the second positioning pattern and the corner point, the image coordinates of the set position of the second positioning pattern are obtained, which are then used as the target image coordinates of the corresponding target position.

10. The method according to claim 8, characterized in that, The first positioning pattern is a feature code, and the second positioning pattern is a checkerboard pattern; The set position is the center position of the second positioning pattern; There are two first positioning patterns, which are distributed on the diagonal of the calibration plate. The second positioning pattern and the hollow part are both located in the quadrilateral area defined by the two first positioning patterns in the calibration plate. Each of the two first positioning patterns has one corner as the diagonal of the quadrilateral area.

11. The method according to claim 7, characterized in that, Before obtaining the target image coordinates corresponding to several target positions based on the positioning pattern of the calibration board in the scene image, the method further includes: The scene image is distorted using the camera's intrinsic parameters and distortion parameters.

12. The method according to claim 7, characterized in that, The number of calibration plates is multiple, and the target positions are the positions on the multiple calibration plates. Each calibration plate is located within the field of view of the camera and the scanning range of the optical scanning device, and each calibration plate does not obstruct the others within the field of view and the scanning range. And / or, the number of the plurality of target positions is at least eight, the number of calibration plates is the same as the number of target positions, and each target position is the center position of a calibration plate.

13. An electronic device, characterized in that, The electronic device includes a memory and a processor. The memory stores program instructions, and the processor executes the program instructions to implement the calibration board position acquisition method as described in any one of claims 1-6 or the joint calibration method as described in any one of claims 7-12.

14. A computer-readable storage medium, characterized in that, The computer-readable storage medium is used to store program instructions that can be executed to implement the calibration plate position acquisition method as described in any one of claims 1-6 or the joint calibration method as described in any one of claims 7-12.

Images