Laser radar and camera external parameter calibration method, system and storage medium

Abstract

The present application provides a laser radar and camera extrinsic calibration method, which is applied to a rigid system of a laser radar and camera combination, and includes: obtaining a camera image of a target scene, the target scene is provided with a calibrator, and the camera image includes a calibrator image of the calibrator; obtaining the edge contour of the camera image, the edge contour includes the contour of the calibrator image; constructing the calibrator edge line based on the contour of the calibrator; obtaining the Planck coordinates of the fitted edge of the calibrator based on the edge contour and the calibrator edge line; obtaining three-dimensional point cloud data of the target scene; extracting the edge point set of the calibrator based on the three-dimensional point cloud data; obtaining the extrinsic parameters of the laser radar and camera based on the fitted edge and the edge point set. The laser radar and camera extrinsic calibration method provided by the present application is conducive to simplifying the extrinsic calibration procedure and enhancing practicality. The present application also provides a laser radar and camera extrinsic calibration system and storage medium.

Description

Technical Field

[0001] The present application relates to the field of three-dimensional scanning, and in particular to a laser radar and camera extrinsic parameter calibration method, a laser radar and camera extrinsic parameter calibration system and a storage medium. Background Art

[0002] Among common three-dimensional scanning devices, laser radar can obtain high-precision three-dimensional information and is not easily affected by lighting and weather conditions, but the three-dimensional information obtained by laser radar lacks color information. Cameras can provide high-resolution color information, but are easily affected by lighting and weather conditions. Combining the data of laser radar and camera can achieve complementarity, thereby obtaining more complete three-dimensional information. However, combining laser radar with camera requires accurate determination of external parameters, that is, the rotation matrix and translation vector between the coordinate system of laser radar and the coordinate system of camera. Common external parameter calibration, one is to provide corresponding features for laser radar and camera for calibration, which makes the calibrator complex to make. Another is to use deep learning to achieve calibration, but the accuracy is not high. Another is to use the intensity information of point cloud for calibration, but the scene requirements are high. Summary of the invention

[0003] The first aspect of the present application provides a laser radar and camera extrinsic calibration method, which is applied to a rigid system composed of a laser radar and a camera, and includes:

[0004] Acquire a camera image of a target scene, wherein the target scene is provided with a calibrator, and the camera image includes a calibrator image of the calibrator;

[0005] Acquire an edge contour of the camera image, wherein the edge contour includes a contour of the calibrator image;

[0006] Constructing the edge line of the calibrator based on the outline of the calibrator;

[0007] Based on the edge contour and the calibrator edge line, obtaining the Planck coordinates of the fitted edge of the calibrator;

[0008] Acquire three-dimensional point cloud data of the target scene;

[0009] Based on the three-dimensional point cloud data, extracting the edge point set of the calibrator;

[0010] Based on the fitted edge and the edge point set, external parameters of the laser radar and the camera are obtained.

[0011] The laser radar and camera extrinsic parameter calibration method provided in the embodiment of the present application can obtain the calibrator image by shooting the target scene with the calibrator with a camera, which is used to calculate the fitting edge of the calibrator image; by scanning the target scene with the laser radar, the three-dimensional point cloud data of the calibrator can be obtained, so that the edge point set of the calibrator can be obtained. By aligning the fitting edge and the edge point set, the external parameters of the laser radar and the camera can be obtained. In other words, it is only necessary to use the camera and the laser radar to shoot the calibrator at the same time, and the external parameters of the laser radar and the camera can be obtained through calculation. There is no need for the calibrator to provide special features, nor a specific scene, and the process is simple and efficient, which is conducive to simplifying the process of external parameter calibration and improving the accuracy of calibration.

[0012] In one embodiment, the step of extracting the edge point set of the calibrator based on the three-dimensional point cloud data specifically includes: acquiring the calibrator point cloud data of the calibrator based on the three-dimensional point cloud data; and extracting the edge point set based on the calibrator point cloud data.

[0013] In one embodiment, based on the three-dimensional point cloud data, the step of obtaining the calibrator point cloud data of the calibrator specifically includes: extracting the plane point cloud data in the three-dimensional point cloud data; based on the connected domain determination method, deleting the non-connected areas in the plane point cloud data; based on the distance and normal data, filtering out the calibrator point cloud data from the plane point cloud data.

[0014] In one embodiment, the step of extracting the edge point set of the calibrator based on the three-dimensional point cloud data specifically includes: obtaining the calibrator point cloud data of the calibrator based on the three-dimensional point cloud data; obtaining the true point cloud data of the calibrator, the accuracy of the true point cloud data is less than or equal to 0.02mm; based on the true point cloud data, extracting the true edge point set of the calibrator; registering the true edge point set to the coordinate system of the calibrator point cloud data to obtain the edge point set.

[0015] In one embodiment, the step of constructing the calibrator edge line based on the contour of the calibrator specifically includes: calibrating two edge points near each edge line on the contour of the calibrator; marking a straight line formed by connecting the two edge points as the calibrator edge line.

[0016] In one embodiment, the step of obtaining the Planck coordinates of the fitted edge of the calibrator based on the edge contour and the calibrator edge line specifically includes: selecting multiple sampling points on the calibrator edge line; pairing each of the sampling points with the edge contour to obtain a target point set; based on the target point set, constructing a first objective function for solving the minimum independent variable point set, the first objective function being the square of the norm of the distance between the target point set and the straight line defined by the Planck coordinates, and the first parameter and the second parameter of the Planck coordinates being the independent variables solved by the first objective function.

[0017] In one embodiment, the step of obtaining the external parameters of the laser radar and the camera based on the fitted edge and the edge point set specifically includes: based on the four corner points of the calibrator, preliminarily corresponding the fitted edge with the edge point set to obtain initial external parameters; based on the fitted edge and the edge point set, constructing a second objective function for solving the minimum independent variable point set; the second objective function is the square norm of the distance between the point projected on the plane after the coordinates of each point in the edge point set are substituted into the initial external parameters and the fitted edge; the third parameter and the fourth parameter of the initial external parameter are the independent variables solved by the second objective function.

[0018] A second aspect of the present application provides a storage medium storing a computer program, which, when executed, implements the steps of any one of the above-mentioned laser radar and camera extrinsic parameter calibration methods.

[0019] The third aspect of the present application provides a laser radar and camera extrinsic calibration system, including:

[0020] A rigid system composed of a laser radar and a camera; the laser radar is used to obtain three-dimensional point cloud data of a target scene, and the camera is used to obtain a camera image of the target scene;

[0021] A calibrator is disposed in the target scene; and

[0022] A control module is electrically connected to the laser radar and the camera, respectively, and the control module includes a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, the control module implements the steps of the laser radar and camera extrinsic parameter calibration method described in any one of the above.

[0023] The laser radar and camera extrinsic parameter calibration system provided in the embodiment of the present application can obtain an image of the calibrator by shooting a target scene with a calibrator placed thereon with a camera, thereby calculating the fitting edge of the calibrator image; by scanning the target scene with a laser radar, the three-dimensional point cloud data of the calibrator can be obtained, thereby obtaining the edge point set of the calibrator. By aligning the fitting edge and the edge point set, the extrinsic parameters of the laser radar and the camera can be obtained. In other words, it is only necessary to use a camera and a laser radar to shoot the calibrator at the same time, and the extrinsic parameters of the laser radar and the camera can be obtained through calculation. There is no need for the calibrator to provide special features, nor a specific scene, and the process is simple and efficient, which is conducive to simplifying the process of extrinsic parameter calibration and improving the accuracy of calibration.

[0024] In one embodiment, the laser radar and camera external parameter calibration system also includes: a scanner, electrically connected to the control module, the scanner is used to scan the true value point cloud data of the calibrator, and the accuracy of the true value point cloud data is less than or equal to 0.02mm. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 This is a flow chart of the laser radar and camera extrinsic calibration method in an embodiment of the present application.

[0026] Figure 2 This is a camera image taken by the camera in the embodiment of the present application.

[0027] Figure 3 Schematic diagram of the outline edge of a camera image in an embodiment of the present application.

[0028] Figure 4 Schematic diagram of the overlap of the camera image and the fitting edge in the embodiment of the present application.

[0029] Figure 5 Schematic diagram of three-dimensional point cloud data in an embodiment of the present application.

[0030] Figure 6 Schematic diagram of plane point cloud extraction in an embodiment of the present application.

[0031] Figure 7 Schematic diagram of an edge point set in an embodiment of the present application.

[0032] Figure 8 This is a schematic diagram of the registration of true point cloud data in an embodiment of the present application.

[0033] Fig. 9 for Figure 8 A partially enlarged schematic diagram of .

[0034] Fig.10 Schematic diagram of the registration of the fitted edge and the edge point set in the embodiment of the present application.

[0035] Fig.11 for Fig.10 A partially enlarged schematic diagram of .

[0036] Fig.12 Schematic diagram of the structure of the laser radar and camera extrinsic calibration system in the embodiment of the present application.

[0037] Main component symbols

[0038] Steps: S1, S2, S3, S4, S5, S6, S7

[0039] Calibrator image: T

[0040] Edge points: M, N

[0041] Marker edge line: L1

[0042] Fitting edge: L j

[0043] Corner points: a, b, c, d

[0044] Calibrator point cloud data: C

[0045] Edge point set: X edge

[0046] True value edge point set: Y

[0047] LiDAR and camera external parameter calibration system: 100

[0048] Rigidity system: 10

[0049] LiDAR: 11

[0050] Camera: 13

[0051] Calibrator: 30

[0052] Control modules: 50

[0053] Scanner:70.

[0054] The following specific implementation methods will further illustrate the present application in conjunction with the above-mentioned drawings. DETAILED DESCRIPTION

[0055] The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments.

[0056] Unless otherwise defined, all technical and scientific terms used in this application have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used in the specification of this application are only for the purpose of describing specific embodiments and are not intended to limit this application.

[0057] In order to further explain the technical means and effects adopted by the present application to achieve the intended purpose, the present application is described in detail below in conjunction with the accompanying drawings and preferred implementation methods.

[0058] See also Figure 1 The laser radar and camera extrinsic calibration method provided in the embodiment of the present application includes:

[0059] Step S1: acquiring a camera image of a target scene, wherein the target scene is provided with a calibrator, and the camera image includes a calibrator image of the calibrator;

[0060] Step S2: obtaining an edge contour of the camera image, wherein the edge contour includes a contour of the calibrator image;

[0061] Step S3: constructing the edge line of the calibrator based on the outline of the calibrator;

[0062] Step S4: obtaining the Planck coordinates of the fitting edge of the calibrator based on the edge contour and the calibrator edge line;

[0063] Step S5: Acquire three-dimensional point cloud data of the target scene;

[0064] Step S6: extracting edge point data of the calibrator based on the three-dimensional point cloud data;

[0065] Step S7: Based on the fitted edge and the edge point data, obtain the external parameters of the laser radar and the camera.

[0066] Specifically, the laser radar and camera extrinsic calibration method provided in the embodiment of the present application is applied to a rigid system composed of a laser radar and a camera, that is, the positions of the laser radar and the camera are relatively fixed. The laser radar is used to scan the target scene, and the camera is used to obtain an image of the same target scene. The three-dimensional point cloud scanned by the laser radar and the image taken by the camera can be overlapped by performing specific rotations and translations. Therefore, the rotation matrix and translation vector that correspond the coordinate system of the three-dimensional point cloud to the coordinate system of the camera image are the extrinsic parameters of the laser radar and the camera.

[0067] In some embodiments, the calibrator is a rectangular flat plate structure, that is, the calibrator has four sides and four corners. The calibrator can be any object with a planar structure, such as paper, book, etc., or a specific rectangular plate, which is not limited in this application.

[0068] See also Figure 2 In step S1, a camera shoots a target scene to obtain a camera image. A calibrator is set in the target scene, so that the camera image shot by the camera contains a calibrator image T of the calibrator.

[0069] See also Figure 3 In step S2, the edge contour of the camera image can be obtained by using OpenCV. For example, the camera image can be converted into a grayscale image and binarized, and the contour in the camera image can be found by using the function in OpenCV, and finally the found contour can be drawn. The edge contour of the camera image includes the contour of the calibrator image T and the contours of other objects in the target scene. Among them, the edge contour is composed of multiple contour points.

[0070] Step S3 specifically includes:

[0071] Step S31: calibrating two edge points near each edge line on the contour of the calibrator;

[0072] Step S32: Mark the straight line formed by connecting the two edge points as the calibrator edge line.

[0073] Specifically, the user needs to manually mark the outline of the calibrator image T in the edge contour of the camera image. In step S31, the user needs to calibrate edge points M and edge points N near each edge line of the outline of the calibrator image T. Edge points M and edge points N are used to locate the edge of the calibrator. In step S32, the edge point M and edge point N are connected to obtain the calibrator edge line L0. The calibrator edge line L0 is close to the actual edge position in the calibrator image T. Therefore, the calibrator edge line L0 can be used to locate the position of the outline of the calibrator image T in the edge contour of the camera image.

[0074] After step S32, the method further includes: setting the intersection point between each calibrator edge line L0 and other calibrator edge lines L0 as the endpoint of the calibrator edge line L0. Specifically, the four calibrator edge lines L0 intersect in sequence to form a wireframe close to the outline of the calibrator image T. The intersection point of two calibrator edge lines L0 can be used as the endpoint of the calibrator edge line L0, thereby limiting the position of the calibrator edge line L0 to correspond to the edge line of the calibrator outline.

[0075] Step S4 specifically includes:

[0076] Step S41: selecting a plurality of sampling points on the edge line of the calibrator;

[0077] Step S42: Pair each of the sampling points with the edge contour to obtain a target point set;

[0078] Step S43: Based on the target point set, construct a first objective function for solving the minimum independent variable point set, wherein the first objective function is the norm square of the distance between the target point set and the straight line defined by the Planck coordinates, and the first parameter and the second parameter of the Planck coordinates are the independent variables solved by the first objective function.

[0079] In some embodiments, step S41 may be to uniformly select fifty points on the edge line L0 of the calibrator as acquisition points. In other embodiments, fifty acquisition points may be randomly selected, or other numbers of acquisition points may be selected, which is not limited in the present application.

[0080] In step S42, each sampling point is paired with an edge contour, specifically by selecting the contour point closest to the sampling point on the edge wheel library. The target point set p t is the set of contour points paired with each sampling point. Specifically, since the sampling point is a manually selected point on the calibrator edge line L0 that is closest to the contour of the calibrator image T, by finding the contour point on the edge contour that is closest to the sampling point, the contour point on the contour of the calibrator can be located with a high probability, thereby obtaining a target point set p that contains most of the contour points of the calibrator. t .

[0081] In step S42, the formula of the first objective function is specifically:

[0082]

[0083] Where L is the Planck coordinate, e(L,p t ) function represents the target point set p t The distance from any point in the equation to the straight line represented by the Planck coordinates L. The formula for the Planck coordinates L is:

[0084]

[0085] Among them, p l1 and p l2 The target point set p t For two random points in , m is the first parameter in Planck coordinates L, and d is the second parameter in Planck coordinates L.

[0086] The formula of the e(L,p) function is as follows:

[0087]

[0088] In summary, the first objective function is based on the target point set p tSolve the values ​​of the first parameter m and the second parameter d corresponding to the minimum value of the norm square of the e(L,p) function, so that the Planck coordinate L represented by the corresponding first parameter m and second parameter d is used as the fitting edge L of the calibrator image T j ,like Figure 4 shown.

[0089] See also Figure 5 Step S5 specifically uses a laser radar to obtain three-dimensional point cloud data of the target scene, and the three-dimensional point cloud data includes the calibrator cloud data C of the calibrator.

[0090] In some embodiments, step S6 includes:

[0091] Step S161: acquiring the calibrator point cloud data of the calibrator based on the three-dimensional point cloud data;

[0092] Step S162: extracting the edge point set based on the calibrator point cloud data.

[0093] Wherein, step S161 further includes:

[0094] Step S1611: extracting plane point cloud data from the three-dimensional point cloud data;

[0095] Step S1612: based on the connected domain determination method, deleting the non-connected areas in the plane point cloud data;

[0096] Step S1613: Based on the distance and normal data, filter out the calibrator point cloud data from the plane point cloud data.

[0097] Specifically, please refer to Figure 6 and Figure 7 In step S1611, the plane in the target scene can be extracted by plane fitting. That is, by analyzing the coordinates of each point in the three-dimensional point cloud data, the plane in the three-dimensional point cloud can be matched. Plane fitting can be implemented by using the least squares method, singular value decomposition method, random sampling consensus algorithm and other methods, and this application does not limit this.

[0098] In step S1612, the connected domain determination method is specifically to determine whether multiple planes are connected. Specifically, a connected domain is an area composed of pixels with the same pixel value and adjacent positions. Since the planes extracted in step S1611 include many planes with small areas and independent of each other, it is necessary to screen out planes that meet the requirements through connected domain determination.

[0099] In step S1613, the distance and normal data are specifically the distance and normal data of the plane of the calibrator compared to the laser radar. Specifically, since the distance and normal between the entire calibrator and the laser radar can be obtained by measurement, and the plane where the calibrator is located is quite different from other planes in the target scene (such as the ground, wall, ceiling), the calibrator point cloud data can be filtered out from the plane point cloud data by measuring the distance and normal data between the calibrator and the laser radar.

[0100] See also Figure 7 Step S162 specifically involves extracting edge point sets from the calibrator point cloud data using the edge extraction algorithm of the Point Cloud Library (PCL). X edge Among them, the edge point set X edge 3D point cloud data representing the edge locations of the calibrator.

[0101] In another embodiment, please refer to Figure 8 and Fig. 9 , step S6 specifically includes:

[0102] Step S261: acquiring the calibrator point cloud data of the calibrator based on the three-dimensional point cloud data;

[0103] Step S262: Acquire the true value point cloud data of the calibrator, wherein the accuracy of the true value point cloud data is less than or equal to 0.02 mm;

[0104] Step S263: extracting a true value edge point set of the calibrator based on the true value point cloud data;

[0105] Step S264: register the true value edge point set into the coordinate system of the calibrator point cloud data to obtain the edge point set.

[0106] The specific steps of step S261 are the same as those of step S161.

[0107] Step S262 specifically includes scanning the calibrator with a higher precision scanner to obtain the true value point cloud data of the calibrator. The scanning precision of the scanner is less than or equal to 0.02 mm, so that the precision of the scanned true value point cloud data is less than or equal to 0.02 mm.

[0108] Step S263 specifically includes: using the edge extraction algorithm of the Point Cloud Library (PCL) to extract the true value edge point set Y from the true value point cloud data.

[0109] In step S264, since the scanner and the laser radar are two independent components, the coordinate system of the true value point cloud data is different from the coordinate system of the calibrator point cloud data. By registering the true value edge point set Y to the coordinate system of the calibrator point cloud data, and using the registered true value edge point set Y as the edge point set of the calibrator point cloud data X edge , which can improve the edge point set X edge Therefore, the accuracy of subsequent registration is improved.

[0110] Step S7 specifically includes:

[0111] Step S71: Based on the four corner points of the calibrator, the fitting edge is preliminarily matched with the edge point set to obtain initial external parameters;

[0112] Step S72: Based on the fitted edge and the edge point set, construct a second objective function for solving the minimum independent variable point set; the second objective function is the square norm of the distance between the point where the coordinates of each point in the edge point set are projected on the plane after the initial external parameters are substituted and the fitted edge; the third parameter and the fourth parameter of the initial external parameter are the independent variables solved by the second objective function.

[0113] Specifically, please refer to Figure 4 and Figure 7 Step S71 specifically includes: fitting edge L j The corner point a and edge point set X edge Corresponding to the corner point a, the fitting edge L j The corner point b and edge point set X edge Corresponding to the corner point b, the fitting edge L j The corner point c and edge point set X edge The corner point c corresponds to the fitted edge L j The corner point d and edge point set X edge The corresponding corner point d is obtained to obtain an initial correspondence relationship between the laser radar and the camera's external parameters, which is used as the initial external parameter for iterative optimization in step S72.

[0114] Please also read Fig.10 and Fig.11 , the formula of the second objective function in step S72 is specifically:

[0115]

[0116] Among them, P is a pre-calibrated projection matrix used to transform the edge point set X edgeThe three-dimensional coordinates in are projected onto a two-dimensional plane. R is the third parameter of the second objective function, and T is the fourth parameter of the second objective function. The second objective function is used to solve the edge point set X edge The coordinates of the points in the initial external parameters after being transformed by the projection matrix P are consistent with the fitting edge L j The third parameter R and the fourth parameter T correspond to the minimum value of the square of the distance between them.

[0117] The laser radar and camera extrinsic parameter calibration method provided in the embodiment of the present application can obtain the extrinsic parameters of the laser radar and camera by obtaining the fitted edge of the calibrator edge in the camera image taken by the camera, and obtaining the edge point set of the calibrator edge scanned by the laser radar, and then aligning the fitted edge with the edge point set. It does not require the calibrator to provide special features or specific scenes, and the process is simple and efficient, which is conducive to simplifying the process of extrinsic parameter calibration and improving the accuracy of calibration.

[0118] See also Fig.12 The embodiment of the present application also provides a laser radar and camera extrinsic parameter calibration system 100, including a rigid system 10, a calibrator 30 and a control module 50, wherein the rigid system 10 includes a laser radar 11 and a camera 13, and the laser radar 11 and the camera 13 are rigidly fixed. The laser radar 11 is used to obtain three-dimensional point cloud data of the target scene, and the camera 13 is used to obtain the camera image of the target scene. The calibrator 30 is arranged in the target scene. The control module is electrically connected to the laser radar 11 and the camera 13 respectively. The control module 50 includes a memory and a processor (not shown in the figure), the memory stores a computer program, and when the processor executes the computer program, the control module 50 implements the steps of the laser radar and camera extrinsic parameter calibration method in the above embodiment.

[0119] In some embodiments, the laser radar and camera external parameter calibration system 100 also includes a scanner 70, which is electrically connected to the control module 50. The scanner 70 is used to scan the calibrator 30 to obtain true point cloud data. The accuracy of the true point cloud data is less than or equal to 0.02 mm, that is, the scanning accuracy of the scanner 70 is less than or equal to 0.02 mm.

[0120] The laser radar and camera extrinsic calibration system 100 provided in the embodiment of the present application can obtain the calibrator image T of the calibrator 30 by shooting the image of the target scene with the camera 13, thereby calculating the fitting edge of the calibrator image. By scanning the target scene with the laser radar 11, the three-dimensional point cloud data of the calibrator 30 can be obtained, so that the edge point set of the calibrator 30 can be obtained. By aligning the fitting edge and the edge point set, the extrinsic parameters of the laser radar 11 and the camera 13 can be obtained, without the need for the calibrator to provide special features or a specific scene, and the process is simple and efficient, which is conducive to simplifying the process of extrinsic calibration and improving the accuracy of calibration.

[0121] An embodiment of the present application also provides a non-volatile computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of any of the above-mentioned laser radar and camera extrinsic parameter calibration methods are implemented.

[0122] When the above-mentioned laser radar and camera extrinsic calibration method is implemented in the form of a separate product for sale or use, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned implementation method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable storage medium may include: any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium, etc.

[0123] The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc. The processor is the control center of the sample processing system / sample processing device, and uses various interfaces and lines to connect various parts of the entire sample processing system / sample processing device.

[0124] The memory is used to store the computer program and / or module, and the processor realizes various functions of the sample processing system / sample processing device by running or executing the computer program and / or module stored in the memory, and calling the data stored in the memory. The memory may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function), etc. In addition, the memory may include a high-speed random access memory, and may also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (SecureDigital, SD) card, a flash card (Flash Card), at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0125] Those skilled in the art should recognize that the above embodiments are only used to illustrate the present application and are not intended to be limiting of the present application. As long as they are within the spirit and scope of the present application, appropriate changes and modifications to the above embodiments are within the scope of protection claimed in the present application.

Claims

1. A laser radar and camera extrinsic calibration method, applied to a rigid system composed of a laser radar and a camera, characterized in that: include: Acquire a camera image of a target scene, wherein the target scene is provided with a calibrator, and the camera image includes a calibrator image of the calibrator; Acquire an edge contour of the camera image, wherein the edge contour includes a contour of the calibrator image; Constructing a calibrator edge line based on the contour of the calibrator; Based on the edge contour and the calibrator edge line, obtaining the Planck coordinates of the fitted edge of the calibrator; Acquire three-dimensional point cloud data of the target scene; Based on the three-dimensional point cloud data, extracting the edge point set of the calibrator; Based on the fitted edge and the edge point set, obtaining external parameters of the laser radar and the camera; The step of constructing the edge line of the calibrator based on the contour of the calibrator specifically includes: Marking two edge points near each edge line on the contour of the marker; Marking a straight line formed by connecting two edge points as the calibrator edge line; The step of obtaining the Planck coordinates of the fitting edge of the calibrator based on the edge contour and the calibrator edge line specifically includes: Selecting a plurality of sampling points on the edge line of the calibrator; Pairing each of the sampling points with the edge contour to obtain a target point set; Based on the target point set, a first objective function for solving the minimum independent variable point set is constructed, wherein the first objective function is the norm square of the distance between the target point set and the straight line defined by the Planck coordinates, and the first parameter and the second parameter of the Planck coordinates are the independent variables solved by the first objective function.

2. The laser radar and camera extrinsic parameter calibration method according to claim 1, characterized in that: Based on the three-dimensional point cloud data, the step of extracting the edge point set of the calibrator specifically includes: Based on the three-dimensional point cloud data, acquiring calibrator point cloud data of the calibrator; Based on the calibrator point cloud data, the edge point set is extracted.

3. The laser radar and camera extrinsic parameter calibration method according to claim 2, characterized in that: Based on the three-dimensional point cloud data, the step of obtaining the calibrator point cloud data of the calibrator specifically includes: Extracting plane point cloud data from the three-dimensional point cloud data; Based on a connected domain determination method, deleting non-connected areas in the plane point cloud data; Based on the distance and normal data, the calibrator point cloud data is filtered out from the plane point cloud data.

4. The laser radar and camera extrinsic calibration method according to claim 1, characterized in that: Based on the three-dimensional point cloud data, the step of extracting the edge point set of the calibrator specifically includes: Based on the three-dimensional point cloud data, acquiring calibrator point cloud data of the calibrator; Acquire true value point cloud data of the calibrator, wherein the accuracy of the true value point cloud data is less than or equal to 0.02 mm; Based on the true value point cloud data, extracting the true value edge point set of the calibrator; The true value edge point set is registered in the coordinate system of the calibrator point cloud data to obtain the edge point set.

5. The laser radar and camera extrinsic calibration method according to claim 1, characterized in that: The step of obtaining the external parameters of the laser radar and the camera based on the fitted edge and the edge point set specifically includes: Based on the four corner points of the calibrator, the fitting edge is preliminarily matched with the edge point set to obtain an initial external parameter; Based on the fitted edge and the edge point set, a second objective function for solving the minimum independent variable point set is constructed; the second objective function is the square norm of the distance between the point where the coordinates of each point in the edge point set are projected on the plane after the initial external parameters are substituted and the fitted edge; the third parameter and the fourth parameter of the initial external parameter are the independent variables solved by the second objective function.

6. A storage medium storing a computer program, characterized in that: When executed, the computer program implements the steps of the laser radar and camera extrinsic parameter calibration method as described in any one of claims 1-5.

7. A laser radar and camera external parameter calibration system, characterized in that: include: A rigid system composed of a laser radar and a camera; the laser radar is used to obtain three-dimensional point cloud data of a target scene, and the camera is used to obtain a camera image of the target scene; A calibrator, arranged in the target scene; as well as A control module is electrically connected to the laser radar and the camera, respectively, and the control module includes a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, the control module implements the steps of the laser radar and camera extrinsic parameter calibration method as described in any one of claims 1 to 5.

8. The laser radar and camera extrinsic calibration system as claimed in claim 7, characterized in that: Also includes: A scanner is electrically connected to the control module, and is used to scan the true value point cloud data of the calibrator, wherein the accuracy of the true value point cloud data is less than or equal to 0.02 mm.

Images