Laser radar joint calibration method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention provides a laser radar joint calibration method and device, electronic equipment and a storage medium, and the method comprises the steps: obtaining original data which comprises laser radar data and image data; generating calibration data according to the laser radar data and the image data; performing angular point identification on the calibration data to obtain an angular point pair set; and calibrating the angular points in the angular point pair set to obtain a calibration result. By implementing the embodiment of the invention, the radar calibration precision can be improved, the three-dimensional point cloud and the panoramic image are effectively combined, the calibration error and data redundancy are reduced, the joint calibration efficiency is improved, the calibration process is simplified, and the cost is reduced.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and more specifically, to a laser radar joint calibration method, device, electronic device and storage medium. Background Art

[0002] The mobile LiDAR scanning system is an efficient measurement system that has developed rapidly in recent years and has been widely used in digital factories, digital mines, deformation monitoring, tunnel engineering and other fields. The system can quickly and accurately obtain a large amount of three-dimensional spatial information of the target object, but it does not contain color information. Panoramic images contain complete panoramic information and can provide richer semantic information.

[0003] By combining the above two technologies, it is possible to achieve an accurate and intuitive description of the target and generate an immersive three-dimensional model. Since panoramic cameras are usually composed of multiple lenses, the traditional calibration algorithm needs to first perform internal calibration on each lens, and then perform joint calibration of the lidar and panoramic camera. The joint calibration method usually splits the entire system into multiple parts, calculates the relative pose between each part in turn, and finally unifies them into a certain coordinate system. This method has cumulative errors after multiple transformations, resulting in the degradation of calibration accuracy. In addition, it takes a lot of time and manpower to calibrate multiple parts separately. The process is cumbersome and complicated, and inefficient. Summary of the invention

[0004] The purpose of the embodiments of the present application is to provide a laser radar joint calibration method, device, electronic device and storage medium, which can improve the accuracy of radar calibration, effectively combine three-dimensional point cloud with panoramic image, reduce calibration error and data redundancy, improve the efficiency of joint calibration, simplify the calibration process and reduce costs.

[0005] In a first aspect, an embodiment of the present application provides a laser radar joint calibration method, the method comprising:

[0006] Acquire raw data, where the raw data includes laser radar data and image data;

[0007] Generate calibration data based on the laser radar data and the image data;

[0008] Performing corner point recognition on the calibration data to obtain a set of corner point pairs;

[0009] The corner points in the corner point pair set are calibrated to obtain a calibration result.

[0010] In the above implementation process, calibration data is generated according to the lidar data and image data, and then corner point recognition and calibration are performed to finally obtain the calibration result, which can improve the accuracy of radar calibration, effectively combine the three-dimensional point cloud with the panoramic image, reduce calibration errors and data redundancy, improve the efficiency of joint calibration, simplify the calibration process, and reduce costs.

[0011] Furthermore, the step of generating calibration data according to the original data comprises:

[0012] Performing three-dimensional reconstruction on the laser radar data according to the SLAM algorithm to obtain an initial calibration scene point cloud;

[0013] Performing spatial cropping on the initial calibration scene point cloud to obtain a spatially cropped initial calibration scene point cloud;

[0014] Downsampling the initial calibration scene point cloud after spatial cropping to obtain a downsampled initial calibration scene point cloud;

[0015] Removing outliers from the downsampled initial calibration scene point cloud to obtain a removed initial calibration scene point cloud;

[0016] Performing smoothing on the initial calibration scene point cloud after the removal process to obtain a calibration scene point cloud;

[0017] Obtaining a calibration scene image according to the image data;

[0018] The calibration data is obtained according to the calibration scene point cloud and the calibration scene image.

[0019] In the above implementation process, after the initial calibration scene point cloud is spatially cropped, downsampled, removed and smoothed, the accuracy of the calibration data can be improved and unnecessary errors generated in the preprocessing process of the calibration data can be reduced.

[0020] Furthermore, the step of performing corner point recognition on the calibration data to obtain a set of corner point pairs includes:

[0021] Performing corner point recognition on the image data in the calibration data to obtain a two-dimensional ordered sequence;

[0022] Performing corner point recognition on the point cloud data in the calibration data to obtain a three-dimensional ordered sequence;

[0023] The set of corner point pairs is constructed according to the two-dimensional ordered sequence and the three-dimensional ordered sequence.

[0024] Furthermore, the step of constructing the set of corner point pairs according to the two-dimensional ordered sequence and the three-dimensional ordered sequence includes:

[0025] Constructing an initial corner point pair set according to the two-dimensional ordered sequence and the three-dimensional ordered sequence;

[0026] Obtaining a two-dimensional average distance of corner points according to the two-dimensional ordered sequence;

[0027] Obtaining a filtering radius according to the average distance of the two-dimensional corner points;

[0028] Traversing the initial corner point pair set, filtering the two-dimensional corner points in the initial corner point pair set according to the filter radius to obtain filtered two-dimensional corner points;

[0029] The corner point pair set is generated according to the initial corner point pairs corresponding to the filtered two-dimensional corner points.

[0030] In the above implementation process, by screening the two-dimensional ordered sequence and the three-dimensional ordered sequence in the initial corner point pair set, the distribution of the corner point data in the entire scene can be made more uniform, the redundant point pair data can be reduced, and the calibration accuracy can be improved.

[0031] Furthermore, the step of calibrating the corner points in the corner point pair set to obtain a calibration result includes:

[0032] Establish a unit sphere based on a pre-constructed coordinate system;

[0033] Projecting any two-dimensional corner point in the set of corner point pairs onto the unit sphere to obtain the projection point coordinates of the two-dimensional corner point;

[0034] A rotation and translation matrix is ​​obtained according to the projection point coordinates of the two-dimensional corner points and the three-dimensional corner points in the corner point pair set, and the rotation and translation matrix is ​​used as the calibration result.

[0035] In the above implementation process, the two-dimensional corner points are projected onto the unit sphere to construct a mapping relationship with the corresponding three-dimensional corner points. The calibration results can be directly obtained through calculation, which simplifies the calibration process and improves the accuracy of the calibration results.

[0036] Furthermore, the step of obtaining the original data includes:

[0037] Setting parameters required for obtaining the original data;

[0038] The acquisition device is controlled according to the parameters to perform acquisition at a preset angle and position to obtain the original data.

[0039] In the above implementation process, the data is collected according to the preset angle and position, which can ensure the availability of the original data and make the original data contain more useful information.

[0040] In a second aspect, an embodiment of the present application further provides a laser radar joint calibration device, the device comprising:

[0041] An acquisition module, used to acquire raw data, wherein the raw data includes laser radar data and image data;

[0042] A generating module, used for generating calibration data according to the laser radar data and the image data;

[0043] A corner point recognition module is used to perform corner point recognition on the calibration data to obtain a set of corner point pairs;

[0044] The calibration module is used to calibrate the corner points in the corner point pair set to obtain a calibration result.

[0045] In the above implementation process, calibration data is generated according to the lidar data and image data, and then corner point recognition and calibration are performed to finally obtain the calibration result, which can improve the accuracy of radar calibration, effectively combine the three-dimensional point cloud with the panoramic image, reduce calibration errors and data redundancy, improve the efficiency of joint calibration, simplify the calibration process, and reduce costs.

[0046] In a third aspect, an embodiment of the present application provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method described in any one of the first aspects when executing the computer program.

[0047] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, on which instructions are stored. When the instructions are executed on a computer, the computer executes the method as described in any one of the first aspects.

[0048] In a fifth aspect, an embodiment of the present application provides a computer program product, which, when running on a computer, enables the computer to execute the method as described in any one of the first aspects.

[0049] Other features and advantages of the present disclosure will be set forth in the following description, or some features and advantages may be inferred or unambiguously determined from the description, or may be learned by implementing the above-mentioned technology of the present disclosure.

[0050] And it can be implemented according to the contents of the specification. The following is a detailed description of the preferred embodiments of the present application with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments of the present application will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the range values. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying any creative work.

[0052] Figure 1 A schematic diagram of a process flow of a laser radar joint calibration method provided in an embodiment of the present application;

[0053] Figure 2 A schematic diagram of the structure of the laser radar joint calibration device provided in an embodiment of the present application;

[0054] Figure 3 A schematic diagram of the structural composition of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0055] The technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application.

[0056] It should be noted that similar reference numerals and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings. At the same time, in the description of this application, the terms "first", "second", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance.

[0057] The specific implementation methods of the present application are further described in detail below in conjunction with the accompanying drawings and examples. The following examples are used to illustrate the present application, but are not used to limit the scope of the present application.

[0058] Embodiment 1

[0059] Figure 1 is a flow chart of the laser radar joint calibration method provided in the embodiment of the present application, such as Figure 1 As shown, the method includes:

[0060] S1, obtaining raw data, which includes lidar data and image data;

[0061] S2, generating calibration data based on the lidar data and image data;

[0062] S3, identifying corner points of the calibration data to obtain a set of corner point pairs;

[0063] S4, calibrating the corner points in the corner point pair set to obtain a calibration result.

[0064] In the above implementation process, calibration data is generated according to the lidar data and image data, and then corner point recognition and calibration are performed to finally obtain the calibration result, which can improve the accuracy of radar calibration, effectively combine the three-dimensional point cloud with the panoramic image, reduce calibration errors and data redundancy, improve the efficiency of joint calibration, simplify the calibration process, and reduce costs.

[0065] Furthermore, S1 includes:

[0066] Set the parameters required to obtain raw data;

[0067] According to the parameters, the acquisition equipment is controlled to collect data at the preset angle and position to obtain the original data.

[0068] In the above implementation process, the data is collected according to the preset angle and position, which can ensure the availability of the original data and make the original data contain more useful information.

[0069] The embodiment of the present application collects data through a mobile SLAM device and sets the parameters of the SLAM device data collection system. The SLAM device mainly includes a laser radar Pandar XT-32 and a panoramic camera insta360 x3. The positions of the two devices are relatively fixed. The collected raw data include: laser radar data (laser point cloud data) and image data (panoramic image data).

[0070] Set the parameters of the SLAM device acquisition system, including the frame rate of the laser radar and the exposure time of the panoramic camera. During the acquisition process, slowly move the SLAM device up, down, left, and right to collect the laser point cloud data and panoramic image data of the calibration scene.

[0071] Repeat the acquisition with different angles and positions as the starting position to obtain N sets of data, that is, the original data contains multiple sets of data collected at different angles and positions. N is set based on experience, and it is recommended that N is not less than 16.

[0072] Furthermore, S2 includes:

[0073] The LiDAR data is reconstructed in three dimensions according to the SLAM algorithm to obtain the initial calibration scene point cloud;

[0074] Perform spatial cropping on the initial calibration scene point cloud to obtain the spatially cropped initial calibration scene point cloud;

[0075] Downsampling the initial calibration scene point cloud after spatial cropping to obtain a downsampled initial calibration scene point cloud;

[0076] Removing outliers from the downsampled initial calibration scene point cloud to obtain the removed initial calibration scene point cloud;

[0077] Smoothing the initial calibration scene point cloud after the removal process to obtain the calibration scene point cloud;

[0078] Obtaining a calibration scene image according to the image data;

[0079] The calibration data is obtained according to the calibration scene point cloud and the calibration scene image.

[0080] In the above implementation process, after the initial calibration scene point cloud is spatially cropped, downsampled, removed and smoothed, the accuracy of the calibration data can be improved and unnecessary errors generated in the preprocessing process of the calibration data can be reduced.

[0081] The embodiment of the present application uses the FAST_LIO method to perform SLAM positioning, posture determination and fusion on the laser point cloud data of the calibration scene to generate an initial calibration scene point cloud.

[0082] The initial calibration scene point cloud is spatially cropped to remove points outside the calibration scene range. The voxel grid filtering method is used to downsample the initial calibration scene point cloud to reduce the point cloud density to reduce the data volume and improve data processing efficiency. The statistical filtering algorithm is used to remove outliers in the initial calibration scene point cloud. The moving least squares algorithm is used to smooth the initial calibration scene point cloud.

[0083] This application processes the three-dimensional lidar data and the two-dimensional image data respectively to obtain calibration data. The embodiment of this application is based on the 360 ​​video synthesis software that comes with the panoramic camera, synthesizes the videos from multiple lenses of the panoramic camera into a 360 video, and extracts the first frame image of the 360 ​​video as the calibration scene image.

[0084] Furthermore, S3 includes:

[0085] Perform corner point recognition on the image data in the calibration data to obtain a two-dimensional ordered sequence;

[0086] Perform corner point recognition on the point cloud data in the calibration data to obtain a three-dimensional ordered sequence;

[0087] Construct a set of corner point pairs based on a two-dimensional ordered sequence and a three-dimensional ordered sequence.

[0088] Furthermore, the step of constructing a set of corner point pairs according to the two-dimensional ordered sequence and the three-dimensional ordered sequence includes:

[0089] Construct an initial set of corner point pairs according to a two-dimensional ordered sequence and a three-dimensional ordered sequence;

[0090] Obtain the average distance of two-dimensional corner points according to the two-dimensional ordered sequence;

[0091] The filter radius is obtained according to the average distance of the two-dimensional corner points;

[0092] Traverse the initial corner point pair set, filter the two-dimensional corner points in the initial corner point pair set according to the filter radius, and obtain the filtered two-dimensional corner points;

[0093] A set of corner point pairs is generated according to the initial corner point pairs corresponding to the filtered two-dimensional corner points.

[0094] In the above implementation process, by screening the two-dimensional ordered sequence and the three-dimensional ordered sequence in the initial corner point pair set, the distribution of the corner point data in the entire scene can be made more uniform, the redundant point pair data can be reduced, and the calibration accuracy can be improved.

[0095] For any set of calibration data, the panoramic image checkerboard 2D corner points (two-dimensional corner points) and the calibration scene point cloud 3D corner points (three-dimensional corner points) are sequentially added to the set P to form the calibration data checkerboard corner point pair set Q (corner point pair set).

[0096] In the embodiment of the present application, the checkerboard corner point pair data in the set P (initial corner point pair set) are screened to obtain better calibration accuracy. and three-dimensional ordered sequence Where M is the number of corner points on the chessboard.

[0097] The screening process can be achieved by the following steps: Calculate the average distance of the 2D corner points of the chessboard (average distance of the two-dimensional corner points): Among them, dist(a,b) is the Euclidean distance between points a and b.

[0098] Set filter radius: r = k*d mean , where k is the filter coefficient, and the recommended value range is (0.2, 1.0). Traverse the checkerboard corner point pair data in set P, and search for 2D corner point data within the filter radius. If there are other corner point data within the filter radius, only one point pair data is retained within the range. One implementation method is to retain the 2D corner point with the farthest distance to the corresponding 3D corner point (i.e., the sum of the squares of the x-axis, y-axis, and z-axis components is the largest). After updating set P, continue traversing until all data in set P are traversed.

[0099] Furthermore, S4 includes:

[0100] Establish a unit sphere based on a pre-constructed coordinate system;

[0101] Project any two-dimensional corner point in the set of corner point pairs onto the unit sphere to obtain the projection point coordinates of the two-dimensional corner point;

[0102] The rotation and translation matrix is ​​obtained according to the projection point coordinates of the two-dimensional corner points and the three-dimensional corner points in the corner point pair set, and the rotation and translation matrix is ​​used as the calibration result.

[0103] In the above implementation process, the two-dimensional corner points are projected onto the unit sphere to construct a mapping relationship with the corresponding three-dimensional corner points. The calibration results can be directly obtained through calculation, which simplifies the calibration process and improves the accuracy of the calibration results.

[0104] Establish a 3D imaging coordinate system for the panoramic camera, and establish a unit sphere with the origin of the coordinate system as the center. m Projecting the point p on the unit sphere ms , p m The coordinates of (x p ,y p ), p ms The coordinates of (x ps ,y ps ,z ps ),in: θ p =y p / h*π, w is the number of columns of the panoramic image data, and h is the number of rows of the panoramic image data.

[0105] The rotation and translation matrix is ​​obtained by the following formula:

[0106]

[0107] Among them, RT is the rotation and translation matrix, M is the number of corner point pairs in the corner point pair set, and p ms is the projection point coordinate of the mth two-dimensional corner point in the set of corner point pairs, q m is the mth three-dimensional corner point in the set of corner point pairs.

[0108] In the above implementation process, based on the projection imaging principle, the extrinsic parameter calibration problem of the panoramic camera and the lidar is transformed into the task of solving the rotation transformation matrix. The required rotation and translation matrix can make the three-dimensional corner points, the corresponding two-dimensional corner points spherical projection points and the center of the sphere collinear. The implementation principle is simple and easy to implement, which can simplify the calibration process and reduce costs.

[0109] There exists a rotation and translation matrix RT that acts on any 3D chessboard corner point q in Q m , we get q mc , that is: q mc =RT*q m , so that q mc The line connecting the center of the unit sphere intersects the unit sphere at point p ms , that is, the center of the unit ball, p ms ,q mc The three points are collinear, which is equivalent to (p ms -RT*q m )×p ms= 0, where × is the cross product. The rotation and translation matrix is ​​the external parameter of the panoramic camera and lidar to be calibrated, which can be solved by minimizing the following objective function.

[0110] The present application provides a joint calibration method for a panoramic camera and a lidar. Based on a panoramic image synthesized by multiple lenses, the checkerboard corner points are projected onto a unit sphere, and an optimization function is constructed with the checkerboard corner points in the lidar point cloud for solution. There is no need to calibrate the intrinsic parameters of multiple lenses, there is no need to separately calibrate the extrinsic parameters between multiple cameras, and there is no need to separately calibrate the extrinsic parameters between multiple cameras and radars. This can simplify the calibration process and save time and labor costs.

[0111] Embodiment 2

[0112] In order to execute the method corresponding to the above embodiment 1 to achieve the corresponding functions and technical effects, a laser radar joint calibration device is provided below, such as Figure 2 As shown, the device comprises:

[0113] An acquisition module 1 is used to acquire raw data, which includes laser radar data and image data;

[0114] A generation module 2, used to generate calibration data according to the laser radar data and the image data;

[0115] Corner point recognition module 3, used to identify corner points of calibration data and obtain a set of corner point pairs;

[0116] The calibration module 4 is used to calibrate the corner points in the corner point pair set to obtain a calibration result.

[0117] In the above implementation process, calibration data is generated according to the lidar data and image data, and then corner point recognition and calibration are performed to finally obtain the calibration result, which can improve the accuracy of radar calibration, effectively combine the three-dimensional point cloud with the panoramic image, reduce calibration errors and data redundancy, improve the efficiency of joint calibration, simplify the calibration process, and reduce costs.

[0118] Furthermore, the acquisition module 1 is also used for:

[0119] Set the parameters required to obtain raw data;

[0120] According to the parameters, the acquisition equipment is controlled to collect data at the preset angle and position to obtain the original data.

[0121] In the above implementation process, the data is collected according to the preset angle and position, which can ensure the availability of the original data and make the original data contain more useful information.

[0122] Furthermore, the generating module 2 is also used for:

[0123] The LiDAR data is reconstructed in three dimensions according to the SLAM algorithm to obtain the initial calibration scene point cloud;

[0124] Perform spatial cropping on the initial calibration scene point cloud to obtain the spatially cropped initial calibration scene point cloud;

[0125] Downsampling the initial calibration scene point cloud after spatial cropping to obtain a downsampled initial calibration scene point cloud;

[0126] Removing outliers from the downsampled initial calibration scene point cloud to obtain the removed initial calibration scene point cloud;

[0127] Smoothing the initial calibration scene point cloud after the removal process to obtain the calibration scene point cloud;

[0128] Obtaining a calibration scene image according to the image data;

[0129] The calibration data is obtained according to the calibration scene point cloud and the calibration scene image.

[0130] In the above implementation process, after the initial calibration scene point cloud is spatially cropped, downsampled, removed and smoothed, the accuracy of the calibration data can be improved and unnecessary errors generated in the preprocessing process of the calibration data can be reduced.

[0131] Furthermore, the corner point recognition module 3 is also used for:

[0132] Perform corner point recognition on the image data in the calibration data to obtain a two-dimensional ordered sequence;

[0133] Perform corner point recognition on the point cloud data in the calibration data to obtain a three-dimensional ordered sequence;

[0134] Construct a set of corner point pairs based on a two-dimensional ordered sequence and a three-dimensional ordered sequence.

[0135] Furthermore, the corner point recognition module 3 is also used for:

[0136] Construct an initial set of corner point pairs according to a two-dimensional ordered sequence and a three-dimensional ordered sequence;

[0137] Obtain the average distance of two-dimensional corner points according to the two-dimensional ordered sequence;

[0138] The filter radius is obtained according to the average distance of the two-dimensional corner points;

[0139] Traverse the initial corner point pair set, filter the two-dimensional corner points in the initial corner point pair set according to the filter radius, and obtain the filtered two-dimensional corner points;

[0140] A set of corner point pairs is generated according to the initial corner point pairs corresponding to the filtered two-dimensional corner points.

[0141] In the above implementation process, by screening the two-dimensional ordered sequence and the three-dimensional ordered sequence in the initial corner point pair set, the distribution of the corner point data in the entire scene can be made more uniform, the redundant point pair data can be reduced, and the calibration accuracy can be improved.

[0142] Furthermore, the calibration module 4 is also used for:

[0143] Establish a unit sphere based on a pre-constructed coordinate system;

[0144] Project any two-dimensional corner point in the set of corner point pairs onto the unit sphere to obtain the projection point coordinates of the two-dimensional corner point;

[0145] The rotation and translation matrix is ​​obtained according to the projection point coordinates of the two-dimensional corner points and the three-dimensional corner points in the corner point pair set, and the rotation and translation matrix is ​​used as the calibration result.

[0146] In the above implementation process, the two-dimensional corner points are projected onto the unit sphere to construct a mapping relationship with the corresponding three-dimensional corner points. The calibration results can be directly obtained through calculation, which simplifies the calibration process and improves the accuracy of the calibration results.

[0147] The above-mentioned laser radar joint calibration device can implement the method of the above-mentioned embodiment 1. The options in the above-mentioned embodiment 1 are also applicable to this embodiment and will not be described in detail here.

[0148] The rest of the contents of the embodiments of the present application can refer to the contents of the above-mentioned embodiment 1, and will not be repeated in this embodiment.

[0149] Embodiment 3

[0150] An embodiment of the present application provides an electronic device, including a memory and a processor, wherein the memory is used to store a computer program, and the processor runs the computer program to enable the electronic device to execute the laser radar joint calibration method of embodiment one.

[0151] Optionally, the above-mentioned electronic device may be a server.

[0152] See also Figure 3 , Figure 3 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. The electronic device may include a processor 31, a communication interface 32, a memory 33 and at least one communication bus 34. The communication bus 34 is used to realize direct connection and communication between these components. The communication interface 32 of the device in the embodiment of the present application is used to communicate signaling or data with other node devices. The processor 31 may be an integrated circuit chip with signal processing capabilities.

[0153] The processor 31 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general-purpose processor can be a microprocessor or the processor 31 can also be any conventional processor, etc.

[0154] The memory 33 may be, but is not limited to, a random access memory (RAM), a read only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable read-only memory (EEPROM), etc. The memory 33 stores computer-readable instructions. When the computer-readable instructions are executed by the processor 31, the device can execute the above-mentioned Figure 1 The method embodiment involves various steps.

[0155] Optionally, the electronic device may further include a storage controller and an input / output unit. The memory 33, the storage controller, the processor 31, the peripheral interface, and the input / output unit are electrically connected to each other directly or indirectly to achieve data transmission or interaction. For example, these components may be electrically connected to each other via a communication bus 34. The processor 31 is used to execute executable modules stored in the memory 33, such as software function modules or computer programs included in the device.

[0156] The input and output unit is used to provide users with the task creation and to create an optional time period or preset execution time for the task to enable interaction between the user and the server. The input and output unit can be, but is not limited to, a mouse and a keyboard.

[0157] Understandably, Figure 3 The structure shown is for illustration only. The electronic device may also include Figure 3 More or fewer components as shown, or with Figure 3 Different configurations are shown. Figure 3 Each component shown in the figure can be implemented by hardware, software or a combination thereof.

[0158] In addition, an embodiment of the present application also provides a computer-readable storage medium, which stores a computer program, and when the computer program is executed by a processor, it implements the laser radar joint calibration method of embodiment one.

[0159] The embodiment of the present application also provides a computer program product, which, when running on a computer, enables the computer to execute the method described in the method embodiment.

[0160] In several embodiments provided in the present application, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely schematic. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the devices, methods and computer program products according to multiple embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, a program segment or a part of a code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or the flowchart, and the combination of boxes in the block diagram and / or the flowchart, can be implemented with a dedicated hardware-based device that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0161] In addition, the functional modules in the various embodiments of the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.

[0162] If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as USB flash drives, mobile hard drives, ROM, RAM, magnetic disks, or optical disks.

[0163] The above description is only an embodiment of the present application and is not intended to limit the protection scope of the present application. For those skilled in the art, the present application may have various changes and variations. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application should be included in the protection scope of the present application. It should be noted that similar numbers and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in the subsequent drawings.

[0164] The above is only a specific implementation of the present application, but the protection range of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical range disclosed in the present application, which should be included in the protection range of the present application. Therefore, the protection range of the present application should be based on the protection range of the claims.

[0165] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

Claims

1. A laser radar joint calibration method, characterized in that: The method comprises: Acquire raw data, where the raw data includes laser radar data and image data; Generate calibration data based on the laser radar data and the image data; Performing corner point recognition on the calibration data to obtain a set of corner point pairs; The corner points in the corner point pair set are calibrated to obtain a calibration result.

2. The laser radar joint calibration method according to claim 1, characterized in that: The step of generating calibration data according to the original data comprises: Performing three-dimensional reconstruction on the laser radar data according to the SLAM algorithm to obtain an initial calibration scene point cloud; Performing spatial cropping on the initial calibration scene point cloud to obtain a spatially cropped initial calibration scene point cloud; Downsampling the initial calibration scene point cloud after spatial cropping to obtain a downsampled initial calibration scene point cloud; Removing outliers from the downsampled initial calibration scene point cloud to obtain a removed initial calibration scene point cloud; Performing smoothing on the initial calibration scene point cloud after the removal process to obtain a calibration scene point cloud; Obtaining a calibration scene image according to the image data; The calibration data is obtained according to the calibration scene point cloud and the calibration scene image.

3. The laser radar joint calibration method according to claim 1, characterized in that: The step of performing corner point recognition on the calibration data to obtain a set of corner point pairs comprises: Performing corner point recognition on the image data in the calibration data to obtain a two-dimensional ordered sequence; Performing corner point recognition on the point cloud data in the calibration data to obtain a three-dimensional ordered sequence; The set of corner point pairs is constructed according to the two-dimensional ordered sequence and the three-dimensional ordered sequence.

4. The laser radar joint calibration method according to claim 3, characterized in that: The step of constructing the set of corner point pairs according to the two-dimensional ordered sequence and the three-dimensional ordered sequence comprises: Constructing an initial corner point pair set according to the two-dimensional ordered sequence and the three-dimensional ordered sequence; Obtaining a two-dimensional average distance of corner points according to the two-dimensional ordered sequence; Obtaining a filtering radius according to the average distance of the two-dimensional corner points; Traversing the initial corner point pair set, filtering the two-dimensional corner points in the initial corner point pair set according to the filter radius to obtain filtered two-dimensional corner points; The corner point pair set is generated according to the initial corner point pairs corresponding to the filtered two-dimensional corner points.

5. The laser radar joint calibration method according to claim 1, characterized in that: The step of calibrating the corner points in the corner point pair set to obtain a calibration result comprises: Establish a unit sphere based on a pre-constructed coordinate system; Projecting any two-dimensional corner point in the set of corner point pairs onto the unit sphere to obtain the projection point coordinates of the two-dimensional corner point; A rotation and translation matrix is ​​obtained according to the projection point coordinates of the two-dimensional corner points and the three-dimensional corner points in the corner point pair set, and the rotation and translation matrix is ​​used as the calibration result.

6. The laser radar joint calibration method according to claim 1, characterized in that: The step of obtaining the original data comprises: Setting parameters required for obtaining the original data; The acquisition device is controlled according to the parameters to perform acquisition at a preset angle and position to obtain the original data.

7. A laser radar joint calibration device, characterized in that: The device comprises: An acquisition module, used to acquire raw data, wherein the raw data includes laser radar data and image data; A generating module, used for generating calibration data according to the laser radar data and the image data; A corner point recognition module, used to perform corner point recognition on the calibration data to obtain a set of corner point pairs; The calibration module is used to calibrate the corner points in the corner point pair set to obtain a calibration result.

8. An electronic device, characterized in that: It includes a memory and a processor, the memory is used to store a computer program, and the processor runs the computer program to enable the electronic device to perform the laser radar joint calibration method according to any one of claims 1 to 6.

9. A storage medium, characterized in that: It stores a computer program, which, when executed by a processor, implements the laser radar joint calibration method as described in any one of claims 1 to 6.

Images