Multi-device calibration method, device and computer-readable storage medium

Abstract

The present application relates to a multi-device calibration method, apparatus and computer-readable storage medium. The method includes: obtaining initial calibration parameters of a first device; obtaining initial calibration parameters of a second device; evaluating the accuracy of the first device and the accuracy of the second device according to known parameters of any first marker in any first preset environment, wherein the known parameters include the position information of any first marker in the first preset environment; if the accuracy of the first device is higher than that of the second device, then the second device is jointly calibrated using the initial calibration parameters of the first device with any second marker in any second preset environment to obtain the final calibration parameters of the second device. The solution provided by the present application can calibrate multiple (kinds) of devices very conveniently, and objectively reduces the cost of calibration.

Description

Technical Field

[0001] The present application relates to the field of intelligent driving technology, and in particular to a multi-device calibration method, apparatus, and computer-readable storage medium. Background Art

[0002] When performing data fusion, the calibration of multiple (or multiple) devices is one of the problems that must be solved. Among them, the calibration of each device's own parameters is to ensure the accuracy of the collected data, and the joint calibration of multiple devices is to accurately match the data of multiple devices to ensure the accuracy of data fusion. In the related art, the calibration method for multiple devices is generally selected in a fixed venue (for example, a calibration room) or using a calibration device similar to a chessboard. However, the defects of this method are mainly limited to specific venues and / or specific devices. Therefore, it is not flexible enough, which brings various inconveniences (for example, it is difficult to make an appointment for a specific venue, the specific device is large and inconvenient to carry, etc.). Summary of the invention

[0003] In order to solve or partially solve the problems existing in the related art, the present application provides a multi-device calibration method, apparatus and computer-readable storage medium, which can calibrate multiple devices more conveniently.

[0004] On the one hand, a multi-device calibration method is provided, comprising:

[0005] Obtaining initial calibration parameters of the first device;

[0006] Obtaining initial calibration parameters of the second device;

[0007] Evaluate the accuracy of the first device and the accuracy of the second device according to known parameters of any first marker in any first preset environment, wherein the known parameters include position information of the any first marker in the first preset environment;

[0008] If the accuracy of the first device is higher than that of the second device, the second device is jointly calibrated using the initial calibration parameters of the first device with any second identifier in any second preset environment to obtain final calibration parameters of the second device.

[0009] A second aspect of the present application provides a multi-device calibration device, which is applied to intelligent driving, comprising:

[0010] A first acquisition module, used to acquire initial calibration parameters of the first device;

[0011] A second acquisition module, used to acquire initial calibration parameters of the second device;

[0012] An evaluation module, configured to evaluate the accuracy of the first device and the accuracy of the second device according to known parameters of any first marker in any first preset environment, wherein the known parameters include posture information of the any first marker in the first preset environment;

[0013] A calibration module is used to, if the accuracy of the first device is higher than that of the second device, jointly calibrate the second device using the initial calibration parameters of the first device in any second preset environment to obtain the final calibration parameters of the second device.

[0014] A third aspect of the present application provides an electronic device, including:

[0015] Processor; and

[0016] The memory stores executable codes thereon, and when the executable codes are executed by the processor, the processor is caused to execute the method as described above.

[0017] A fourth aspect of the present application provides a computer-readable storage medium having executable code stored thereon. When the executable code is executed by a processor of an electronic device, the processor is caused to execute the method as described above.

[0018] The technical solution provided by the present application may include the following beneficial effects: whether it is evaluating the accuracy of the first device and the accuracy of the second device, or using the initial calibration parameters of the first device to jointly calibrate the second device to obtain the final calibration parameters of the second device, it can be performed on any identifier in any preset environment, that is, when jointly calibrating multiple devices, no specific place and / or specific equipment is required, and it is not restricted by the environment or conditions. Therefore, multiple (types) of devices can be calibrated very conveniently, which objectively reduces the cost of calibration.

[0019] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The above and other objects, features and advantages of the present application will become more apparent by describing in more detail exemplary embodiments of the present application in conjunction with the accompanying drawings, wherein the same reference numerals generally represent the same components in the exemplary embodiments of the present application.

[0021] Figure 1 is a flowchart of a multi-device calibration method shown in an embodiment of the present application;

[0022] Figure 2 is a schematic diagram of the structure of a multi-device calibration device shown in an embodiment of the present application;

[0023] Figure 3 It is a schematic diagram of the structure of an electronic device shown in an embodiment of the present application. DETAILED DESCRIPTION

[0024] The embodiments of the present application will be described in more detail below with reference to the accompanying drawings. Although the embodiments of the present application are shown in the accompanying drawings, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments described herein. On the contrary, these embodiments are provided to make the present application more thorough and complete, and to fully convey the scope of the present application to those skilled in the art.

[0025] The terms used in this application are for the purpose of describing specific embodiments only and are not intended to limit this application. The singular forms of "a", "said" and "the" used in this application and the appended claims are also intended to include plural forms unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0026] It should be understood that although the terms "first", "second", "third", etc. may be used in this application to describe various information, this information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of this application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of this application, the meaning of "multiple" is two or more, unless otherwise clearly and specifically defined.

[0027] When performing data fusion, in order to ensure the accuracy of the collected data, the calibration of multiple (individual) devices is one of the problems that must be solved, and the joint calibration of multiple devices is also for the accurate matching of multiple device data to ensure the accuracy of data fusion. In the related art, the calibration method for multiple devices is generally selected in a fixed venue (for example, a calibration room) using a calibration device similar to a chessboard. Obviously, the existing multi-device calibration method is limited to a specific venue and / or a specific device, which will bring various inconveniences such as difficulty in reserving a specific venue, large specific devices, and inconvenience in carrying. Therefore, it is not flexible enough.

[0028] In response to the above problems, an embodiment of the present application provides a multi-device calibration method, which can calibrate multiple devices more conveniently.

[0029] The technical solution of the embodiments of the present application is described in detail below with reference to the accompanying drawings.

[0030] See also Figure 1, is a flowchart of a multi-device calibration method shown in an embodiment of the present application, which mainly includes steps S101 to S104, as described below:

[0031] Step S101: Acquire initial calibration parameters of the first device.

[0032] In an embodiment of the present application, the first device may be the most common and important laser sensor in the field of intelligent driving, such as a laser radar, or a visual sensor, such as a monocular camera, a binocular camera, or a depth camera. As for the acquisition of the initial calibration parameters of the first device, it may be the calibration parameters directly read from the history of the first device, or it may be the calibration parameters obtained when the first device is calibrated in a calibration room under offline conditions, or it may be the calibration parameters obtained when calibrating online, for example, the calibration parameters obtained by calibrating the first device when the unmanned vehicle equipped with the first device is driving, etc.; the present application does not limit the method for obtaining the initial calibration parameters of the first device. In addition, the reason why it is called the initial calibration parameter is relative to the final calibration parameter of the device, which means that the initial calibration parameters of the device may have inaccurate defects.

[0033] Step S102: Acquire initial calibration parameters of the second device.

[0034] Similar to the method of obtaining the initial calibration parameters of the first device, the initial calibration parameters of the second device can be obtained by directly reading the historical calibration parameters of the second device, or by calibrating the second device in a calibration room under offline conditions, or by calibrating the second device online, for example, when an unmanned vehicle equipped with the second device calibrates the second device while driving and obtains the calibration parameters, etc. The second device can also be a visual sensor, such as a monocular camera, a binocular camera or a depth camera, or the most common and important laser sensor in the field of intelligent driving, such as a lidar.

[0035] Step S103: Evaluate the accuracy of the first device and the accuracy of the second device based on known parameters of any first marker in any first preset environment, wherein the known parameters include the position information of any first marker in the first preset environment.

[0036] The so-called accuracy of the device can be considered as the accuracy of the device when it performs its function. For example, for a laser radar device, its accuracy is manifested as the accuracy of measuring parameters such as the position and posture of the target. The accuracy of the first device and the accuracy of the second device can be evaluated based on the known parameters of any first marker in any first preset environment. The known parameters of any first marker here include the posture information of the arbitrary first marker in the first preset environment. Taking the first device as a laser radar and the second device as an example, based on the known parameters of any first marker in any first preset environment, the accuracy of the first device and the accuracy of the second device can be evaluated specifically as follows: determine the first parameter of any first marker in any first preset environment based on the posture data obtained by the inertial navigation device and the point cloud data collected by the laser radar; determine the second parameter of any first marker in any first preset environment based on the posture data obtained by the inertial navigation device and the image collected by the camera; if the absolute value of the error between the first parameter and the known parameter is less than the absolute value of the error between the second parameter and the known parameter, it is determined that the accuracy of the first device is higher than the accuracy of the second device. In the above embodiments, the inertial navigation device includes an inertial measurement unit (IMU) and a navigation module, such as a global positioning system (GPS). The former can more accurately measure parameters such as the angular velocity and orientation of the target, while the latter can locate the target.

[0037] As an embodiment of the present application, according to the pose data obtained by the inertial navigation device and the point cloud data collected by the laser radar, determining the first parameter of any first marker in any first preset environment can be achieved through steps S1031 to S1033, which are described in detail as follows:

[0038] Step S1031: Scan any first preset environment through a laser radar to collect point cloud data of the area of ​​interest.

[0039] In the embodiment of the present application, in principle, any area where the first marker may exist can be an area of ​​interest, which can be determined according to the requirements of the perception range. Taking the laser radar mounted on an unmanned vehicle as an example, the area of ​​interest can be within a preset range from the left outer side of the vehicle body to the right outer side of the vehicle body, within a preset range from the front of the vehicle to the rear of the vehicle, within a preset range from the left and right sides of the front of the vehicle body, and / or within a preset range vertically up and down on the front of the laser radar. In order to reduce the unnecessary consumption of computing resources, when collecting point cloud data of the area of ​​interest, point clouds that obviously do not meet the requirements can be filtered out, for example, point clouds whose parameters such as size and coordinates are outside the range of the area of ​​interest.

[0040] Step S1032: By processing the point cloud data, the center coordinates of the first marker in the laser radar coordinate system are detected.

[0041] Specifically, the point cloud cluster features of the target in the region of interest can be obtained through data statistics; the maximum distance D between two points in the point cloud cluster of the target in the region of interest is determined. max And the number of point clouds N of the largest point cloud cluster in the same type of target max ; Filter out the maximum distance between two points is D max And the number of point clouds of the same type of target is N max Among the largest point cloud clusters, the point cloud cluster whose standard deviation of any one-dimensional coordinate of its three-dimensional coordinates is greater than the preset standard deviation threshold, and the point cloud cluster whose absolute difference between the standard deviation of any one-dimensional coordinate of its three-dimensional coordinates and the preset standard deviation threshold is the smallest; calculate the average values ​​of the X coordinate, Y coordinate, and Z coordinate of the above-mentioned retained point cloud clusters as the central coordinates of the first marker in the lidar coordinate system.

[0042] Step S1033: According to the center coordinates of the first marker in the laser radar coordinate system and the posture result determined by the inertial navigation device for the laser radar, the world posture of the first marker is solved as the first parameter of any first marker in any first preset environment.

[0043] In the embodiment of the present application, assuming that the laser radar and the inertial navigation device are installed on the unmanned vehicle, when the inertial navigation device is started, the real-time dynamic (RTK) signal provided by the differential positioning service module can be read and analyzed to obtain the world posture of the unmanned vehicle; the installation parameters of the laser radar are added to the world posture of the unmanned vehicle to obtain the posture result determined by the inertial navigation device for the laser radar; according to the posture result determined by the inertial navigation device for the laser radar, the center coordinates of the first marker in the laser radar coordinate system are converted to the world coordinate system to obtain the world posture of the first marker, that is, the horizontal coordinate, the vertical coordinate and the orientation angle, which are used as the first parameters of any first marker in any first preset environment.

[0044] As an embodiment of the present application, according to the posture data obtained by the inertial navigation device and the image captured by the camera, determining the second parameter of any first marker in any first preset environment can be achieved by steps S'1031 to S'1035, which are described in detail as follows:

[0045] Step S'1031: Acquire inertial navigation device position and posture data corresponding to each marker image in a plurality of marker image frames, wherein the marker image is an image taken by a camera of any first marker in any first preset environment.

[0046] Step S'1032: perform feature detection on the specific mark of each frame of the marker image in the multiple frames of the marker image to obtain the semantic features in each frame of the marker image.

[0047] In an embodiment of the present application, when the camera is mounted on an unmanned vehicle, the marker may be a road sign, a road, a signboard, etc. The specific sign of the marker is a distinguishing feature in the marker that has a specific meaning or a specific form. For example, for a road marker, the specific sign may be a lane line, a street light pole, a traffic sign, a road edge line, a stop line, a ground sign, a traffic light, etc., and for a road sign marker, the specific sign may be the corner points of the four corners of the road sign.

[0048] Step S'1033: According to the semantic features in each frame of the marker image, data matching is performed between the marker images of each frame to obtain matching semantic features belonging to the same specific mark between the marker images of each frame.

[0049] Step S'1034: Determine the first pose of the matching semantic feature in the world coordinate system.

[0050] Step S'1035: Determine the second posture of the matching semantic feature in the world coordinate system as the second parameter of any first marker in any first preset environment according to the first posture and the inertial navigation device posture data corresponding to each frame of the marker image.

[0051] Specifically, the reprojection error between the matching semantic feature and the corresponding semantic feature in each frame of the marker image can be determined based on the first pose and the inertial navigation device pose data corresponding to each frame of the marker image, and the projection error and value between each reprojection error can be determined. Based on the projection error and value, the second pose of the matching semantic feature in the world coordinate system is determined as the second parameter of any first marker in any first preset environment. It should be noted here that the inertial navigation device pose data mainly refers to the angular velocity, orientation and other data of the camera or the first marker obtained by the IMU, and the position and other data of the camera or the first marker obtained by the RTK signal provided by the GPS.

[0052] After determining the first parameter of any first marker in any first preset environment and determining the second parameter of any first marker in any first preset environment, the first parameter is subtracted from the known parameter and the absolute value is taken to obtain the absolute value of the error between the first parameter and the known parameter; similarly, the absolute value of the error between the second parameter and the known parameter can also be obtained; comparing these two absolute values ​​of the error, if the absolute value of the error between the first parameter and the known parameter is smaller than the absolute value of the error between the second parameter and the known parameter, it is determined that the accuracy of the first device is higher than the accuracy of the second device.

[0053] It should be noted that although the above uses the example of the first device being a lidar and the second device being a camera to illustrate the technical solution of evaluating the accuracy of the first device and the accuracy of the second device according to the known parameters of any first marker in any first preset environment, those skilled in the art can understand that when the first device is a camera and the second device is a lidar, the technical solution is similar or basically the same. Therefore, this application no longer uses the example of the first device being a camera and the second device being a lidar to elaborate on the technical solution of evaluating the accuracy of the first device and the accuracy of the second device.

[0054] Step S104: If the accuracy of the first device is higher than that of the second device, the second device is jointly calibrated using the initial calibration parameters of the first device with any second marker in any second preset environment to obtain final calibration parameters of the second device.

[0055] Since the accuracy of the first device is higher than that of the second device, it means that the initial calibration parameters of the first device are more accurate. Therefore, the second device can be jointly calibrated with the initial calibration parameters of the first device by any second marker in any second preset environment to obtain the final calibration parameters of the second device. Still taking the first device as a laser radar and the second device as a camera as an example, in one embodiment of the present application, the final calibration parameters of the second device can be obtained by jointly calibrating the second device with the initial calibration parameters of the first device by any second marker in any second preset environment: extracting a preset number of feature points of any second marker; determining a preset number of first positions P1 of any second marker in any second preset environment by a laser radar and a preset number of second positions P2 of any second marker in any second preset environment by a camera; using the least squares method, solving T W1 ×P1=T′2×T W2 ×P2 holds true for T′2, and T′2×T W2 As the final calibration parameters of the camera, T W1 is the initial calibration parameter of the laser radar, T W2is the initial calibration parameter of the camera, and T′2 is the compensation parameter of the camera. It should be noted that, in this embodiment, the first position P1 of any second marker in any second preset environment is determined by a laser radar. Specifically, a similar technical solution can be adopted in the aforementioned embodiment to determine the first parameter of any first marker in any first preset environment based on the posture data obtained by the inertial navigation device and the point cloud data collected by the laser radar, which will not be repeated here; similarly, the second position P2 of any second marker in any second preset environment is determined by a camera. Specifically, a similar technical solution can be adopted in the aforementioned embodiment to determine the second parameter of any first marker in any first preset environment based on the posture data obtained by the inertial navigation device and the image collected by the camera, so it will not be repeated here, and the reader can refer to the relevant description of the aforementioned embodiment.

[0056] In another embodiment of the present application, the second device is jointly calibrated using the initial calibration parameters of the first device with any second marker in any second preset environment, and the final calibration parameters of the second device can be: when the camera is at a first distance from the second marker, the first projection of the second marker is obtained according to the initial calibration parameters of the camera; when the camera is at a second distance from the second marker, the three-dimensional information of the second marker is obtained according to the initial calibration parameters of the lidar; the second projection of the second marker is determined based on the three-dimensional information of the second marker; and the relative position of the camera relative to the lidar is determined based on the first projection and the second projection; and T′ is solved. w2 =ΔP×T W2 , T′ w2 As the final calibration parameter of the second device, the second projection is the virtual projection of the second marker when the distance between the camera and the target is the first distance, the first distance is greater than the second distance, ΔP is the relative position of the camera relative to the laser radar, T W2 The initial calibration parameters of the camera.

[0057] From the above Figure 1 It can be seen from the example multi-device calibration method that no matter whether it is evaluating the accuracy of the first device and the accuracy of the second device, or using the initial calibration parameters of the first device to jointly calibrate the second device to obtain the final calibration parameters of the second device, it can be performed on any identifier in any preset environment. That is, when jointly calibrating multiple devices, no specific place and / or specific equipment is required, and it is not restricted by the environment or conditions. Therefore, multiple (types) of devices can be calibrated very conveniently, which objectively reduces the cost of calibration.

[0058] Corresponding to the aforementioned application function implementation method embodiment, the present application also provides a multi-device calibration device, an electronic device and corresponding embodiments.

[0059] See also Figure 2, is a schematic diagram of the structure of a multi-device calibration device shown in an embodiment of the present application. For ease of explanation, only the parts related to the embodiment of the present application are shown. Figure 2 The exemplary multi-device calibration device can be applied to intelligent driving. The device mainly includes a first acquisition module 201, a second acquisition module 202, an evaluation module 203 and a calibration module 204, wherein:

[0060] A first acquisition module 201, used to acquire initial calibration parameters of a first device;

[0061] A second acquisition module 202, used to acquire initial calibration parameters of a second device;

[0062] An evaluation module 203, configured to evaluate the accuracy of the first device and the accuracy of the second device according to known parameters of any first marker in any first preset environment, wherein the known parameters include posture information of any first marker in the first preset environment;

[0063] The calibration module 204 is used to, if the accuracy of the first device is higher than that of the second device, jointly calibrate the second device using the initial calibration parameters of the first device in any second preset environment to obtain the final calibration parameters of the second device.

[0064] From the above Figure 2 It can be seen from the example multi-device calibration device that no matter whether it is evaluating the accuracy of the first device and the accuracy of the second device, or using the initial calibration parameters of the first device to jointly calibrate the second device to obtain the final calibration parameters of the second device, it can be performed on any identifier in any preset environment. That is, when jointly calibrating multiple devices, no specific place and / or specific equipment is required, and it is not restricted by the environment or conditions. Therefore, multiple (types) of devices can be calibrated very conveniently, which objectively reduces the cost of calibration.

[0065] Optionally, the first device in the above example is a camera, and the second device is a lidar, or the first device is a lidar, and the second device is a camera.

[0066] Optionally, when the first device is a laser radar and the second device is a camera, Figure 2 The exemplary evaluation module 203 may include a first determination unit, a second determination unit, and a third determination unit, wherein:

[0067] A first determination unit, configured to determine a first parameter of any first marker in any first preset environment according to the posture data obtained by the inertial navigation device and the point cloud data collected by the laser radar;

[0068] A second determination unit is used to determine a second parameter of any first marker in any first preset environment according to the posture data obtained by the inertial navigation device and the image collected by the camera;

[0069] The third determining unit is configured to determine that the accuracy of the first device is higher than the accuracy of the second device if the absolute value of the error between the first parameter and the known parameter is smaller than the absolute value of the error between the second parameter and the known parameter.

[0070] Optionally, the first determination unit in the above example may include a point cloud acquisition unit, a first detection unit and a solution unit, wherein:

[0071] A point cloud acquisition unit, which collects point cloud data of an area of ​​interest by scanning any first preset environment with a laser radar;

[0072] A first detection unit is used to detect and obtain the center coordinates of the first marker in the laser radar coordinate system by processing the point cloud data;

[0073] A solving unit is used to solve the world posture of the first marker as the first parameter of any first marker in any first preset environment according to the central coordinates of the first marker in the laser radar coordinate system and the posture result determined by the inertial navigation device for the laser radar.

[0074] Optionally, the second determination unit in the above example may include a posture data acquisition unit, a second detection unit, a matching unit, a fourth determination unit, and a fifth determination unit, wherein:

[0075] A posture data acquisition unit, used to acquire the posture data of the inertial navigation device corresponding to each frame of the marker image in the multiple frames of the marker image, wherein the marker image is an image taken by a camera of any first marker in any first preset environment;

[0076] A second detection unit is used to perform feature detection on a specific mark of each frame of the marker image in the multiple frames of the marker image to obtain a semantic feature in each frame of the marker image;

[0077] A matching unit, used to perform data matching between the marker images of each frame according to the semantic features in each marker image, and obtain matching semantic features between the marker images of each frame that belong to the same specific mark;

[0078] A fourth determination unit, used to determine the first position of the matching semantic feature in the world coordinate system;

[0079] The fifth determination unit is used to determine the second posture of the matching semantic feature in the world coordinate system as the second parameter of any first marker in any first preset environment based on the first posture and the inertial navigation device posture data corresponding to each frame of the marker image.

[0080] Optionally, the third determining unit in the above example may include an extracting unit, a sixth determining unit, and a first calculating unit, wherein:

[0081] An extraction unit, used to extract a preset number of feature points of any second marker;

[0082] A sixth determining unit, configured to determine, according to a preset number of feature points, a preset number of first positions P1 of any second marker in any second preset environment by a laser radar and a preset number of second positions P2 of any second marker in any second preset environment by a camera;

[0083] The first computing unit is used to use the least square method to solve T W1 ×P1=T′2×T W2 ×P2 holds true for T′2, and T′2×T W2 As the final calibration parameters of the camera, T W1 is the initial calibration parameter of the laser radar, T W2 is the initial calibration parameter of the camera, and T′2 is the compensation parameter of the camera.

[0084] Optionally, the third determination unit in the above example may include a first projection acquisition unit, a three-dimensional information acquisition unit, a second projection determination unit, a relative posture determination unit, and a second calculation unit, wherein:

[0085] A first projection acquisition unit, configured to acquire a first projection of the second marker according to an initial calibration parameter of the camera when the camera is at a first distance from the second marker;

[0086] a three-dimensional information acquisition unit, configured to acquire three-dimensional information of the second marker according to an initial calibration parameter of the laser radar when the camera is at a second distance from the second marker, wherein the first distance is greater than the second distance;

[0087] a second projection determining unit, configured to determine a second projection of the second marker based on the three-dimensional information of the second marker, wherein the second projection is a virtual projection of the second marker when the distance between the camera and the target object is the first distance;

[0088] A relative posture determination unit, used to determine the relative posture of the camera relative to the laser radar based on the first projection and the second projection;

[0089] The second calculation unit is used to solve T' w2 =ΔP×T W2 , the T′ w2 As the final calibration parameters of the second device, ΔP is the relative position of the camera to the lidar, T W2 The initial calibration parameters of the camera.

[0090] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated again here.

[0091] Figure 3 It is a schematic diagram of the structure of an electronic device shown in an embodiment of the present application.

[0092] See also Figure 3 , the electronic device 300 includes a memory 310 and a processor 320.

[0093] The processor 320 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may be any conventional processor, etc.

[0094] The memory 310 may include various types of storage units, such as system memory, read-only memory (ROM), and permanent storage devices. Among them, ROM can store static data or instructions required by the processor 320 or other modules of the computer. The permanent storage device may be a readable and writable storage device. The permanent storage device may be a non-volatile storage device that does not lose the stored instructions and data even after the computer is powered off. In some embodiments, the permanent storage device uses a large-capacity storage device (such as a magnetic or optical disk, flash memory) as a permanent storage device. In some other embodiments, the permanent storage device may be a removable storage device (such as a floppy disk, optical drive). The system memory may be a readable and writable storage device or a volatile readable and writable storage device, such as a dynamic random access memory. The system memory may store some or all instructions and data required by the processor at run time. In addition, the memory 310 may include any combination of computer-readable storage media, including various types of semiconductor memory chips (such as DRAM, SRAM, SDRAM, flash memory, programmable read-only memory), and disks and / or optical disks may also be used. In some embodiments, the memory 310 may include a readable and / or writable removable storage device, such as a laser disc (CD), a read-only digital versatile disc (such as a DVD-ROM, a double-layer DVD-ROM), a read-only Blu-ray disc, an ultra-density optical disc, a flash memory card (such as an SD card, a mini SD card, a Micro-SD card, etc.), a magnetic floppy disk, etc. Computer-readable storage media do not include carrier waves and transient electronic signals transmitted wirelessly or wired.

[0095] The memory 310 stores executable codes, and when the executable codes are processed by the processor 320 , the processor 320 can execute part or all of the methods described above.

[0096] In addition, the method according to the present application may also be implemented as a computer program or a computer program product, which includes computer program code instructions for executing some or all of the steps in the above method of the present application.

[0097] Alternatively, the present application can also be implemented as a computer-readable storage medium (or non-transitory machine-readable storage medium or machine-readable storage medium) on which executable code (or computer program or computer instruction code) is stored. When the executable code (or computer program or computer instruction code) is executed by a processor of an electronic device (or server, etc.), the processor executes part or all of the steps of the above-mentioned method according to the present application.

[0098] The embodiments of the present application have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A multi-device calibration method, characterized in that: The method comprises: Obtaining initial calibration parameters of the first device; Obtaining initial calibration parameters of the second device; Evaluate the accuracy of the first device and the accuracy of the second device according to known parameters of any first marker in any first preset environment, wherein the known parameters include position information of the any first marker in the first preset environment; If the accuracy of the first device is higher than that of the second device, the second device is jointly calibrated using the initial calibration parameters of the first device with any second identifier in any second preset environment to obtain final calibration parameters of the second device.

2. The multi-device calibration method according to claim 1, characterized in that: The first device is a camera and the second device is a lidar, or the first device is a lidar and the second device is a camera.

3. The multi-device calibration method according to claim 2, characterized in that: If the first device is a laser radar and the second device is a camera, the accuracy of the first device and the accuracy of the second device are evaluated according to known parameters of any first marker in any first preset environment, including: Determine a first parameter of any first marker in any first preset environment according to the pose data obtained by the inertial navigation device and the point cloud data collected by the laser radar; Determine a second parameter of any first marker in any first preset environment according to the posture data obtained by the inertial navigation device and the image captured by the camera; If the absolute value of the error between the first parameter and the known parameter is smaller than the absolute value of the error between the second parameter and the known parameter, it is determined that the accuracy of the first device is higher than the accuracy of the second device.

4. The multi-device calibration method according to claim 3, characterized in that: Determining the first parameter of any first marker in any first preset environment according to the posture data obtained by the inertial navigation device and the point cloud data collected by the laser radar includes: Scanning the arbitrary first preset environment by the laser radar to collect point cloud data of the area of ​​interest; By processing the point cloud data, the center coordinates of the first marker in the laser radar coordinate system are detected and obtained; According to the central coordinates of the first marker in the laser radar coordinate system and the posture result determined by the inertial navigation device for the laser radar, the world posture of the first marker is calculated as the first parameter of any first marker in the any first preset environment.

5. The multi-device calibration method according to claim 3, characterized in that: Determining the second parameter of any first marker in any first preset environment according to the posture data obtained by the inertial navigation device and the image captured by the camera includes: Acquire inertial navigation device position and posture data corresponding to each marker image in a plurality of marker images, wherein the marker image is an image captured by the camera of any first marker in the any first preset environment; Performing feature detection on a specific mark of each frame of the marker image in the multiple frames of the marker image to obtain a semantic feature in each frame of the marker image; According to the semantic features in each frame of the marker image, data matching is performed between the marker images of each frame to obtain matching semantic features belonging to the same specific mark between the marker images of each frame; Determine the first position of the matching semantic feature in the world coordinate system; According to the first posture and the inertial navigation device posture data corresponding to each frame of the marker image, the second posture of the matching semantic feature in the world coordinate system is determined as the second parameter of any first marker in any first preset environment.

6. The multi-device calibration method according to claim 2, characterized in that: The method of obtaining the final calibration parameters of the second device by jointly calibrating the second device using the initial calibration parameters of the first device in any second identifier in any second preset environment includes: Extracting a preset number of feature points of the arbitrary second marker; According to the preset number of feature points, determining the preset number of first positions P1 of the arbitrary second marker in any second preset environment by the laser radar and determining the preset number of second positions P2 of the arbitrary second marker in any second preset environment by the camera; Using the least squares method, we can solve = Established , the As the final calibration parameter of the camera, the is the initial calibration parameter of the laser radar, is the initial calibration parameter of the camera, is the compensation parameter of the camera.

7. The multi-device calibration method according to claim 2, characterized in that: The method of obtaining the final calibration parameters of the second device by jointly calibrating the second device using the initial calibration parameters of the first device in any second identifier in any second preset environment includes: When the camera is at a first distance from the second marker, acquiring a first projection of the second marker according to an initial calibration parameter of the camera; When the camera is at a second distance from the second marker, acquiring three-dimensional information of the second marker according to the initial calibration parameters of the laser radar, and the first distance is greater than the second distance; determining a second projection of the second marker based on the three-dimensional information, where the second projection is a virtual projection of the second marker when the distance between the camera and the target object is the first distance; Determine a relative position of the camera relative to the laser radar based on the first projection and the second projection; Solution = , the As the final calibration parameter of the second device, the is the relative position of the camera relative to the laser radar, are the initial calibration parameters of the camera.

8. A multi-device calibration device, characterized in that: The device comprises: A first acquisition module, used to acquire initial calibration parameters of the first device; A second acquisition module, used to acquire initial calibration parameters of the second device; An evaluation module, configured to evaluate the accuracy of the first device and the accuracy of the second device according to known parameters of any first marker in any first preset environment, wherein the known parameters include posture information of the any first marker in the first preset environment; A calibration module is used to, if the accuracy of the first device is higher than that of the second device, jointly calibrate the second device using the initial calibration parameters of the first device in any second preset environment to obtain the final calibration parameters of the second device.

9. An electronic device, characterized in that: include: processor; as well as A memory having executable codes stored thereon, which, when executed by the processor, causes the processor to execute the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having executable codes stored thereon, which, when executed by a processor of an electronic device, causes the processor to execute the method according to any one of claims 1 to 7.

Images