A calibration method, device, movable platform and storage medium

Abstract

An embodiment of the present invention provides a calibration method, device, movable platform and storage medium, wherein the method comprises: (S201) acquiring first point cloud data of the surrounding environment of the movable platform collected by a laser scanning device and image data collected by a camera; (S202) determining second point cloud data based on the first point cloud data, wherein the second point cloud data is used to indicate invalid point cloud data and / or discontinuous point cloud data; (S203) projecting the second point cloud data to a three-dimensional grid space under a camera coordinate system to obtain a projected three-dimensional space; (S204) when each grid area in the projected three-dimensional space meets a preset condition, projecting the projected three-dimensional space onto the image data collected by the camera, and acquiring the optimal position of the projected three-dimensional space projected onto the image data, thereby realizing the calibration of the surrounding environment of the movable platform when there is no specific marker, thereby improving the calibration accuracy.

Description

Technical Field

[0001] The present invention relates to the field of control technology, and in particular to a calibration method, equipment, a movable platform and a storage medium. Background Art

[0002] At present, the calibration methods between LiDAR and camera mainly include external parameter calibration with and without targets. Usually, the external parameter calibration method with a target relies on specific markers such as calibration plates or labels, and the external parameter calibration process is mostly offline. This type of method can achieve high-precision external parameter calibration while relying on specific markers, and the calibration results are more consistent.

[0003] However, the above external parameter calibration method requires specific markers, the calibration process is cumbersome, the applicable scenarios are limited, it is not suitable for outdoor calibration, and it requires relatively dense point cloud data, so it has high performance requirements for the equipment itself. Therefore, how to improve the calibration accuracy and consistency of the calibration results when there are no specific markers has become the focus of research. Summary of the invention

[0004] The embodiments of the present invention provide a calibration method, a device, a movable platform and a storage medium, which realize the calibration of the surrounding environment of the movable platform when there are no specific markers, thereby improving the calibration accuracy.

[0005] In a first aspect, an embodiment of the present invention provides a calibration method, which is applied to a movable platform, on which a laser scanning device and a camera are arranged, and the method comprises:

[0006] Acquire first point cloud data of the surrounding environment of the movable platform collected by a laser scanning device and image data collected by a camera;

[0007] Determining second point cloud data according to the first point cloud data, wherein the second point cloud data is used to indicate invalid point cloud data and / or discontinuous point cloud data;

[0008] Projecting the second point cloud data into a three-dimensional grid space in a camera coordinate system to obtain a projected three-dimensional space;

[0009] When each grid area in the projected three-dimensional space meets a preset condition, the projected three-dimensional space is projected onto the image data collected by the camera, and an optimal position of the projected three-dimensional space projected onto the image data is obtained.

[0010] In a second aspect, an embodiment of the present invention provides a calibration device, including a memory and a processor;

[0011] The memory is used to store programs;

[0012] The processor is used to call the program, and when the program is executed, is used to perform the following operations:

[0013] Acquire first point cloud data of the surrounding environment of the movable platform collected by a laser scanning device and image data collected by a camera;

[0014] Determining second point cloud data according to the first point cloud data, wherein the second point cloud data is used to indicate invalid point cloud data and / or discontinuous point cloud data;

[0015] Projecting the second point cloud data into a three-dimensional grid space in a camera coordinate system to obtain a projected three-dimensional space;

[0016] When each grid area in the projected three-dimensional space meets a preset condition, the projected three-dimensional space is projected onto the image data collected by the camera, and an optimal position of the projected three-dimensional space projected onto the image data is obtained.

[0017] In a third aspect, an embodiment of the present invention provides a movable platform, the movable platform comprising:

[0018] body;

[0019] A power system configured on the fuselage, used to provide power for the movable platform to move;

[0020] The calibration device as described in the second aspect above.

[0021] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the method described in the first aspect above.

[0022] In an embodiment of the present invention, a calibration device obtains first point cloud data of the surrounding environment of a movable platform collected by a laser scanning device and image data collected by a camera, determines second point cloud data based on the first point cloud data, and projects the second point cloud data to a three-dimensional grid space under a camera coordinate system to obtain a projected three-dimensional space. When each grid area in the projected three-dimensional space meets a preset condition, the projected three-dimensional space is projected onto the image data, and the optimal position of the projected three-dimensional space projected onto the image data is obtained, thereby realizing the calibration of the surrounding environment of the movable platform in the absence of specific markers and improving the calibration accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0024] Figure 1 is a structural schematic diagram of a calibration system provided by an embodiment of the present invention;

[0025] Figure 2 is a flow chart of a calibration method provided by an embodiment of the present invention;

[0026] Figure 3 is a schematic diagram of a three-dimensional grid space provided by an embodiment of the present invention;

[0027] Figure 4 is a schematic diagram of a discontinuous point cloud provided by an embodiment of the present invention;

[0028] Figure 5 It is a flowchart of an offline calibration method provided by an embodiment of the present invention;

[0029] Figure 6 It is a flow chart of an online calibration method provided by an embodiment of the present invention;

[0030] Figure 7 It is a structural schematic diagram of a calibration device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0031] The technical solutions in the embodiments of the present invention will be described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0032] Some embodiments of the present invention are described in detail below in conjunction with the accompanying drawings. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0033] The calibration method provided in the embodiment of the present invention can be executed by a calibration system, specifically, can be executed by a calibration device in the calibration system. Wherein, the calibration system includes a calibration device and a movable platform. In some embodiments, the calibration device can be installed on a movable platform; in some embodiments, the calibration device can be spatially independent of the movable platform; in some embodiments, the calibration device can be a component of the movable platform, that is, the movable platform includes the calibration device. In other embodiments, the calibration method can also be applied to other movable devices, such as robots that can move autonomously, unmanned vehicles, unmanned ships and other movable devices.

[0034] The calibration device in the calibration system can obtain the first point cloud data corresponding to the surrounding environment of the movable platform collected by the laser scanning device and the image data collected by the camera; in some embodiments, the laser scanning device and the camera are respectively detachably connected to the movable platform, and in other embodiments, the laser scanning device and the camera can also be fixedly set on the movable platform, which is not limited here. Further, in some embodiments, the laser scanning device includes any one or more of a laser radar, a millimeter wave radar, and an ultrasonic radar; in some embodiments, the first point cloud data can be collected by a laser radar, or can be obtained by a millimeter wave radar, an ultrasonic radar, etc. on a movable platform, which is not specifically limited in the embodiments of the present invention.

[0035] The laser radar is a perception sensor that can obtain three-dimensional information of the scene. Its basic principle is to actively transmit a laser pulse signal to the detected object, obtain the pulse signal reflected back, and calculate the depth information of the distance detector of the detected object based on the time difference between the transmitted signal and the received signal; based on the known emission direction of the laser radar, obtain the angle information of the detected object relative to the laser radar; combine the aforementioned depth and angle information to obtain a large number of detection points (called point cloud), and based on the point cloud, the spatial three-dimensional information of the detected object relative to the laser radar can be reconstructed.

[0036] The present invention provides a method for calibrating a laser radar and a camera in a natural scene that does not rely on specific markers, and also provides a solution for online detection and correction of calibration results. Through the diversity of point cloud distribution in space, sufficient data is collected, and a variety of point cloud features and image information are used for matching to obtain a calibration result with higher calibration accuracy. In some embodiments, this solution can calibrate the camera and laser radar offline; in some embodiments, this solution can also calibrate the camera and laser radar online, and detect the calibration error between the laser radar and the camera to correct the calibration error and improve the calibration accuracy.

[0037] The following is combined with Figure 1The calibration system provided in the embodiment of the present invention is schematically described.

[0038] See also Figure 1 , Figure 1 It is a structural diagram of a calibration system provided by an embodiment of the present invention. The calibration system includes: a calibration device 11 and a movable platform 12. Among them, a communication connection can be established between the movable platform 12 and the calibration device 11 through a wireless communication connection. Among them, in some scenarios, a communication connection can also be established between the movable platform 12 and the calibration device 11 through a wired communication connection. The movable platform 12 can be a movable device such as an unmanned vehicle, an unmanned ship, a movable robot, etc. The movable platform 12 includes a power system 121, and the power system 121 is used to provide power for the movable platform 12 to move. In other embodiments, the movable platform 12 and the calibration device 11 are independent of each other. For example, the calibration device 11 is set in a cloud server and establishes a communication connection with the movable platform 12 through a wireless communication connection.

[0039] In an embodiment of the present invention, the calibration device may acquire first point cloud data of the surrounding environment of the movable platform acquired by the laser scanning device and image data acquired by the camera, and determine second point cloud data based on the first point cloud data, wherein the second point cloud data is used to indicate invalid point cloud data and / or discontinuous point cloud data. The calibration device may project the second point cloud data to a three-dimensional grid space under a camera coordinate system to obtain a projected three-dimensional space, and when each grid area in the projected three-dimensional space meets a preset condition, the projected three-dimensional space is projected onto the image data acquired by the camera, thereby acquiring the optimal position of the projected three-dimensional space projected onto the image data, so as to realize a calibration method that does not rely on a calibration object and improve the consistency of the calibration results.

[0040] The calibration method provided by the embodiment of the present invention is schematically described below with reference to the accompanying drawings.

[0041] For details, please see Figure 2 , Figure 2 1 is a flow chart of a calibration method provided by an embodiment of the present invention, and the method can be executed by a calibration device, wherein the specific explanation of the calibration device is as described above. Specifically, the method of the embodiment of the present invention includes the following steps.

[0042] S201: Acquire first point cloud data of the surrounding environment of the movable platform collected by a laser scanning device and image data collected by a camera.

[0043] In the embodiment of the present invention, the calibration device can obtain the first point cloud data of the surrounding environment of the movable platform collected by the laser scanning device and the image data collected by the camera.

[0044] In some embodiments, the laser scanning device includes any one or more of a laser radar, a millimeter wave radar, and an ultrasonic radar.

[0045] In some embodiments, the camera can be mounted on a movable platform. In some embodiments, the camera can also be independent of the movable platform and installed in the environment where the movable platform is located. In some embodiments, the camera of the camera includes but is not limited to binocular cameras, monocular cameras, TOF cameras and other camera devices.

[0046] In some embodiments, the calibration device can convert the first point cloud data into a camera coordinate system based on a preset transformation matrix to obtain the first point cloud data in the camera coordinate system corresponding to the surrounding environment of the movable platform; wherein the preset transformation matrix includes an intrinsic parameter matrix and an extrinsic parameter matrix, and the extrinsic parameter matrix includes a rotation matrix and / or a translation vector. In some embodiments, when the origin of the camera coordinate system is set on the movable platform, the extrinsic parameter matrix only includes a rotation matrix.

[0047] In some embodiments, the intrinsic parameter matrix is ​​determined based on a plurality of intrinsic parameters, and the intrinsic parameters may be parameters of the camera, such as focal length, image principal point coordinates, etc. In some embodiments, the extrinsic parameter matrix may be parameters obtained by calibrating the camera and the laser scanning device, for example, it may include a rotation matrix and / or a translation vector, wherein the rotation matrix may be determined by the posture of the camera, and the translation vector may be determined by the positioning information of the camera.

[0048] In one embodiment, when the moving speed of the movable platform is less than a preset speed threshold, the calibration device can determine that the movable platform is in an offline low-speed state, and obtain the first point cloud data of the surrounding environment of the movable platform in the offline low-speed state collected by the laser scanning device and the image data collected by the camera to achieve offline calibration. Through offline calibration, sufficient calibration data can be quickly collected at one time, reducing the impact of motion on calibration accuracy and improving calibration accuracy.

[0049] In one embodiment, before acquiring the first point cloud data of the surrounding environment of the movable platform when it is in an offline low-speed state collected by the laser scanning device, the calibration device may establish a three-dimensional grid space relative to the camera coordinate system. After acquiring the first point cloud data of the surrounding environment of the movable platform when it is in an offline low-speed state collected by the laser scanning device, the calibration device may project the first point cloud data onto the coordinate system of the camera through external references. Figure 3 The three-dimensional grid space in the camera coordinate system shown, Figure 3It is a schematic diagram of a three-dimensional grid space provided by an embodiment of the present invention. When the amount of first point cloud data in the three-dimensional point cloud space is greater than a preset amount threshold, it is determined that enough point cloud data has been collected offline at a low speed, and step S202 is executed.

[0050] In one embodiment, the calibration device can determine that the movable platform is in motion when the moving speed of the movable platform is greater than or equal to the preset speed threshold, and obtain the first point cloud data of the surrounding environment of the movable platform in motion collected by the laser scanning device and the image data collected by the camera to achieve online error detection. Through online error detection, calibration data that meets certain scene requirements will be continuously collected during the movement of the movable platform, and whether the current calibration is optimal will be detected. If a better calibration result is continuously found, the current calibration effect will be updated to ensure the consistency of the calibration result.

[0051] S202: Determine second point cloud data according to the first point cloud data, where the second point cloud data is used to indicate invalid point cloud data and / or discontinuous point cloud data.

[0052] In an embodiment of the present invention, the calibration device may determine second point cloud data based on the first point cloud data, where the second point cloud data is used to indicate invalid point cloud data and / or discontinuous point cloud data.

[0053] In one embodiment, the second point cloud data is used to indicate discontinuous point cloud data. Specifically, when the calibration device determines the second point cloud data based on the first point cloud data, it can determine the distance between two adjacent first point cloud data in the first point cloud data, and determine the discontinuous second point cloud data based on the distance between the two adjacent first point cloud data.

[0054] In one embodiment, when the calibration device determines the discontinuous second point cloud data based on the distance between the two adjacent first point cloud data, it can determine whether the distance between the two adjacent first point cloud data is greater than a first preset threshold. When it is determined that the distance between the two adjacent first point cloud data is greater than the first preset threshold, it is determined that the two adjacent first point cloud data are discontinuous second point cloud data.

[0055] In a specific embodiment, due to the working characteristics of the laser radar, the data collected by the laser radar is continuous. If the distance between the two point cloud data changes significantly, it means that this is a depth jump and belongs to discontinuous point cloud data. For example, the distance between the two point clouds can be obtained by a suitable algorithm based on the depth information of the two point cloud data.

[0056] Specific can Figure 4 Take this as an example to illustrate: Figure 4 is a schematic diagram of a discontinuous point cloud provided by an embodiment of the present invention. Figure 4 If the distance between two adjacent first point cloud data point clouds 41 and 42 is greater than a first preset threshold, then the point clouds 41 and 42 can be determined to be discontinuous second point cloud data. For example, the first preset threshold can be a certain value.

[0057] In another embodiment, the distance between the first point cloud data and the origin can also be obtained, and then the distance between the first point cloud data and the origin and the distance between two adjacent first point cloud data can be used to determine whether the two adjacent point cloud data are discontinuous second point cloud data. Specifically, the first point cloud data whose distance from the origin is greater than a preset value can be determined, and from the first point cloud data whose distance from the origin is greater than the preset value, it is determined whether the distance between the two adjacent first point cloud data is greater than a preset distance threshold. When the distance between the two adjacent first point cloud data is greater than the preset distance threshold, the two adjacent first point cloud data are determined to be discontinuous second point cloud data. In some embodiments, due to the influence of the divergence angle, the distance between the two adjacent first point cloud data whose distance from the origin is greater than the preset value can be set as a preset distance threshold. In one embodiment, the preset distance threshold can be a function related to the distance from the origin, for example, when the distance from the origin is farther, the preset distance threshold gradually increases, and when the distance from the origin is closer, the preset distance threshold gradually decreases. In this way, the error caused by the divergence angle can be compensated, the probability of false detection can be reduced, and the calibration accuracy can be improved.

[0058] In one embodiment, the second point cloud data is used to indicate invalid point cloud data. Specifically, when the calibration device determines the second point cloud data based on the first point cloud data, it can determine whether there is depth information in the first point cloud data, and determine the second point cloud data that is invalid in the first point cloud data based on the depth information. Through this implementation, invalid point cloud data can be determined in a scene without radar echo. In some embodiments, the scene without radar echo includes a background of sky, water, etc.

[0059] In one embodiment, when determining the second point cloud data according to the depth information, the calibration device may determine from the first point cloud data that the first point cloud data without depth information is invalid second point cloud data.

[0060] For example, assuming that the background of the first point cloud data collected by the camera and the lidar is the sky, since the lidar actively emits laser pulse signals to the detected object to obtain the pulse signals reflected back, when the background of the lidar collecting the first point cloud data is the sky, there is no detected object in the sky, so the lidar cannot receive the pulse signal returned by the detected object, and thus cannot obtain the depth information of the first point cloud data. Therefore, if the acquired first point cloud data does not have depth information, it can be determined that the first point cloud data is invalid second point cloud data.

[0061] In one embodiment, when the calibration device determines the second point cloud data based on the depth information, it can obtain the change value of the depth information of the first point cloud data. When the change value of the depth information of the first point cloud data is greater than a second preset threshold, it is determined that the first point cloud data corresponding to the depth information greater than the second preset threshold is invalid second point cloud data.

[0062] For example, assuming that the background of the first point cloud data collected by the camera and the laser radar is a scene such as a fence or a bush, since the laser radar will obtain a large amount of depth information with large fluctuations when passing through the fence or the bush, such depth information is invalid point cloud data. When the laser radar passes through the fence or the bush, a lot of first point cloud data is obtained. If the change value of the depth information of the obtained multiple first point cloud data is greater than the second preset threshold, at this time, the depth information of the obtained multiple first point cloud data fluctuates greatly, then it can be determined that the first point cloud data is invalid second point cloud data.

[0063] In another embodiment, the second point cloud data is used to indicate invalid point cloud data and discontinuous point cloud data. Specifically, the method for the calibration device to determine the invalid point cloud data and discontinuous point cloud data in the second point cloud data based on the first point cloud data is as described above and will not be repeated here.

[0064] In one embodiment, before determining the second point cloud data based on the first point cloud data, the calibration device may match the first point cloud data of the current frame acquired with the first point cloud data already acquired, and determine the similarity between the spatial distribution of the first point cloud data of the current frame and the spatial distribution of the first point cloud data already acquired. If the similarity is greater than a preset similarity threshold, the calibration device may delete the first point cloud data of the current frame; if the similarity is less than or equal to the preset similarity threshold, it may be determined to add the first point cloud data of the current frame to the first point cloud data already acquired.

[0065] It can be seen that through this implementation method, it is possible to avoid repeated detection of data of repeated scenes, thereby reducing the amount of invalid point cloud data and improving computing efficiency. The first point cloud data detected for each frame will be compared with the first point cloud data that has been acquired. If the spatial distribution is similar, the first point cloud data of the frame will be deleted, thereby ensuring that the first point cloud data of each frame selected can cover different scenes as much as possible.

[0066] S203: Projecting the second point cloud data to a three-dimensional grid space in a camera coordinate system to obtain a projected three-dimensional space.

[0067] In the embodiment of the present invention, the calibration device may project the second point cloud data into a three-dimensional grid space in a camera coordinate system to obtain a projected three-dimensional space.

[0068] In one embodiment, when the calibration device projects the second point cloud data to the three-dimensional grid space in the camera coordinate system to obtain the projected three-dimensional space, it can determine the relative position information between the laser scanning device and the camera, and project the second point cloud data to the three-dimensional grid space in the camera coordinate system according to the relative position information to obtain the projected three-dimensional space.

[0069] In one embodiment, before the calibration device projects the second point cloud data to the three-dimensional grid space in the camera coordinate system according to the relative position information to obtain the projected three-dimensional space, it can determine the spatial distribution similarity between the second point cloud data and the existing point cloud data in the three-dimensional grid space, and delete the second point cloud data whose spatial distribution similarity is greater than a preset similarity threshold. In this way, redundant point cloud data can be deleted in advance to improve computing efficiency.

[0070] In one embodiment, when the calibration device projects the second point cloud data to the three-dimensional grid space in the camera coordinate system according to the relative position information to obtain the projected three-dimensional space, it can project the deleted second point cloud data to the three-dimensional grid space in the camera coordinate system according to the relative position information to obtain the projected three-dimensional space.

[0071] In one embodiment, when determining the spatial distribution similarity between the second point cloud data and the point cloud data already existing in the three-dimensional grid space, the calibration device can determine the position information of the second point cloud data and the position information of the point cloud data already existing in the three-dimensional grid space, and determine the spatial distribution similarity between the second point cloud data and the point cloud data already existing in the three-dimensional grid space based on the position information of the second point cloud data and the position information of the point cloud data already existing in the three-dimensional grid space.

[0072] In one embodiment, before the calibration device projects the second point cloud data to the three-dimensional grid space in the camera coordinate system, it can determine whether the viewing angle of the camera is smaller than the viewing angle of the laser scanning device. When it is determined that the viewing angle of the camera is smaller than the viewing angle of the laser scanning device, the step of projecting the second point cloud data to the three-dimensional grid space in the camera coordinate system can be executed.

[0073] It can be understood that in one embodiment, the order of step S202 and step S203 can also be swapped. For example, the point cloud data can be first projected into the three-dimensional grid space of the camera, and then the second point cloud data can be determined based on the first point cloud data. The second point cloud data is used to indicate invalid point cloud data and / or discontinuous point cloud data. This is only an exemplary description and is not limited here.

[0074] S204: When each grid area in the projected three-dimensional space meets a preset condition, the projected three-dimensional space is projected onto the image data collected by the camera, and an optimal position of the projected three-dimensional space projected onto the image data is obtained.

[0075] In the embodiment of the present invention, when each grid area in the projected three-dimensional space meets a preset condition, the calibration device can project the projected three-dimensional space onto the image data collected by the camera, and obtain the optimal position of the projected three-dimensional space projected onto the image data. Specifically, at the optimal position, the projected three-dimensional space and the image data position match best.

[0076] In some embodiments, satisfying the preset condition includes that the amount of second point cloud data in each grid area in the projected three-dimensional space is greater than a preset amount threshold.

[0077] In one embodiment, when the calibration device projects the projected three-dimensional space onto the image data collected by the camera and obtains the optimal position of the projected three-dimensional space projected onto the image data, it can determine the gradient image corresponding to the image data according to the image data collected by the camera, and project the second point cloud data of the projected three-dimensional space onto the gradient image. When it is determined that the second point cloud data of the projected three-dimensional space is projected onto the gradient image and the second point cloud data of the projected three-dimensional space is completely integrated with the gradient image, the calibration device can determine the optimal position of the projected three-dimensional space projected onto the image data.

[0078] In one embodiment, when it is determined that the second point cloud data of the projected three-dimensional space is projected onto the gradient image and the second point cloud data of the projected three-dimensional space is completely integrated with the gradient image, the calibration device can determine the optimal position of the projected three-dimensional space projected onto the image data according to the following formula (1).

[0079]

[0080] Among them, D p is the gradient of the corresponding projection point on the image.

[0081] In one embodiment, when the calibration device determines the gradient image corresponding to the image data based on the image data collected by the camera, it can determine the grayscale image corresponding to the image data based on the image data collected by the camera, and extract gradient information and / or edge information from the grayscale image, thereby determining the gradient image based on the gradient information and / or edge information.

[0082] In one embodiment, when the calibration device projects the projected three-dimensional space onto the image data collected by the camera and obtains the optimal position of the projected three-dimensional space projected onto the image data, it can obtain a target image obtained by projecting the projected three-dimensional space onto the image data collected by the camera, and determine the reflectivity of the second point cloud data in the target image, and determine the grayscale value of the grayscale image corresponding to the target image, thereby determining the optimal position of the projected three-dimensional space projected onto the image data based on the reflectivity of the second point cloud data in the target image and the grayscale value of the grayscale image corresponding to the target image.

[0083] In one embodiment, when the calibration device determines the optimal position of the projected three-dimensional space projected onto the image data based on the reflectivity of the second point cloud data in the target image and the grayscale value of the grayscale image corresponding to the target image, the optimal position of the projected three-dimensional space projected onto the image data can be determined according to formula (2).

[0084]

[0085] Among them, I p is the gray value of the corresponding projection point on the image.

[0086] In one embodiment, when the calibration device projects the projected three-dimensional space onto the image data collected by the camera and obtains the optimal position of the projected three-dimensional space projected onto the image data, the motion information of the movable platform during movement can be obtained, and the compensation information of the second point cloud data can be determined based on the motion information, and the second point cloud data in the projected three-dimensional space can be compensated based on the compensation information, so as to project the compensated second point cloud data onto the image data collected by the camera to obtain the optimal position of the projected three-dimensional space projected onto the image data. In some embodiments, the motion information includes any one or more of position information, velocity information, and acceleration information.

[0087] It can be seen that through this implementation method, it is possible to avoid the situation where the accumulated point cloud data becomes blurred due to the movable platform moving too fast, and the consistency between the point cloud data and the image data collected during the movement of the movable platform can be improved by compensating the point cloud data.

[0088] In one embodiment, when the calibration device projects the projected three-dimensional space onto the image data collected by the camera and obtains the optimal position of the projected three-dimensional space projected onto the image data, second point cloud data of the movable platform during the movement of the movable platform within a preset time range can be obtained, and the second point cloud data in the projected three-dimensional space obtained within the preset time range can be projected onto the image data collected by the camera to obtain the optimal position of the projected three-dimensional space projected onto the image data.

[0089] It can be seen that by reducing the accumulation time of the point cloud and using point data of the shortest possible time, the consistency of the point cloud data and the image data collected during the movement of the movable platform can be improved.

[0090] It can be understood that the image data collected by the camera is not limited to grayscale images. This embodiment is only for illustrative purposes and is not limited here. For example, color image data collected by the camera can also be processed. Specifically, specific objects in the scene, such as lane lines, telephone poles, etc., can be first identified through machine learning and other algorithms, and the optimal position of the projected three-dimensional space projected onto the image data can be determined based on the physical information such as the reflectivity and brightness of the identified specific objects. In this way, the probability of false detection can be reduced and the calibration accuracy can be improved.

[0091] In one embodiment, when the calibration device projects the projected three-dimensional space onto the image data collected by the camera and obtains the optimal position of the projected three-dimensional space projected onto the image data, multiple target images obtained by the movable platform projecting the projected three-dimensional space onto the image data collected by the camera during movement can be obtained, and the data of each target image can be compared. If it is determined that the data of each target image is consistent, the position information of the target image can be determined as the optimal position of the projected three-dimensional space projected onto the image data.

[0092] In one embodiment, when the calibration device compares the data of each target image, if it is determined that the data of each target image is inconsistent, it can be determined that the external parameters of the laser scanning device have changed; further, the external parameters of the laser scanning device can be updated.

[0093] In one embodiment, when the calibration device compares the data of each target image, if it is determined that the data of each target image is inconsistent, it can trigger a preset alarm device to sound an alarm to prompt the user that the external parameters of the laser scanning device have changed. Furthermore, it can also prompt the user to check the laser scanning device, or automatically check the laser scanning device, which is not limited here.

[0094] In an embodiment of the present invention, the calibration device obtains first point cloud data of the surrounding environment of the movable platform collected by a laser scanning device and image data collected by a camera, determines second point cloud data based on the first point cloud data, and projects the second point cloud data to a three-dimensional grid space under a camera coordinate system to obtain a projected three-dimensional space. When each grid area in the projected three-dimensional space meets a preset condition, the projected three-dimensional space is projected onto the image data, and the optimal position of the projected three-dimensional space projected onto the image data is obtained, thereby realizing the calibration of the surrounding environment of the movable platform in the absence of specific markers and improving the calibration accuracy.

[0095] The process of offline calibration and online calibration is described below with examples in conjunction with the accompanying drawings.

[0096] See also Figure 5 , Figure 5 is a flow chart of an off-line calibration method provided by an embodiment of the present invention, such as Figure 5As shown, in the process of the offline calibration, the first point cloud data of the surrounding environment of the movable platform is collected by the laser radar, the image data is collected by the camera, the point cloud depth discontinuity is detected according to the first point cloud data, and the point cloud depth discontinuity is determined to be the second point cloud data, the second point cloud data is projected into the three-dimensional grid space to obtain the projected three-dimensional space, the projected three-dimensional space and the existing data are compared for similarity, if similar, the frame data is discarded, if not similar, the data of the projected three-dimensional space is added to the database, and when it is determined that the data in the database is sufficient, the optimal position of the projected three-dimensional space projected onto the image data is obtained.

[0097] See also Figure 6 , Figure 6 is a flow chart of an online calibration method provided by an embodiment of the present invention, such as Figure 6 As shown, the online calibration process includes the offline calibration process. The offline calibration process included in the online calibration process is not described here. The difference between the online calibration process and the offline calibration process is that, in the online calibration process, after obtaining the optimal position of the projection three-dimensional space onto the image data, the optimal position can be subjected to consistency detection. In some embodiments, the consistency detection includes: storing the result of the optimal position in a result queue, and detecting whether each optimal position is consistent according to the multiple optimal positions stored in the result queue, and outputting the detection result, judging whether the optimal position is consistent with the image data according to the detection result, if consistent, it means that the external parameter has changed, and the optimal position needs to be updated, if inconsistent, it means that the structure may be loose and the calibration cannot be completed. Further, when the optimal position is inconsistent with the image data, a preset alarm device can be triggered to alarm to prompt the user that the external parameter of the laser scanning device has changed, or prompt the user to check the laser scanning device, or the laser scanning device can be automatically checked, which is not limited here.

[0098] See also Figure 7 , Figure 7 701 and 702. The calibration device is a schematic diagram of a calibration device provided by an embodiment of the present invention. Specifically, the calibration device includes: a memory 701 and a processor 702.

[0099] In one embodiment, the calibration device further includes a data interface 703, and the data interface 703 is used to transfer data information between the calibration device and other devices.

[0100] The memory 701 may include a volatile memory; the memory 701 may also include a non-volatile memory; the memory 701 may also include a combination of the above-mentioned types of memories. The processor 702 may be a central processing unit (CPU). The processor 702 may further include a hardware chip. The above-mentioned hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD) or a combination thereof. The above-mentioned PLD may be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA) or any combination thereof.

[0101] The memory 701 is used to store programs, and the processor 702 can call the program stored in the memory 701 to perform the following steps:

[0102] Acquire first point cloud data of the surrounding environment of the movable platform collected by the laser scanning device and image data collected by the camera;

[0103] Determining second point cloud data according to the first point cloud data, wherein the second point cloud data is used to indicate invalid point cloud data and / or discontinuous point cloud data;

[0104] Projecting the second point cloud data into a three-dimensional grid space in a camera coordinate system to obtain a projected three-dimensional space;

[0105] When each grid area in the projected three-dimensional space meets a preset condition, the projected three-dimensional space is projected onto the image data collected by the camera, and an optimal position of the projected three-dimensional space projected onto the image data is obtained.

[0106] Further, when the processor 702 determines the second point cloud data according to the first point cloud data, it is specifically used to:

[0107] Determine a distance between two adjacent first point cloud data in the first point cloud data;

[0108] Discontinuous second point cloud data are determined according to the distance between the two adjacent first point cloud data.

[0109] Further, when the processor 702 determines the discontinuous second point cloud data according to the distance between the two adjacent first point cloud data, it is specifically configured to:

[0110] Determine whether the distance between the two adjacent first point cloud data is greater than a first preset threshold;

[0111] When it is determined that the distance between the two adjacent first point cloud data is greater than a first preset threshold, it is determined that the two adjacent first point cloud data are discontinuous second point cloud data.

[0112] Further, the processor 702 is further configured to:

[0113] Acquire the distance between the first point cloud data and the origin;

[0114] The discontinuous second point cloud data are determined according to the distance between the first point cloud data and the origin and the distance between two adjacent first point cloud data.

[0115] Further, when the processor 702 determines the discontinuous second point cloud data according to the distance between the first point cloud data and the origin and the distance between the two adjacent first point cloud data, it is specifically used to:

[0116] Determine the first point cloud data whose distance from the origin is greater than a preset value;

[0117] Determine, from the first point cloud data whose distance from the origin is greater than a preset value, whether the distance between two adjacent first point cloud data is greater than a preset distance threshold;

[0118] If it is greater than the preset distance threshold, it is determined that the two adjacent first point cloud data are discontinuous second point cloud data.

[0119] Further, when the processor 702 determines the second point cloud data according to the first point cloud data, it is specifically used to:

[0120] Determining whether there is depth information in the first point cloud data;

[0121] The second point cloud data that is invalid in the first point cloud data is determined according to the depth information.

[0122] Further, when the processor 702 determines the second point cloud data according to the depth information, it is specifically configured to:

[0123] The first point cloud data without depth information is determined from the first point cloud data as invalid second point cloud data.

[0124] Further, when the processor 702 determines the second point cloud data according to the depth information, it is specifically configured to:

[0125] Obtaining a change value of the depth information of the first point cloud data;

[0126] When the change value of the depth information of the first point cloud data is greater than a second preset threshold, the first point cloud data corresponding to the depth information greater than the second preset threshold is determined to be invalid second point cloud data.

[0127] Furthermore, before determining the second point cloud data according to the first point cloud data, the processor 702 is further configured to:

[0128] Matching the acquired first point cloud data of the current frame with the first point cloud data already acquired;

[0129] Determine the similarity between the spatial distribution of the first point cloud data of the current frame and the spatial distribution of the first point cloud data that has been acquired;

[0130] If the similarity is greater than a preset similarity threshold, deleting the first point cloud data of the current frame;

[0131] If the similarity is less than or equal to the preset similarity threshold, it is determined to add the first point cloud data of the current frame to the first point cloud data that has been acquired.

[0132] Further, the processor 702 projects the second point cloud data into a three-dimensional grid space in a camera coordinate system to obtain a projected three-dimensional space, which is specifically used for:

[0133] Determining relative position information between the laser scanning device and the camera;

[0134] The second point cloud data is projected to a three-dimensional grid space in a camera coordinate system according to the relative position information to obtain a projected three-dimensional space.

[0135] Furthermore, the processor 702 projects the second point cloud data to a three-dimensional grid space in a camera coordinate system according to the relative position information, before obtaining the projected three-dimensional space, and is further used for:

[0136] Determining the spatial distribution similarity between the second point cloud data and the point cloud data already existing in the three-dimensional grid space;

[0137] Deleting the second point cloud data whose spatial distribution similarity is greater than a preset similarity threshold;

[0138] The processor 702 projects the second point cloud data to a three-dimensional grid space in a camera coordinate system according to the relative position information to obtain a projected three-dimensional space, specifically for:

[0139] The deleted second point cloud data is projected to a three-dimensional grid space in a camera coordinate system according to the relative position information to obtain a projected three-dimensional space.

[0140] Further, when the processor 702 determines the spatial distribution similarity between the second point cloud data and the existing point cloud data in the three-dimensional grid space, it is specifically used to:

[0141] Determine the position information of the second point cloud data and the position information of the point cloud data already existing in the three-dimensional grid space;

[0142] The spatial distribution similarity between the second point cloud data and the point cloud data already existing in the three-dimensional grid space is determined according to the position information of the second point cloud data and the position information of the point cloud data already existing in the three-dimensional grid space.

[0143] Furthermore, before the processor 702 projects the second point cloud data into the three-dimensional grid space in the camera coordinate system, it is further configured to:

[0144] determining whether the viewing angle of the camera is smaller than the viewing angle of the laser scanning device;

[0145] When it is determined that the viewing angle of the camera is smaller than the viewing angle of the laser scanning device, the step of projecting the second point cloud data to the three-dimensional grid space in the camera coordinate system is performed.

[0146] Furthermore, the satisfying of the preset conditions includes:

[0147] The amount of second point cloud data in each grid area in the projected three-dimensional space is greater than a preset amount threshold.

[0148] Further, when the processor 702 projects the projected three-dimensional space onto the image data collected by the camera and obtains the optimal position of the projected three-dimensional space projected onto the image data, it is specifically configured to:

[0149] Determining a gradient image corresponding to the image data according to the image data collected by the camera;

[0150] Projecting the second point cloud data of the projected three-dimensional space to the gradient image;

[0151] When it is determined that the second point cloud data of the projected three-dimensional space is projected onto the gradient image and the second point cloud data of the projected three-dimensional space is completely integrated with the gradient image, the optimal position of the projected three-dimensional space projected onto the image data is determined.

[0152] Further, when the processor 702 determines the gradient image corresponding to the image data according to the image data collected by the camera, it is specifically used to:

[0153] Determine, based on the image data collected by the camera, a grayscale image corresponding to the image data;

[0154] Extracting gradient information and / or edge information from the grayscale image;

[0155] The gradient image is determined according to the gradient information and / or the edge information.

[0156] Further, when the processor 702 projects the projected three-dimensional space onto the image data collected by the camera and obtains the optimal position of the projected three-dimensional space projected onto the image data, it is specifically configured to:

[0157] Acquire a target image obtained by projecting the projected three-dimensional space onto the image data collected by the camera;

[0158] determining a reflectivity of second point cloud data in the target image;

[0159] Determining a grayscale value of a grayscale image corresponding to the target image;

[0160] An optimal position where the projected three-dimensional space is projected onto the image data is determined according to the reflectivity of the second point cloud data in the target image and the grayscale value of the grayscale image corresponding to the target image.

[0161] Further, when the processor 702 projects the projected three-dimensional space onto the image data collected by the camera and obtains the optimal position of the projected three-dimensional space projected onto the image data, it is specifically configured to:

[0162] Acquiring motion information of the movable platform during movement;

[0163] Determining compensation information of the second point cloud data according to the motion information;

[0164] Compensating the second point cloud data in the projected three-dimensional space according to the compensation information;

[0165] The compensated second point cloud data is projected onto the image data collected by the camera to obtain an optimal position of the projected three-dimensional space projected onto the image data.

[0166] Furthermore, the motion information includes any one or more of position information, speed information, and acceleration information.

[0167] Further, when the processor 702 projects the projected three-dimensional space onto the image data collected by the camera and obtains the optimal position of the projected three-dimensional space projected onto the image data, it is specifically configured to:

[0168] Acquire second point cloud data during the movement of the movable platform within a preset time range;

[0169] The second point cloud data in the projected three-dimensional space acquired within the preset time range is projected onto the image data collected by the camera to obtain an optimal position of the projected three-dimensional space projected onto the image data.

[0170] Further, when the processor 702 projects the projected three-dimensional space onto the image data collected by the camera and obtains the optimal position of the projected three-dimensional space projected onto the image data, it is specifically configured to:

[0171] Acquire a plurality of target images obtained by projecting the projection three-dimensional space onto the image data collected by the camera during the movement of the movable platform;

[0172] Compare the data of each target image;

[0173] If it is determined that the data of each target image is consistent, the position information of the target image is determined to be the optimal position of the projected three-dimensional space projected onto the image data.

[0174] Further, the processor 702 is further configured to:

[0175] If it is determined that the data of each target image is inconsistent, it is determined that the external parameters of the laser scanning device have changed, and the external parameters of the laser scanning device are updated.

[0176] Further, the method of the processor 702 is also used for:

[0177] If it is determined that the data of each target image is inconsistent, a preset alarm device is triggered to sound an alarm to prompt the user to check the laser scanning device.

[0178] Furthermore, the laser scanning device includes any one or more of a laser radar, a millimeter wave radar, and an ultrasonic radar.

[0179] In an embodiment of the present invention, a calibration device obtains first point cloud data of the surrounding environment of a movable platform collected by a laser scanning device and image data collected by a camera, determines second point cloud data based on the first point cloud data, and projects the second point cloud data to a three-dimensional grid space under a camera coordinate system to obtain a projected three-dimensional space. When each grid area in the projected three-dimensional space meets a preset condition, the projected three-dimensional space is projected onto the image data, and the optimal position of the projected three-dimensional space projected onto the image data is obtained, thereby realizing the calibration of the surrounding environment of the movable platform in the absence of specific markers and improving the calibration accuracy.

[0180] The embodiment of the present invention also provides a movable platform, which includes: a fuselage; a power system configured on the fuselage, used to provide power for the movable platform to move; and the above-mentioned calibration device. In the embodiment of the present invention, the movable platform obtains the first point cloud data of the surrounding environment of the movable platform collected by the laser scanning device and the image data collected by the camera, and determines the second point cloud data according to the first point cloud data, and projects the second point cloud data to the three-dimensional grid space under the camera coordinate system to obtain the projected three-dimensional space. When each grid area in the projected three-dimensional space meets the preset conditions, the projected three-dimensional space is projected onto the image data, and the optimal position of the projected three-dimensional space projected onto the image data is obtained, thereby realizing the calibration of the surrounding environment of the movable platform in the absence of specific markers and improving the calibration accuracy.

[0181] The embodiment of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the present invention is implemented. Figure 2 The method described in the corresponding embodiment can also be implemented Figure 7 The equipment corresponding to the embodiment of the present invention will not be described in detail here.

[0182] The computer-readable storage medium may be an internal storage unit of the device described in any of the foregoing embodiments, such as a hard disk or memory of the device. The computer-readable storage medium may also be an external storage device of the device, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the device. Furthermore, the computer-readable storage medium may also include both an internal storage unit of the device and an external storage device. The computer-readable storage medium is used to store the computer program and other programs and data required by the terminal. The computer-readable storage medium may also be used to temporarily store data that has been output or is to be output.

[0183] The above disclosure is only part of the embodiments of the present invention, which certainly cannot be used to limit the scope of the present invention. Therefore, equivalent changes made according to the claims of the present invention are still within the scope of the present invention.

Claims

1. A calibration method, characterized in that: Applied to a movable platform, the movable platform comprising a laser scanning device and a camera, the method comprising: Acquire first point cloud data of the surrounding environment of the movable platform collected by the laser scanning device and image data collected by the camera; Determining second point cloud data according to the first point cloud data, wherein the second point cloud data is used to indicate invalid point cloud data and / or discontinuous point cloud data; Projecting the second point cloud data into a three-dimensional grid space in a camera coordinate system to obtain a projected three-dimensional space; When each grid area in the projected three-dimensional space meets a preset condition, the projected three-dimensional space is projected onto the image data collected by the camera, and an optimal position of the projected three-dimensional space projected onto the image data is obtained.

2. The method according to claim 1, characterized in that The determining the second point cloud data according to the first point cloud data comprises: Determine a distance between two adjacent first point cloud data in the first point cloud data; Discontinuous second point cloud data are determined according to the distance between the two adjacent first point cloud data.

3. The method according to claim 2, characterized in that The step of determining discontinuous second point cloud data according to the distance between the two adjacent first point cloud data comprises: Determine whether the distance between the two adjacent first point cloud data is greater than a first preset threshold; When it is determined that the distance between the two adjacent first point cloud data is greater than a first preset threshold, it is determined that the two adjacent first point cloud data are discontinuous second point cloud data.

4. The method according to claim 2, characterized in that: The method further comprises: Acquire the distance between the first point cloud data and the origin; The discontinuous second point cloud data are determined according to the distance between the first point cloud data and the origin and the distance between two adjacent first point cloud data.

5. The method according to claim 4, characterized in that The determining the discontinuous second point cloud data according to the distance between the first point cloud data and the origin and the distance between two adjacent first point cloud data comprises: Determine the first point cloud data whose distance from the origin is greater than a preset value; Determine, from the first point cloud data whose distance from the origin is greater than a preset value, whether the distance between two adjacent first point cloud data is greater than a preset distance threshold; If it is greater than the preset distance threshold, it is determined that the two adjacent first point cloud data are discontinuous second point cloud data.

6. The method according to claim 1, characterized in that The determining the second point cloud data according to the first point cloud data comprises: Determining whether there is depth information in the first point cloud data; The second point cloud data that is invalid in the first point cloud data is determined according to the depth information.

7. The method according to claim 6, characterized in that The determining the second point cloud data according to the depth information includes: The first point cloud data without depth information is determined from the first point cloud data as invalid second point cloud data.

8. The method according to claim 6, characterized in that The determining the second point cloud data according to the depth information includes: Obtaining a change value of the depth information of the first point cloud data; When the change value of the depth information of the first point cloud data is greater than a second preset threshold, the first point cloud data corresponding to the depth information greater than the second preset threshold is determined to be invalid second point cloud data.

9. The method according to claim 1, characterized in that: Before determining the second point cloud data according to the first point cloud data, the method further includes: Matching the acquired first point cloud data of the current frame with the first point cloud data already acquired; Determine the similarity between the spatial distribution of the first point cloud data of the current frame and the spatial distribution of the first point cloud data that has been acquired; If the similarity is greater than a preset similarity threshold, deleting the first point cloud data of the current frame; If the similarity is less than or equal to the preset similarity threshold, it is determined to add the first point cloud data of the current frame to the first point cloud data that has been acquired.

10. The method according to claim 1, characterized in that The step of projecting the second point cloud data to a three-dimensional grid space in a camera coordinate system to obtain a projected three-dimensional space includes: Determining relative position information between the laser scanning device and the camera; The second point cloud data is projected to a three-dimensional grid space in a camera coordinate system according to the relative position information to obtain a projected three-dimensional space.

11. The method according to claim 10, characterized in that Before projecting the second point cloud data to a three-dimensional grid space in a camera coordinate system according to the relative position information to obtain a projected three-dimensional space, the method further includes: Determining the spatial distribution similarity between the second point cloud data and the point cloud data already existing in the three-dimensional grid space; Deleting the second point cloud data whose spatial distribution similarity is greater than a preset similarity threshold; The step of projecting the second point cloud data to a three-dimensional grid space in a camera coordinate system according to the relative position information to obtain a projected three-dimensional space includes: The deleted second point cloud data is projected to the three-dimensional grid space in the camera coordinate system according to the relative position information to obtain a projected three-dimensional space.

12. The method according to claim 11, characterized in that The determining of the spatial distribution similarity between the second point cloud data and the existing point cloud data in the three-dimensional grid space includes: Determine the position information of the second point cloud data and the position information of the point cloud data already existing in the three-dimensional grid space; The spatial distribution similarity between the second point cloud data and the point cloud data already existing in the three-dimensional grid space is determined according to the position information of the second point cloud data and the position information of the point cloud data already existing in the three-dimensional grid space.

13. The method according to claim 10, characterized in that Before projecting the second point cloud data to the three-dimensional grid space in the camera coordinate system, the method further includes: determining whether the viewing angle of the camera is smaller than the viewing angle of the laser scanning device; When it is determined that the viewing angle of the camera is smaller than the viewing angle of the laser scanning device, the step of projecting the second point cloud data to the three-dimensional grid space in the camera coordinate system is performed.

14. The method according to claim 1, characterized in that The satisfying of the preset conditions includes: The amount of second point cloud data in each grid area in the projected three-dimensional space is greater than a preset amount threshold.

15. The method according to claim 1, characterized in that The projecting the projected three-dimensional space onto the image data collected by the camera, and obtaining the optimal position of the projected three-dimensional space projected onto the image data, comprises: Determining a gradient image corresponding to the image data according to the image data collected by the camera; Projecting the second point cloud data of the projected three-dimensional space to the gradient image; When it is determined that the second point cloud data of the projected three-dimensional space is projected onto the gradient image and the second point cloud data of the projected three-dimensional space is completely integrated with the gradient image, the optimal position of the projected three-dimensional space projected onto the image data is determined.

16. The method according to claim 15, characterized in that The step of determining a gradient image corresponding to the image data according to the image data collected by the camera includes: Determine, based on the image data collected by the camera, a grayscale image corresponding to the image data; Extracting gradient information and / or edge information from the grayscale image; The gradient image is determined according to the gradient information and / or the edge information.

17. The method according to claim 1, characterized in that The projecting the projected three-dimensional space onto the image data collected by the camera, and obtaining the optimal position of the projected three-dimensional space projected onto the image data, comprises: Acquire a target image obtained by projecting the projected three-dimensional space onto the image data collected by the camera; determining a reflectivity of second point cloud data in the target image; Determining a grayscale value of a grayscale image corresponding to the target image; An optimal position where the projected three-dimensional space is projected onto the image data is determined according to the reflectivity of the second point cloud data in the target image and the grayscale value of the grayscale image corresponding to the target image.

18. The method according to claim 1, characterized in that The projecting the projected three-dimensional space onto the image data collected by the camera, and obtaining the optimal position of the projected three-dimensional space projected onto the image data, comprises: Acquiring motion information of the movable platform during movement; Determining compensation information of the second point cloud data according to the motion information; Compensating the second point cloud data in the projected three-dimensional space according to the compensation information; The compensated second point cloud data is projected onto the image data collected by the camera to obtain an optimal position of the projected three-dimensional space projected onto the image data.

19. The method according to claim 18, characterized in that The motion information includes any one or more of position information, speed information, and acceleration information.

20. The method according to claim 1, characterized in that The projecting the projected three-dimensional space onto the image data collected by the camera, and obtaining the optimal position of the projected three-dimensional space projected onto the image data, comprises: Acquire second point cloud data during the movement of the movable platform within a preset time range; The second point cloud data in the projected three-dimensional space acquired within the preset time range is projected onto the image data collected by the camera to obtain an optimal position of the projected three-dimensional space projected onto the image data.

21. The method according to claim 18 or 20, characterized in that The projecting the projected three-dimensional space onto the image data collected by the camera, and obtaining the optimal position of the projected three-dimensional space projected onto the image data, comprises: Acquire a plurality of target images obtained by projecting the projection three-dimensional space onto the image data collected by the camera during the movement of the movable platform; Compare the data of each target image; If it is determined that the data of each target image is consistent, the position information of the target image is determined to be the optimal position of the projected three-dimensional space projected onto the image data.

22. The method according to claim 21, characterized in that The method further comprises: If it is determined that the data of each target image is inconsistent, it is determined that the external parameters of the laser scanning device have changed, and the external parameters of the laser scanning device are updated.

23. The method according to claim 21, characterized in that The method further comprises: If it is determined that the data of each target image is inconsistent, a preset alarm device is triggered to sound an alarm to prompt the user to check the laser scanning device.

24. The method according to claim 1, characterized in that The laser scanning device includes any one or more of a laser radar, a millimeter wave radar, and an ultrasonic radar.

25. A calibration device, characterized in that: Applied to a movable platform, the movable platform includes a laser scanning device and a camera, and the device includes a memory and a processor; The memory is used to store programs; The processor is used to call the program, and when the program is executed, is used to perform the following operations: Acquire first point cloud data of the surrounding environment of the movable platform collected by the laser scanning device and image data collected by the camera; Determining second point cloud data according to the first point cloud data, wherein the second point cloud data is used to indicate invalid point cloud data and / or discontinuous point cloud data; Projecting the second point cloud data into a three-dimensional grid space in a camera coordinate system to obtain a projected three-dimensional space; When each grid area in the projected three-dimensional space meets a preset condition, the projected three-dimensional space is projected onto the image data collected by the camera, and an optimal position of the projected three-dimensional space projected onto the image data is obtained.

26. The device according to claim 25, characterized in that When the processor determines the second point cloud data according to the first point cloud data, it is specifically used to: Determine a distance between two adjacent first point cloud data in the first point cloud data; Discontinuous second point cloud data are determined according to the distance between the two adjacent first point cloud data.

27. The device according to claim 26, characterized in that When the processor determines the discontinuous second point cloud data according to the distance between the two adjacent first point cloud data, it is specifically used to: Determine whether the distance between the two adjacent first point cloud data is greater than a first preset threshold; When it is determined that the distance between the two adjacent first point cloud data is greater than a first preset threshold, it is determined that the two adjacent first point cloud data are discontinuous second point cloud data.

28. The device according to claim 26, characterized in that The processor is further configured to: Acquire the distance between the first point cloud data and the origin; The discontinuous second point cloud data are determined according to the distance between the first point cloud data and the origin and the distance between two adjacent first point cloud data.

29. The device according to claim 28, characterized in that When the processor determines the discontinuous second point cloud data according to the distance between the first point cloud data and the origin and the distance between the two adjacent first point cloud data, it is specifically used to: Determine the first point cloud data whose distance from the origin is greater than a preset value; Determine, from the first point cloud data whose distance from the origin is greater than a preset value, whether the distance between two adjacent first point cloud data is greater than a preset distance threshold; If it is greater than the preset distance threshold, it is determined that the two adjacent first point cloud data are discontinuous second point cloud data.

30. The apparatus according to claim 25, characterized in that When the processor determines the second point cloud data according to the first point cloud data, it is specifically used to: Determining whether there is depth information in the first point cloud data; The second point cloud data that is invalid in the first point cloud data is determined according to the depth information.

31. The device according to claim 30, characterized in that When the processor determines the second point cloud data according to the depth information, it is specifically used to: The first point cloud data without depth information is determined from the first point cloud data as invalid second point cloud data.

32. The device according to claim 30, characterized in that When the processor determines the second point cloud data according to the depth information, it is specifically used to: Obtaining a change value of the depth information of the first point cloud data; When the change value of the depth information of the first point cloud data is greater than a second preset threshold, the first point cloud data corresponding to the depth information greater than the second preset threshold is determined to be invalid second point cloud data.

33. The device according to claim 25, characterized in that Before the processor determines the second point cloud data according to the first point cloud data, the processor is further configured to: Matching the acquired first point cloud data of the current frame with the first point cloud data already acquired; Determine the similarity between the spatial distribution of the first point cloud data of the current frame and the spatial distribution of the first point cloud data that has been acquired; If the similarity is greater than a preset similarity threshold, deleting the first point cloud data of the current frame; If the similarity is less than or equal to the preset similarity threshold, it is determined to add the first point cloud data of the current frame to the first point cloud data that has been acquired.

34. The device according to claim 25, characterized in that When the processor projects the second point cloud data into a three-dimensional grid space in a camera coordinate system to obtain a projected three-dimensional space, the processor is specifically used to: Determining relative position information between the laser scanning device and the camera; The second point cloud data is projected to a three-dimensional grid space in a camera coordinate system according to the relative position information to obtain a projected three-dimensional space.

35. The device according to claim 34, characterized in that The processor projects the second point cloud data to a three-dimensional grid space in a camera coordinate system according to the relative position information, before obtaining the projected three-dimensional space, and is further used for: Determining the spatial distribution similarity between the second point cloud data and the point cloud data already existing in the three-dimensional grid space; Deleting the second point cloud data whose spatial distribution similarity is greater than a preset similarity threshold; The processor projects the second point cloud data to a three-dimensional grid space in a camera coordinate system according to the relative position information to obtain a projected three-dimensional space, specifically for: The deleted second point cloud data is projected to the three-dimensional grid space in the camera coordinate system according to the relative position information to obtain a projected three-dimensional space.

36. The device according to claim 35, characterized in that When the processor determines the spatial distribution similarity between the second point cloud data and the existing point cloud data in the three-dimensional grid space, it is specifically used to: Determine the position information of the second point cloud data and the position information of the point cloud data already existing in the three-dimensional grid space; The spatial distribution similarity between the second point cloud data and the point cloud data already existing in the three-dimensional grid space is determined according to the position information of the second point cloud data and the position information of the point cloud data already existing in the three-dimensional grid space.

37. The device according to claim 34, characterized in that Before the processor projects the second point cloud data into the three-dimensional grid space in the camera coordinate system, the processor is further configured to: determining whether the viewing angle of the camera is smaller than the viewing angle of the laser scanning device; When it is determined that the viewing angle of the camera is smaller than the viewing angle of the laser scanning device, the step of projecting the second point cloud data to the three-dimensional grid space in the camera coordinate system is performed.

38. The apparatus according to claim 25, characterized in that The satisfying of the preset conditions includes: The amount of second point cloud data in each grid area in the projected three-dimensional space is greater than a preset amount threshold.

39. The apparatus according to claim 25, characterized in that When the processor projects the projected three-dimensional space onto the image data collected by the camera and obtains the optimal position of the projected three-dimensional space projected onto the image data, it is specifically used to: Determining a gradient image corresponding to the image data according to the image data collected by the camera; Projecting the second point cloud data of the projected three-dimensional space to the gradient image; When it is determined that the second point cloud data of the projected three-dimensional space is projected onto the gradient image and the second point cloud data of the projected three-dimensional space is completely integrated with the gradient image, the optimal position of the projected three-dimensional space projected onto the image data is determined.

40. The device according to claim 39, characterized in that When the processor determines the gradient image corresponding to the image data according to the image data collected by the camera, it is specifically used to: Determine, based on the image data collected by the camera, a grayscale image corresponding to the image data; Extracting gradient information and / or edge information from the grayscale image; The gradient image is determined according to the gradient information and / or the edge information.

41. The apparatus according to claim 25, characterized in that When the processor projects the projected three-dimensional space onto the image data collected by the camera and obtains the optimal position of the projected three-dimensional space projected onto the image data, it is specifically used to: Acquire a target image obtained by projecting the projected three-dimensional space onto the image data collected by the camera; determining a reflectivity of second point cloud data in the target image; Determining a grayscale value of a grayscale image corresponding to the target image; An optimal position where the projected three-dimensional space is projected onto the image data is determined according to the reflectivity of the second point cloud data in the target image and the grayscale value of the grayscale image corresponding to the target image.

42. The apparatus according to claim 26, characterized in that When the processor projects the projected three-dimensional space onto the image data collected by the camera and obtains the optimal position of the projected three-dimensional space projected onto the image data, it is specifically used to: Acquiring motion information of the movable platform during movement; Determining compensation information of the second point cloud data according to the motion information; Compensating the second point cloud data in the projected three-dimensional space according to the compensation information; The compensated second point cloud data is projected onto the image data collected by the camera to obtain an optimal position of the projected three-dimensional space projected onto the image data.

43. The device according to claim 42, characterized in that The motion information includes any one or more of position information, speed information, and acceleration information.

44. The apparatus according to claim 25, characterized in that When the processor projects the projected three-dimensional space onto the image data collected by the camera and obtains the optimal position of the projected three-dimensional space projected onto the image data, it is specifically used to: Acquire second point cloud data during the movement of the movable platform within a preset time range; The second point cloud data in the projected three-dimensional space acquired within the preset time range is projected onto the image data collected by the camera to obtain an optimal position of the projected three-dimensional space projected onto the image data.

45. Apparatus according to claim 42 or 44, characterised in that When the processor projects the projected three-dimensional space onto the image data collected by the camera and obtains the optimal position of the projected three-dimensional space projected onto the image data, it is specifically used to: Acquire a plurality of target images obtained by projecting the projection three-dimensional space onto the image data collected by the camera during the movement of the movable platform; Compare the data of each target image; If it is determined that the data of each target image is consistent, the position information of the target image is determined to be the optimal position of the projected three-dimensional space projected onto the image data.

46. ​​The device according to claim 45, characterized in that The processor is further configured to: If it is determined that the data of each target image is inconsistent, it is determined that the external parameters of the laser scanning device have changed, and the external parameters of the laser scanning device are updated.

47. The device according to claim 45, characterized in that The processor is further configured to: If it is determined that the data of each target image is inconsistent, a preset alarm device is triggered to sound an alarm to prompt the user to check the laser scanning device.

48. The apparatus according to claim 25, characterized in that The laser scanning device includes any one or more of a laser radar, a millimeter wave radar, and an ultrasonic radar.

49. A movable platform, characterized in that: include: body; A power system configured on the fuselage, used to provide power for the movable platform to move; A processor, configured to obtain first point cloud data of the surrounding environment of the movable platform collected by a laser scanning device and image data collected by a camera; determine second point cloud data according to the first point cloud data, wherein the second point cloud data is used to indicate invalid point cloud data and / or discontinuous point cloud data; Projecting the second point cloud data into a three-dimensional grid space in a camera coordinate system to obtain a projected three-dimensional space; When each grid area in the projected three-dimensional space meets a preset condition, the projected three-dimensional space is projected onto the image data collected by the camera, and an optimal position of the projected three-dimensional space projected onto the image data is obtained.

50. The movable platform according to claim 49, characterized in that: When the processor determines the second point cloud data according to the first point cloud data, it is specifically used to: Determine a distance between two adjacent first point cloud data in the first point cloud data; Discontinuous second point cloud data are determined according to the distance between the two adjacent first point cloud data.

51. The movable platform according to claim 50, characterized in that: When the processor determines the discontinuous second point cloud data according to the distance between the two adjacent first point cloud data, it is specifically used to: Determine whether the distance between the two adjacent first point cloud data is greater than a first preset threshold; When it is determined that the distance between the two adjacent first point cloud data is greater than a first preset threshold, it is determined that the two adjacent first point cloud data are discontinuous second point cloud data.

52. The movable platform according to claim 50, characterized in that: The processor is further configured to: Acquire the distance between the first point cloud data and the origin; The discontinuous second point cloud data are determined according to the distance between the first point cloud data and the origin and the distance between two adjacent first point cloud data.

53. The movable platform according to claim 52, characterized in that: When the processor determines the discontinuous second point cloud data according to the distance between the first point cloud data and the origin and the distance between the two adjacent first point cloud data, it is specifically used to: Determine the first point cloud data whose distance from the origin is greater than a preset value; Determine, from the first point cloud data whose distance from the origin is greater than a preset value, whether the distance between two adjacent first point cloud data is greater than a preset distance threshold; If it is greater than the preset distance threshold, it is determined that the two adjacent first point cloud data are discontinuous second point cloud data.

54. The movable platform according to claim 49, characterized in that: When the processor determines the second point cloud data according to the first point cloud data, it is specifically used to: Determining whether there is depth information in the first point cloud data; The second point cloud data that is invalid in the first point cloud data is determined according to the depth information.

55. The movable platform according to claim 54, characterized in that When the processor determines the second point cloud data according to the depth information, it is specifically used to: The first point cloud data without depth information is determined from the first point cloud data as invalid second point cloud data.

56. The movable platform according to claim 54, characterized in that When the processor determines the second point cloud data according to the depth information, it is specifically used to: Obtaining a change value of the depth information of the first point cloud data; When the change value of the depth information of the first point cloud data is greater than a second preset threshold, the first point cloud data corresponding to the depth information greater than the second preset threshold is determined to be invalid second point cloud data.

57. The movable platform according to claim 49, characterized in that: Before the processor determines the second point cloud data according to the first point cloud data, the processor is further configured to: Matching the acquired first point cloud data of the current frame with the first point cloud data already acquired; Determine the similarity between the spatial distribution of the first point cloud data of the current frame and the spatial distribution of the first point cloud data that has been acquired; If the similarity is greater than a preset similarity threshold, deleting the first point cloud data of the current frame; If the similarity is less than or equal to the preset similarity threshold, it is determined to add the first point cloud data of the current frame to the first point cloud data that has been acquired.

58. The movable platform according to claim 49, characterized in that: When the processor projects the second point cloud data into a three-dimensional grid space in a camera coordinate system to obtain a projected three-dimensional space, the processor is specifically used to: Determining relative position information between the laser scanning device and the camera; The second point cloud data is projected to a three-dimensional grid space in a camera coordinate system according to the relative position information to obtain a projected three-dimensional space.

59. The movable platform according to claim 58, characterized in that The processor projects the second point cloud data to a three-dimensional grid space in a camera coordinate system according to the relative position information, before obtaining the projected three-dimensional space, and is further used for: Determining the spatial distribution similarity between the second point cloud data and the point cloud data already existing in the three-dimensional grid space; Deleting the second point cloud data whose spatial distribution similarity is greater than a preset similarity threshold; The processor projects the second point cloud data to a three-dimensional grid space in a camera coordinate system according to the relative position information to obtain a projected three-dimensional space, specifically for: The deleted second point cloud data is projected to the three-dimensional grid space in the camera coordinate system according to the relative position information to obtain a projected three-dimensional space.

60. The movable platform according to claim 59, characterized in that: When the processor determines the spatial distribution similarity between the second point cloud data and the existing point cloud data in the three-dimensional grid space, it is specifically used to: Determine the position information of the second point cloud data and the position information of the point cloud data already existing in the three-dimensional grid space; The spatial distribution similarity between the second point cloud data and the point cloud data already existing in the three-dimensional grid space is determined according to the position information of the second point cloud data and the position information of the point cloud data already existing in the three-dimensional grid space.

61. The movable platform according to claim 60, characterized in that: Before the processor projects the second point cloud data into the three-dimensional grid space in the camera coordinate system, the processor is further configured to: determining whether the viewing angle of the camera is smaller than the viewing angle of the laser scanning device; When it is determined that the viewing angle of the camera is smaller than the viewing angle of the laser scanning device, the step of projecting the second point cloud data to the three-dimensional grid space in the camera coordinate system is performed.

62. The movable platform according to claim 49, characterized in that: The satisfying of the preset conditions includes: The amount of second point cloud data in each grid area in the projected three-dimensional space is greater than a preset amount threshold.

63. The movable platform according to claim 49, characterized in that: When the processor projects the projected three-dimensional space onto the image data collected by the camera and obtains the optimal position of the projected three-dimensional space projected onto the image data, it is specifically used to: Determining a gradient image corresponding to the image data according to the image data collected by the camera; Projecting the second point cloud data of the projected three-dimensional space to the gradient image; When it is determined that the second point cloud data of the projected three-dimensional space is projected onto the gradient image and the second point cloud data of the projected three-dimensional space is completely integrated with the gradient image, the optimal position of the projected three-dimensional space projected onto the image data is determined.

64. The movable platform according to claim 63, characterized in that: When the processor determines the gradient image corresponding to the image data according to the image data collected by the camera, it is specifically used to: Determine, based on the image data collected by the camera, a grayscale image corresponding to the image data; Extracting gradient information and / or edge information from the grayscale image; The gradient image is determined according to the gradient information and / or the edge information.

65. The movable platform according to claim 49, characterized in that: When the processor projects the projected three-dimensional space onto the image data collected by the camera and obtains the optimal position of the projected three-dimensional space projected onto the image data, it is specifically used to: Acquire a target image obtained by projecting the projected three-dimensional space onto the image data collected by the camera; determining a reflectivity of second point cloud data in the target image; Determining a grayscale value of a grayscale image corresponding to the target image; An optimal position where the projected three-dimensional space is projected onto the image data is determined according to the reflectivity of the second point cloud data in the target image and the grayscale value of the grayscale image corresponding to the target image.

66. The movable platform according to claim 50, characterized in that: When the processor projects the projected three-dimensional space onto the image data collected by the camera and obtains the optimal position of the projected three-dimensional space projected onto the image data, it is specifically used to: Acquiring motion information of the movable platform during movement; Determining compensation information of the second point cloud data according to the motion information; Compensating the second point cloud data in the projected three-dimensional space according to the compensation information; The compensated second point cloud data is projected onto the image data collected by the camera to obtain an optimal position of the projected three-dimensional space projected onto the image data.

67. The movable platform according to claim 66, characterized in that: The motion information includes any one or more of position information, speed information, and acceleration information.

68. The movable platform according to claim 49, characterized in that: When the processor projects the projected three-dimensional space onto the image data collected by the camera and obtains the optimal position of the projected three-dimensional space projected onto the image data, it is specifically used to: Acquire second point cloud data during the movement of the movable platform within a preset time range; The second point cloud data in the projected three-dimensional space acquired within the preset time range is projected onto the image data collected by the camera to obtain an optimal position of the projected three-dimensional space projected onto the image data.

69. The movable platform according to claim 66 or 68, characterized in that When the processor projects the projected three-dimensional space onto the image data collected by the camera and obtains the optimal position of the projected three-dimensional space projected onto the image data, it is specifically used to: Acquire a plurality of target images obtained by projecting the projection three-dimensional space onto the image data collected by the camera during the movement of the movable platform; Compare the data of each target image; If it is determined that the data of each target image is consistent, the position information of the target image is determined to be the optimal position of the projected three-dimensional space projected onto the image data.

70. The movable platform according to claim 69, characterized in that: The processor is further configured to: If it is determined that the data of each target image is inconsistent, it is determined that the external parameters of the laser scanning device have changed, and the external parameters of the laser scanning device are updated.

71. The movable platform according to claim 69, characterized in that: The processor is further configured to: If it is determined that the data of each target image is inconsistent, a preset alarm device is triggered to sound an alarm to prompt the user to check the laser scanning device.

72. The movable platform according to claim 49, characterized in that: The laser scanning device includes any one or more of a laser radar, a millimeter wave radar, and an ultrasonic radar.

73. The movable platform according to claim 49, characterized in that: The laser scanning device and the camera are respectively detachably connected to the movable platform.

74. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 24 is implemented.

Images