Point cloud coloring processing method and laser scanning equipment

Abstract

According to the point cloud coloring processing method and the laser scanning equipment provided by the invention, pose acquisition and point cloud projection processing are carried out by using a laser radar and a camera which are synchronized by a clock, so that pose errors caused by clock differences between the camera and the laser radar can be avoided; the method can improve the point cloud coloring accuracy and the consistency of point cloud coloring corresponding to the same object, carries out the second pose calculation and point cloud coordinate conversion of the camera through employing the first external parameter calibrated in advance, projects the point cloud to the corresponding position of the image plane coordinate system of the image, and determines the color value of the point. Point cloud coloring processing can be completed in the laser scanning equipment, the point cloud coloring efficiency can be improved, and therefore the real-time performance of point cloud generation and display is improved.

Description

Technical Field

[0001] The present invention relates to three-dimensional reconstruction technology and point cloud processing technology, in particular to a point cloud coloring processing method and a laser scanning device. Background Art

[0002] At present, laser simultaneous localization and mapping (SLAM) scanning devices are usually equipped with inertial measurement units (IMU) and laser radars, which can generate SLAM poses and point clouds in real time for scanning scenes. Users can connect laser scanning devices through terminal devices such as mobile phones and view the point clouds collected by the laser scanning devices in real time on the terminal devices. In order to achieve a better point cloud preview effect, laser scanning devices usually need to color the point clouds to obtain colored point clouds for users to view.

[0003] In order to realize the function of coloring point clouds, one or more color cameras are usually installed on the body of the laser scanning device to color the point cloud according to the content captured by the color camera. However, since the camera and laser radar respectively perform image acquisition and scene scanning based on their own internal clocks, there is a time difference between the image acquisition time and the point cloud generation time, which can easily lead to errors in point cloud coloring and poor coloring effect. Summary of the invention

[0004] The embodiments of the present disclosure provide a point cloud coloring processing method and a laser scanning device, which can improve the accuracy and efficiency of point cloud coloring.

[0005] In one aspect of an embodiment of the present disclosure, a point cloud coloring processing method is provided, which is applied to a laser scanning device, wherein the laser scanning device is equipped with a laser radar and a camera, wherein the laser radar is used to generate a point cloud for a target scene, and the camera is used to collect an image for the target scene, and the clocks between the laser radar and the camera are synchronized; the method comprises:

[0006] For each frame of image captured by the camera, based on the capture time corresponding to the image, obtain the first pose data of the laser radar at the capture time, where the first pose data is used to characterize the position and pose of the laser radar in the world coordinate system corresponding to the target scene;

[0007] Based on the first pose data and a first external parameter of the camera relative to the laser radar, obtaining second pose data of the camera at the acquisition time, wherein the second pose data is used to characterize the position and posture of the camera in the world coordinate system, and the first external parameter is used to characterize the relative position and relative posture of the camera relative to the laser radar in the world coordinate system;

[0008] Based on the second pose data and the first coordinate of at least one point in the point cloud in the world coordinate system, performing coordinate transformation on the at least one point to obtain a second coordinate of the at least one point in the image plane coordinate system corresponding to the image;

[0009] The at least one point is colored based on the color value at the second coordinate in the image.

[0010] Optionally, the laser scanning device is also equipped with an independent clock. For each frame of image captured by the camera, based on the acquisition time of the image, before obtaining the first pose data of the laser radar at the acquisition time, the method includes:

[0011] The camera is used to collect images to generate an image data stream, and the laser radar is used to perform laser scanning to generate a point cloud data stream;

[0012] Based on the preset clock synchronization method, the independent clock is used to synchronize the internal clock of the laser radar and the internal clock of the camera, so as to obtain the acquisition time of each frame of the image in the image data stream and the generation time of each frame of the point cloud in the point cloud data stream.

[0013] Optionally, for each frame of image captured by the camera, obtaining the first position data of the laser radar at the capture time based on the capture time of the image, includes:

[0014] For each frame of image in the image data stream, comparing the acquisition time of the image with the timestamp of each first position data in the first position list;

[0015] In response to the existence of a target timestamp that is the same as the acquisition time, determining the first position data corresponding to the target timestamp as the first position data of the laser radar at the acquisition time;

[0016] In response to the absence of the target timestamp, an interpolation calculation is performed based on the first position data corresponding to at least two groups of timestamps adjacent to the acquisition time to obtain the first position data of the laser radar at the acquisition time.

[0017] Optionally, performing coordinate transformation on the at least one point based on the second pose data and the first coordinate of at least one point in the point cloud in the world coordinate system to obtain the second coordinate of the at least one point in the image plane coordinate system corresponding to the image includes:

[0018] Based on the second posture data and the first coordinates, performing a first coordinate transformation on the at least one point to obtain a third coordinate of the at least one point in a camera coordinate system corresponding to the camera;

[0019] Based on the third coordinate and the camera intrinsic parameter of the camera, a second coordinate transformation is performed on the at least one point to obtain the second coordinate, wherein the camera intrinsic parameter is used to characterize a mapping relationship between the camera coordinate system and the image plane coordinate system.

[0020] Optionally, performing a first coordinate transformation on the at least one point based on the camera pose data and the first coordinate to obtain a third coordinate of the at least one point in a camera coordinate system corresponding to the camera includes:

[0021] A matrix multiplication operation is performed on the first coordinate using a second external parameter of the laser radar relative to the camera, second posture data of the camera at the generation time, and second posture data of the camera at the acquisition time to obtain the third coordinate of the at least one point generated at the generation time in the camera coordinate system at the acquisition time, wherein the second external parameter is used to characterize the relative position and relative posture of the laser radar relative to the camera in the world coordinate system.

[0022] Optionally, performing a second coordinate transformation on the at least one point based on the third coordinate and a camera intrinsic parameter of the camera to obtain the second coordinate includes:

[0023] A matrix operation is performed on the third coordinate of the at least one point using a parameter matrix corresponding to the camera intrinsic parameter to obtain the second coordinate, wherein the elements in the parameter matrix corresponding to the camera intrinsic parameter include a focal length in the coordinate axis direction of the image plane coordinate system and an offset of the origin of the image plane coordinate system relative to the origin of the camera coordinate system.

[0024] Optionally, acquiring second pose data of the camera at the acquisition time based on the first pose data and a first external parameter of the camera relative to the laser radar includes:

[0025] Performing a matrix multiplication operation based on the first posture data and the first external parameter to obtain the second posture data;

[0026] Among them, the parameter matrix corresponding to the first posture data is composed of a translation matrix used to represent the position of the laser radar in the world coordinate system and a rotation matrix representing the orientation, and the parameter matrix corresponding to the first external parameter is composed of a translation matrix used to represent the relative position of the camera relative to the laser radar in the world coordinate system and a rotation matrix representing the relative orientation.

[0027] Optionally, performing coordinate transformation on the at least one point based on the second pose data and the first coordinate of at least one point in the point cloud in the world coordinate system to obtain the second coordinate of the at least one point in the image plane coordinate system corresponding to the image includes:

[0028] Based on the second position data of the camera at the acquisition time corresponding to at least two frames of target images and the first coordinates of the at least one point, coordinate transformation is performed on the at least one point to obtain the second coordinates of the at least one point in the image plane coordinate system of the at least two frames of target images, wherein the target image is an image whose acquisition time is earlier than or equal to the generation time of the at least one point;

[0029] The coloring process of the at least one point based on the color value at the second coordinate in the image includes:

[0030] The at least one point is subjected to coloring processing based on the color value of the at least one point at the second coordinates in the at least two frames of target images.

[0031] Optionally, the coloring process of the at least one point based on the color value of the at least one point at the second coordinate in the at least two frames of target images includes:

[0032] Taking the average of the color values ​​of the at least one point at the second coordinates in the at least two frames of target images as the actual color value of the at least one point, and performing coloring processing on the at least one point;

[0033] or,

[0034] The at least two frames of target images are traversed in a preset order, and the uncolored points in the at least one point are colored using the color value of the uncolored point at the second coordinate in the current target image.

[0035] Another aspect of the embodiments of the present disclosure provides a point cloud coloring processing device, which is applied to a laser scanning device, wherein the laser scanning device is equipped with a laser radar and a camera, wherein the laser radar is used to generate a point cloud for a target scene, and the camera is used to collect images for the target scene, and the clocks between the laser radar and the camera are synchronized; the device comprises:

[0036] A first acquisition module is used to acquire, for each frame of image captured by the camera, first position data of the laser radar at the acquisition time based on the acquisition time of the image, wherein the first position data is used to characterize the position and posture of the laser radar in the world coordinate system corresponding to the target scene;

[0037] A second acquisition module is used to acquire second pose data of the camera at the acquisition time based on the first pose data and a first external parameter of the camera relative to the laser radar, wherein the second pose data is used to characterize the position and posture of the camera in the world coordinate system, and the first external parameter is used to characterize the relative position and relative posture of the camera relative to the laser radar in the world coordinate system;

[0038] a coordinate conversion module, configured to perform coordinate conversion on at least one point in the point cloud based on the second posture data and a first coordinate of the at least one point in the world coordinate system, so as to obtain a second coordinate of the at least one point in an image plane coordinate system corresponding to the image;

[0039] A coloring module is used to color the at least one point based on the color value at the second coordinate in the image.

[0040] Another aspect of the embodiments of the present disclosure provides a laser scanning device, including: an inertial measurement unit, a laser radar, a camera, and a point cloud coloring processing device; the laser radar is used to generate a point cloud for a target scene, the camera is used to collect images for the target scene, and the laser radar and the camera are clock-synchronized;

[0041] The point cloud coloring processing device is used to:

[0042] For each frame of image captured by the camera, based on the capture time of the image, obtain first pose data of the laser radar at the capture time, where the first pose data is used to characterize the position and pose of the laser radar in the world coordinate system corresponding to the target scene;

[0043] Based on the first pose data and a first external parameter of the camera relative to the laser radar, obtaining second pose data of the camera at the acquisition time, wherein the second pose data is used to characterize the position and posture of the camera in the world coordinate system, and the first external parameter is used to characterize the relative position and relative posture of the camera relative to the laser radar in the world coordinate system;

[0044] Based on the second pose data and the first coordinate of at least one point in the point cloud in the world coordinate system, performing coordinate transformation on the at least one point to obtain a second coordinate of the at least one point in the image plane coordinate system corresponding to the image;

[0045] The at least one point is colored based on the color value at the second coordinate in the image.

[0046] Another aspect of the present disclosure provides an electronic device, including:

[0047] Memory for storing computer programs;

[0048] The processor is used to execute the computer program stored in the memory, and when the computer program is executed, the method described in the above aspects is implemented.

[0049] Another aspect of the embodiments of the present disclosure provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the method described in the above aspects is implemented.

[0050] Another aspect of the embodiments of the present disclosure provides a computer program, including computer program instructions, which implement the methods described in the above aspects when executed by a processor.

[0051] Based on the embodiment of the present disclosure, when the clocks of the laser radar and the camera are synchronized, the first pose data of the laser radar at the time of image acquisition is obtained, and the second pose data of the camera at the time of acquisition is calculated based on the first pose data and the first external parameter of the camera relative to the laser radar, so as to convert the first coordinate of at least one point in the point cloud in the world coordinate system into the second coordinate in the image plane coordinate system, and colorize at least one point in the point cloud based on the color value at the second coordinate in the image. The laser radar and camera with synchronized clocks are used to perform pose acquisition and point cloud projection processing, which can avoid the pose error caused by the clock difference between the camera and the laser radar, and can improve the accuracy of point cloud coloring and the consistency of point cloud coloring corresponding to the same object. In addition, the second pose calculation of the camera and the point cloud coordinate conversion are performed using the parameters such as the first external parameter calibrated in advance, and the point cloud is projected to the corresponding position of the image plane coordinate system where the image is located and the color value of the point is determined. The point cloud coloring processing can be completed inside the laser scanning device, which can improve the efficiency of point cloud coloring, thereby improving the real-time performance of point cloud generation and display.

[0052] The technical solution of the present disclosure is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] The accompanying drawings, which constitute a part of the specification, illustrate embodiments of the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0054] The present disclosure may be more clearly understood from the following detailed description with reference to the accompanying drawings, in which:

[0055] Figure 1 A flowchart of an embodiment of the point cloud coloring processing method disclosed in the present invention;

[0056] Figure 2 A flowchart of another embodiment of the point cloud coloring processing method disclosed in the present invention;

[0057] Figure 3 A flowchart of another embodiment of the point cloud coloring processing method disclosed in the present invention;

[0058] Figure 4 It is a structural schematic diagram of an embodiment of the point cloud coloring device disclosed in the present invention;

[0059] Figure 5 It is a structural schematic diagram of another embodiment of the point cloud coloring device disclosed in the present invention;

[0060] Figure 6 The figure is a schematic diagram of the structure of an application embodiment of the electronic device disclosed in the present invention. DETAILED DESCRIPTION

[0061] Various exemplary embodiments of the present disclosure will now be described in detail with reference to the accompanying drawings. It should be noted that the relative arrangement of components and steps, numerical expressions and numerical values ​​set forth in these embodiments do not limit the scope of the present disclosure unless otherwise specifically stated.

[0062] Those skilled in the art can understand that the terms "first" and "second" in the embodiments of the present disclosure are only used to distinguish different steps, devices or modules, etc., and neither represent any specific technical meaning nor indicate the necessary logical order between them.

[0063] It should also be understood that in the embodiments of the present disclosure, “plurality” may refer to two or more than two, and “at least one” may refer to one, two, or more than two.

[0064] It should also be understood that any component, data or structure mentioned in the embodiments of the present disclosure can generally be understood as one or more, unless explicitly limited or otherwise indicated in the context.

[0065] In addition, the term "and / or" in the present disclosure is only a description of the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in the present disclosure generally indicates that the associated objects before and after are in an "or" relationship.

[0066] It should also be understood that the description of the various embodiments in the present disclosure focuses on the differences between the various embodiments, and the same or similar aspects thereof can be referenced to each other, and for the sake of brevity, they will not be described one by one.

[0067] At the same time, it should be understood that for the convenience of description, the sizes of the various parts shown in the drawings are not drawn according to the actual proportional relationship.

[0068] The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the present disclosure, its application, or uses.

[0069] Technologies, methods, and equipment known to ordinary technicians in the relevant art may not be discussed in detail, but where appropriate, the technologies, methods, and equipment should be considered as part of the specification.

[0070] It should be noted that like reference numerals and letters refer to similar items in the following figures, and therefore, once an item is defined in one figure, it need not be further discussed in subsequent figures.

[0071] Figure 1 A flowchart of a point cloud coloring processing method provided for an exemplary embodiment of the present disclosure. The point cloud coloring processing method of the embodiment of the present disclosure can be implemented by a laser scanning device equipped with a laser radar and a camera. The laser scanning device may include, for example, but is not limited to, a SLAM device, a visual SLAM (Visual SLAM, VSLAM), a three-dimensional scanner, etc. The present disclosure is described by taking the SLAM device as an example. Among them, the camera is used to collect images for the target scene, the laser radar is used to collect images for the target scene, and the clock between the camera and the laser radar is synchronized. The point cloud is a data set of points in a three-dimensional space, which can characterize the shape of objects in the three-dimensional space. The laser radar emits a large number of laser signals to the target scene. The laser signal is reflected by the surface of the object in the target scene to generate a reflection signal received by the laser radar. The laser radar can calculate the position information of each point on the surface of the object in the world coordinate system corresponding to the target scene based on the reflection signal, thereby generating a point cloud corresponding to the target scene, and the point cloud includes the position information of each point in the target scene. In addition, the position information, color value, laser reflection intensity, etc. of each point can also be associated and stored in the point cloud file corresponding to the point cloud to improve the point cloud information.

[0072] like Figure 1 As shown, the method comprises the following steps:

[0073] Step 101, for each frame of image captured by the camera, based on the image capture time, obtain the first position data of the laser radar at the capture time.

[0074] Among them, the first position data is used to characterize the position and posture of the laser radar in the world coordinate system corresponding to the target scene. The first position data may include, but is not limited to, at least one of the coordinates of the laser radar in the world coordinate system, Euler angles, and direction cosine matrices. Optionally, the first position data can be obtained by point cloud matching based on the point cloud generated by the laser radar, or by converting the position data generated by other components such as the IMU in the laser scanning device. The disclosed embodiment does not limit the source of the first position data.

[0075] When using the image captured by the camera to color the point cloud generated by the lidar, it is necessary to project the points in the point cloud to the plane where the image is located, so that the points can be colored using the color values ​​of the corresponding positions in the image. Therefore, it is necessary to first determine the camera posture at the time of acquisition, so as to combine the camera intrinsic parameters and other parameters to perform coordinate transformation on at least one point in the point cloud to obtain the coordinates in the image plane coordinate system.

[0076] In a possible implementation, the camera captures images at a preset image capture frequency. Therefore, when performing point cloud coloring, the capture time of each frame of the captured image stream is determined, and the first position data at each capture time is obtained.

[0077] Step 102, based on the first pose data and the first external parameter of the camera relative to the laser radar, obtain the second pose data of the camera at the collection time.

[0078] The second pose data is used to characterize the position and posture of the camera in the world coordinate system corresponding to the target scene. The second pose data may include, but is not limited to, at least one of the coordinates of the camera in the world coordinate system, Euler angles, and direction cosine matrix. The first external parameter is used to characterize the relative position and relative posture of the camera in the world coordinate system relative to the laser radar, and may include, but is not limited to, at least one of the translation matrix and rotation matrix of the camera relative to the laser radar.

[0079] Optionally, the camera and the laser radar are fixedly arranged in the laser scanning device, and the first external parameter is fixed and unchanged, and is a pre-calibrated fixed parameter, which can be directly read from a pre-stored file. Based on the first external parameter and the first pose data at each acquisition time, the second pose data of the camera at each acquisition time can be calculated.

[0080] Step 103: Based on the second posture data and the first coordinates of at least one point in the point cloud in the world coordinate system, coordinate transformation is performed on at least one point to obtain a second coordinate of the at least one point in the image plane coordinate system corresponding to the image.

[0081] The first coordinate is used to represent the position of the point in the world coordinate system corresponding to the target scene, which can be calculated directly based on the reflection signal when the laser radar generates the point cloud. The second coordinate is used to represent the position of the point in the image plane coordinate system corresponding to the image. The images collected at different times and by different cameras may have different corresponding image plane coordinate systems.

[0082] In one possible implementation, based on the second pose data, a camera coordinate system at the time of acquisition can be constructed, and the point cloud is first converted from the world coordinate system to the camera coordinate system, and then based on the relative relationship between the camera coordinate system and the image plane coordinate system, the point cloud is converted to the image plane coordinate system, thereby obtaining the second coordinate of at least one point in the point cloud in the image plane coordinate system.

[0083] Optionally, in the process of coloring the point cloud in the disclosed embodiment, for a frame of image, the corresponding point cloud to be colored is a point cloud composed of points accumulated by the laser radar from the acquisition time of the previous frame of image to the acquisition time of the current frame of image. For example, the camera captures an image every 1 second, then for the image captured at the 3rd second, the corresponding point cloud to be colored includes the points accumulated by the laser radar from the 2nd second to the 3rd second. Alternatively, since the field of view of the camera and the laser radar are different, in order to avoid the second coordinates of some point clouds exceeding the area corresponding to the image and thus being unable to be colored, multiple frames of images may be selected to color the point cloud. For example, 3 frames of images captured from the 1st second to the 3rd second are selected to color the point cloud accumulated from the 2nd second to the 3rd second. The disclosed embodiment does not limit the method for selecting the point cloud to be colored corresponding to the image.

[0084] Step 104: coloring at least one point based on the color value at the second coordinate in the image.

[0085] Optionally, when a group of point clouds are colored using a frame of an environment image, the color value at the second coordinate corresponding to the point can be directly saved as the color value of the point; when a group of point clouds are colored using multiple frames of images, the color value of the point at the second coordinate of the image in one of the frames of images can be selected as the color value of the point (for example, traversing multiple frames of images, if a point has been colored, the color value at the corresponding position of the point in the subsequent image is not considered), or uniform coloring can be performed, such as saving the average value of the color value of a point at the second coordinate of the image in multiple frames of images as the color value of the point. The disclosed embodiments do not limit the specific method of calculating color values.

[0086] Based on the embodiment of the present disclosure, when the clocks of the laser radar and the camera are synchronized, the first pose data of the laser radar at the time of image acquisition is obtained, and the second pose data of the camera at the time of acquisition is calculated based on the first pose data and the first external parameter of the camera relative to the laser radar, so as to convert the first coordinate of at least one point in the point cloud in the world coordinate system into the second coordinate in the image plane coordinate system, and colorize at least one point in the point cloud based on the color value at the second coordinate in the image. The laser radar and camera with synchronized clocks are used to perform pose acquisition and point cloud projection processing, which can avoid the pose error caused by the clock difference between the camera and the laser radar, and can improve the accuracy of point cloud coloring and the consistency of point cloud coloring corresponding to the same object. In addition, the second pose calculation of the camera and the point cloud coordinate conversion are performed using the parameters such as the first external parameter calibrated in advance, and the point cloud is projected to the corresponding position of the image plane coordinate system where the image is located and the color value of the point is determined. The point cloud coloring processing can be completed inside the laser scanning device, which can improve the efficiency of point cloud coloring, thereby improving the real-time performance of point cloud generation and display.

[0087] In a possible implementation, before starting the point cloud coloring process, it is first necessary to synchronize the internal clock of the camera and the internal clock of the lidar. Figure 2 A flowchart of a point cloud coloring processing method provided by another exemplary embodiment of the present disclosure. The point cloud coloring processing method of the embodiment of the present disclosure can be implemented by a laser scanning device equipped with a laser radar and a camera. Figure 2 As shown, the method comprises the following steps:

[0088] Step 201, collect images through a camera to generate an image data stream, and perform laser scanning through a laser radar to generate a point cloud data stream.

[0089] The camera and lidar first collect information based on their own internal clock and information collection frequency to obtain image data stream and point cloud data stream.

[0090] Step 202, based on a preset clock synchronization method, uses independent clocks to synchronize the internal clock of the lidar and the internal clock of the camera, to obtain the acquisition time of each frame of the image in the image data stream and the point cloud generation time of each frame of the point cloud in the point cloud data stream.

[0091] Illustratively, the clock synchronization method may include, but is not limited to, at least one of a pulse per second (PPS) synchronization method, a global navigation satellite system (GNSS) synchronization method, and a precision time protocol (PTP) synchronization method.

[0092] Among them, the independent clock is a high-precision clock source that exists independently of the camera and laser radar (for example, a high-precision active crystal oscillator, etc.), which is used to provide stable clock information with extremely small errors. Optionally, the time provided by the independent clock can be the standard time provided by the National Time Service Center, or an independently provided time that can be used as the revelation time at any time (for example, the time when the scanning device starts running is used as time 0). For example, when the PPS synchronization method is adopted, the independent clock can send a PPS signal to the camera and laser radar according to a preset period, control the internal clocks of the camera and laser radar to return to zero, and the camera and laser radar respectively collect information and add internal timestamps. The independent clock can determine the time offset value corresponding to the internal clock of the camera and laser radar based on the internal timestamp carried by the information sent by the camera and laser radar, thereby calculating the acquisition time of each frame of the above-mentioned image and the generation time of each frame of the point cloud based on the time of the independent clock and the internal offset value.

[0093] Step 203: for each frame of image in the image data stream, compare the image acquisition time with the timestamp of each first position data in the first position list.

[0094] Step 204, in response to the existence of a target timestamp that is the same as the acquisition time, the first bit pose data corresponding to the target timestamp is determined as the first bit pose data of the laser radar at the acquisition time.

[0095] Step 205, in response to the absence of a target timestamp, interpolation calculation is performed based on the first position data corresponding to at least two groups of timestamps adjacent to the acquisition time to obtain the first position data of the laser radar at the acquisition time.

[0096] In one possible implementation, the actual time when the camera captures the image and the actual time when the first pose data is generated may not correspond one to one. Therefore, if there is an image captured and the first pose data generated at the same time, the first pose data at that moment is directly determined as the first pose data of the laser radar at the capture moment. If there is no image captured and the first pose data generated at the same time, for the image, the first pose data corresponding to the laser radar at the capture moment of the image can be obtained by interpolation calculation.

[0097] Optionally, the first pose data and the timestamp may be associated and stored in the first pose list. Schematically, the timestamps of the first pose data in the first pose list include 00:00:01, 00:00:03, 00:00:05, etc. For the image collected at 00:00:02, the first pose data at 00:00:01 and the first pose data at 00:00:03 may be interpolated to obtain the first pose data at 00:00:02.

[0098] Step 206, based on the first pose data and the first external parameter of the camera relative to the laser radar, obtain the second pose data of the camera at the acquisition time.

[0099] The specific implementation of step 206 may refer to the above step 102, and will not be described in detail in this embodiment of the present disclosure.

[0100] In a possible implementation, a matrix multiplication operation can be performed based on the first pose data and the first external parameter to obtain the second pose data. The parameter matrix corresponding to the first pose data is composed of a translation matrix for representing the position of the laser radar in the world coordinate system and a rotation matrix for representing the orientation, and the parameter matrix corresponding to the first external parameter is composed of a translation matrix for representing the relative position of the camera relative to the laser radar in the world coordinate system and a rotation matrix for representing the relative orientation. Schematically, the calculation formula for the camera pose data is as follows:

[0101] T camera→world =T lidar→world T camera→lidar (1)

[0102] Among them, T camera→world is the second pose data, T lidar→world is the first pose data, T camera→lidar It is the first external reference.

[0103] Step 207: Based on the second posture data and the first coordinates of the at least one point in the world coordinate system, coordinate transformation is performed on the at least one point to obtain the second coordinates of the at least one point in the image plane coordinate system corresponding to the image.

[0104] The specific implementation of step 207 may refer to the above step 104, and will not be described in detail in this embodiment of the present disclosure.

[0105] Optionally, in order to achieve global coloring of the point cloud, multiple cameras may be set in the laser scanning device, and the point cloud may be colored using images captured by the multiple cameras. In a possible implementation, the laser scanning device is equipped with at least two cameras, and based on the second pose data of each camera and the first coordinate of at least one point, the coordinate transformation is performed on at least one point to obtain the second coordinate of at least one point in the image captured by each camera, so as to color the point cloud based on the image captured by each camera.

[0106] Optionally, considering that the field of view of the camera is different from that of the laser radar, the image acquisition field of view of the camera is usually smaller than the scanning field of view of the laser radar, so multiple frames of images can be used to color a frame of point cloud at a certain generation time or a group of point clouds accumulated over a period of time. Step 207 may include the following steps:

[0107] Based on the second posture data of the camera at the acquisition time corresponding to at least two frames of target images, and the first coordinates of at least one point, coordinate transformation is performed on at least one point to obtain the second coordinates of the at least one point in the image plane coordinate system of the at least two frames of target images, and the target image is an image whose acquisition time is earlier than or equal to the generation time of at least one point.

[0108] In a possible implementation, for a set of point clouds, a preset number of target images collected between the generation time of the set of point clouds can be selected for point cloud coloring. For example, for a set of point clouds, 5 frames of target images collected when the point cloud is generated and before the point cloud is generated can be selected as the basis for point cloud coloring, so it is necessary to calculate the second coordinates of each point in the point cloud in the 5 frames of target images.

[0109] Step 208: coloring the at least one point based on the color value at the second coordinate in the image.

[0110] In one possible implementation, in response to the second coordinate of the point being within the coordinate range corresponding to the image, the color value at the second coordinate in the image is associated with the corresponding point and stored in the point cloud file. If the second coordinate of a point is outside the coordinate range corresponding to the image, the point is not colored using the image.

[0111] Optionally, in order to achieve global coloring of the point cloud, the above step 207 uses multiple frames of target images before the point cloud is generated to color it. The points in the point cloud may be projected within the coordinate range of one frame of the target image, or may be projected within the coordinate range of multiple frames of the target image. In a possible implementation, step 208 may specifically include the following steps:

[0112] The average value of the color value of at least one point at the second coordinate in at least two frames of target images is used as the actual color value of the at least one point, and the at least one point is colored.

[0113] or,

[0114] At least two frames of target images are traversed in a preset order, and the uncolored points are colored using the color value of the uncolored points at the second coordinates in the current target image among at least one point.

[0115] For the first coloring method mentioned above, a uniform color method is used to determine the color value of a point, and the average of the color values ​​of the point at the corresponding position in multiple frames of the target image is determined as the color value of the point. This can avoid coloring errors caused by poor exposure of a certain frame of the image or incorrect posture calculation, thereby improving the coloring accuracy.

[0116] For the second coloring method mentioned above, the points to be colored in the point cloud are colored in sequence according to a preset order (for example, in order of acquisition time from early to late). If a point has been colored by a certain frame of target graphics, it will not participate in the subsequent coloring process of the point cloud by other target images. This can improve the efficiency of point cloud coloring and reduce the performance loss of the laser scanning equipment.

[0117] Based on the embodiments of the present disclosure, point cloud coloring is performed using multiple frames of images from the same camera and / or images from multiple cameras, thereby avoiding coloring omissions due to different fields of view between the camera and the lidar, and achieving global real-time coloring.

[0118] In a possible implementation, based on the second pose data of the camera, the coordinates of the point cloud can be first converted from the world coordinate system to the camera coordinate system, and then based on the relative relationship between the camera coordinate system and the image plane coordinate system, the coordinates of the point cloud can be converted to the image plane coordinate system, thereby obtaining the second coordinates of at least one point. Figure 1 Based on the examples shown, please refer to Figure 2 , the above step 103 may specifically include the following steps:

[0119] Step 103a: Based on the second posture data and the first coordinates, perform a first coordinate transformation on at least one point to obtain a third coordinate of the at least one point in a camera coordinate system corresponding to the camera.

[0120] The third coordinate is used to represent the position of the point in the camera coordinate system corresponding to the camera, and the first coordinate is used to represent the position of the point in the world coordinate system corresponding to the target scene.

[0121] In a possible implementation, a matrix multiplication operation is performed on the first coordinate using a second external parameter of the laser radar relative to the camera, second position and posture data of the camera at the generation time t1, and second position and posture data of the camera at the acquisition time t0 to obtain a third coordinate of at least one point generated at the generation time t1 in the camera coordinate system at the acquisition time t0. The second external parameter is used to characterize the relative position and relative posture of the laser radar relative to the camera in the world coordinate system.

[0122] Optionally, for the point generated at time t1, after matrix operation of the first external parameter, the second pose data at time t1 and the second pose data at time t0, the third coordinate in the camera coordinate system corresponding to the camera pose at time t0 can be converted. Schematically, the calculation formula of the third coordinate is as follows:

[0123]

[0124] Among them, t0 is the time of image acquisition, t1 is the time of point cloud generation, is the third coordinate of the point in the camera coordinate system at time t0, is the first coordinate of the point generated at time t1 in the world coordinate system, T lidar→camera is the second external parameter, which is used to characterize the position and attitude of the laser radar relative to the camera. This parameter can be read from a pre-stored file. is the second posture data at time t1, It is the second pose data at time t0.

[0125] Step 103b: Based on the third coordinate and the camera intrinsic parameter of the camera, perform a second coordinate transformation on at least one point to obtain a second coordinate.

[0126] The camera intrinsic parameters are used to characterize the mapping relationship between the camera coordinate system and the image plane coordinate system. The camera intrinsic parameters are parameter matrices used to describe the internal properties of the camera, including focal length, principal point (optical center) coordinates, distortion coefficients, etc. The camera intrinsic parameters are usually determined during camera calibration and do not change over time, so they can be directly read from pre-stored parameter files.

[0127] In a possible implementation, a parameter matrix corresponding to the camera intrinsic parameter is used to perform a matrix operation on the third coordinate of at least one point to obtain a second coordinate of the point in the image plane coordinate system corresponding to the image. The elements in the parameter matrix corresponding to the camera intrinsic parameter include the focal length in the coordinate axis direction of the image plane coordinate system and the offset of the origin of the image plane coordinate system relative to the origin of the camera coordinate system.

[0128] Optionally, the environmental image in the embodiment of the present disclosure may be distorted in advance, so the coordinate transformation process may not consider the camera distortion. The calculation formula for removing the camera distortion is as follows:

[0129] x ″ =x ′ (1+k 1 r 2 +k 2 r 4 +k 3 r 6 )+2p 1 x ′ y ′ +p 2 (r 2 +2x ′2 ) (3)

[0130] y ″ =y ′ (1+k 1 r 2 +k 2 r 4 +k 3 r 6 )+2p 2 x′ y ′ +p 1 (r 2 +2y ′2 ) (4)

[0131] Among them, (x ′ ,y ′ ) represents the coordinates of the pixel in the image without camera distortion, (x ″ ,y ″ ) represents the coordinates of the pixel in the image after camera distortion, r represents the distance from the pixel to the center of the image, k1, k2, k3 are the radial distortion correction coefficients, and p1, p2 are the tangential distortion correction coefficients.

[0132] Schematically, the calculation formula of the second coordinate is as follows:

[0133]

[0134] in, is the second coordinate, is the third coordinate, is the camera internal parameter, f x 、f y They represent the focal length in the x-axis direction and the focal length in the y-axis direction, respectively. x 、c y They respectively represent the offset of the origin of the image plane coordinate system relative to the origin of the camera coordinate system.

[0135] Based on the embodiments of the present disclosure, by calibrating the camera intrinsic parameters and the first extrinsic parameters, the second extrinsic parameters and other parameters in advance, the second pose data corresponding to the acquisition moment of each image is calculated using the above-mentioned parameter matrix, and the coordinates of the point cloud in the world coordinate system are converted to coordinates in the image plane coordinate system. It is possible to achieve fast real-time coloring of the point cloud, and the point cloud coloring efficiency is high, and it can run in real time in the laser scanning equipment.

[0136] Please refer to Figure 4 , which shows a structural block diagram of a point cloud coloring device provided by an exemplary embodiment of the present disclosure. The point cloud coloring device is applied to a laser scanning device, wherein the laser scanning device is equipped with a laser radar and a camera, the laser radar is used to generate a point cloud for a target scene, the camera is used to collect images for the target scene, and the clocks between the laser radar and the camera are synchronized; the device includes:

[0137] The first acquisition module 401 is used to acquire the first position data of the laser radar at the acquisition time for each frame of image acquired by the camera based on the acquisition time of the image, where the first position data is used to represent the position and posture of the laser radar in the world coordinate system corresponding to the target scene;

[0138] A second acquisition module 402 is used to acquire second pose data of the camera at the acquisition time based on the first pose data acquired by the first acquisition module 401 and the first external parameter of the camera relative to the laser radar, wherein the second pose data is used to characterize the position and pose of the camera in the world coordinate system, and the first external parameter is used to characterize the relative position and relative pose of the camera relative to the laser radar in the world coordinate system;

[0139] A coordinate conversion module 403 is used to perform coordinate conversion on at least one point based on the second posture data acquired by the second acquisition module 402 and the first coordinate of at least one point in the point cloud in the world coordinate system to obtain a second coordinate of the at least one point in the image plane coordinate system corresponding to the image;

[0140] The coloring module 404 is used to perform coloring processing on at least one point based on the color value at the second coordinate in the image.

[0141] Optionally, in a possible implementation, the laser scanning device is also equipped with an independent clock, such as Figure 5 As shown, the device may also include:

[0142] The data acquisition module 501 is used to acquire images through a camera to generate an image data stream, and to perform laser scanning through a laser radar to generate a point cloud data stream;

[0143] The clock synchronization module 502 is used to perform clock synchronization based on a preset clock synchronization method, using independent clocks as the internal clock of the laser radar and the internal clock of the camera, to obtain the acquisition time of each frame of the image in the image data stream and the generation time of each frame of the point cloud in the point cloud data stream.

[0144] Optionally, in a possible implementation manner, the first acquisition module 401 may also be used to:

[0145] For each frame of image in the image data stream collected by the data collection module 501, compare the collection time of the image with the timestamp of each first position data in the first position list;

[0146] In response to the existence of a target timestamp that is the same as the acquisition time, determining the first position data corresponding to the target timestamp as the first position data of the laser radar at the acquisition time;

[0147] In response to the absence of a target timestamp, an interpolation calculation is performed based on the first position data corresponding to at least two groups of timestamps adjacent to the acquisition time to obtain the first position data of the laser radar at the acquisition time.

[0148] Optionally, in a possible implementation manner, the coordinate conversion module 403 may also be used for:

[0149] Based on the second posture data and the first coordinates acquired by the second acquisition module 402, performing a first coordinate transformation on at least one point to obtain a third coordinate of the at least one point in a camera coordinate system corresponding to the camera;

[0150] Based on the third coordinate and the camera intrinsic parameter of the camera, a second coordinate transformation is performed on at least one point to obtain a second coordinate, where the camera intrinsic parameter is used to characterize a mapping relationship between the camera coordinate system and the image plane coordinate system.

[0151] Optionally, in a possible implementation manner, the coordinate conversion module 403 may also be used for:

[0152] A matrix multiplication operation is performed on the first coordinate using a second external parameter of the laser radar relative to the camera, second posture data of the camera at the generation time, and second posture data of the camera at the acquisition time to obtain a third coordinate in the camera coordinate system at the acquisition time of at least one point generated at the generation time, wherein the second external parameter is used to characterize the relative position and relative posture of the laser radar relative to the camera in the world coordinate system.

[0153] Optionally, in a possible implementation manner, the coordinate conversion module 403 may also be used for:

[0154] A matrix operation is performed on the third coordinate of at least one point using a parameter matrix corresponding to the camera intrinsic parameter to obtain a second coordinate, wherein the elements in the parameter matrix corresponding to the camera intrinsic parameter include the focal length in the coordinate axis direction of the image plane coordinate system and the offset of the origin of the image plane coordinate system relative to the origin of the camera coordinate system.

[0155] Optionally, in a possible implementation manner, the second acquisition module 402 may also be used to:

[0156] Performing a matrix multiplication operation based on the first pose data acquired by the first acquisition module 401 and the first external parameter to obtain second pose data;

[0157] Among them, the parameter matrix corresponding to the first pose data is composed of a translation matrix used to represent the position of the laser radar in the world coordinate system and a rotation matrix representing the orientation. The parameter matrix corresponding to the first external parameter is composed of a translation matrix used to represent the relative position of the camera relative to the laser radar in the world coordinate system and a rotation matrix representing the relative orientation.

[0158] Optionally, in a possible implementation manner, the coordinate conversion module 403 may also be used for:

[0159] Based on the second position data of the camera at the acquisition time corresponding to at least two frames of target images and the first coordinates of at least one point, coordinate transformation is performed on at least one point to obtain the second coordinates of the at least one point in the image plane coordinate system of the at least two frames of target images, where the target image is an image whose acquisition time is earlier than or equal to the generation time of the at least one point;

[0160] The above-mentioned coloring module 404 can also be used for:

[0161] The at least one point is colored based on the color value of the at least one point at the second coordinate in the at least two frames of the target image.

[0162] Optionally, in a possible implementation manner, the coloring module 404 may also be used to:

[0163] Taking the average of the color values ​​of at least one point at the second coordinate in at least two frames of target images as the actual color value of at least one point, and performing coloring processing on at least one point;

[0164] or,

[0165] At least two frames of target images are traversed in a preset order, and the uncolored points are colored using the color value of the uncolored points at the second coordinates in the current target image among at least one point.

[0166] The disclosed embodiment also provides a laser scanning device, including: an IMU, a laser radar, a camera, and a point cloud coloring processing device; the laser radar is used to generate a point cloud for a target scene, the camera is used to collect images for the target scene, and the clocks between the laser radar and the camera are synchronized;

[0167] The point cloud coloring processing device is used for:

[0168] For each frame of image captured by the camera, based on the image capture time, the first position data of the laser radar at the capture time is obtained. The first position data is used to characterize the position and posture of the laser radar in the world coordinate system corresponding to the target scene.

[0169] Based on the first pose data and the first external parameter of the camera relative to the laser radar, obtain the second pose data of the camera at the acquisition time, the second pose data is used to characterize the position and posture of the camera in the world coordinate system, and the first external parameter is used to characterize the relative position and relative posture of the camera relative to the laser radar in the world coordinate system;

[0170] Based on the second pose data and a first coordinate of at least one point in the point cloud in the world coordinate system, performing coordinate transformation on the at least one point to obtain a second coordinate of the at least one point in an image plane coordinate system corresponding to the image;

[0171] At least one point is colored based on a color value at a second coordinate in the image.

[0172] In addition, an embodiment of the present disclosure further provides an electronic device, including:

[0173] Memory for storing computer programs;

[0174] The processor is used to execute the computer program stored in the memory, and when the computer program is executed, the point cloud coloring processing method described in any of the above embodiments of the present disclosure is implemented.

[0175] Figure 6 This is a schematic diagram of the structure of an application embodiment of the electronic device disclosed in the present invention. Figure 6 The electronic device according to the embodiment of the present disclosure is described. The electronic device may be any one or both of the first device and the second device, or a stand-alone device independent of them, and the stand-alone device may communicate with the first device and the second device to receive the collected input signals from them.

[0176] like Figure 6 As shown, the electronic device includes one or more processors and memory.

[0177] The processor may be a central processing unit (CPU) or other forms of processing units having data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device to perform desired functions.

[0178] The memory may include one or more computer program products, and the computer program product may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, a random access memory (RAM) and / or a cache memory (cache), etc. The non-volatile memory may include, for example, a read-only memory (ROM), a hard disk, a flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor may execute the program instructions to implement the point cloud coloring processing method of each embodiment of the present disclosure described above and / or other desired functions.

[0179] In one example, the electronic device may further include: an input device and an output device, and these components are interconnected via a bus system and / or other forms of connection mechanisms (not shown).

[0180] In addition, the input device may also include, for example, a keyboard, a mouse, and the like.

[0181] The output device can output various information to the outside, including the determined distance information, direction information, etc. The output device can include, for example, a display, a speaker, a printer, a communication network and a remote output device connected thereto, and the like.

[0182] Of course, to simplify, Figure 6 Only some of the components related to the present disclosure in the electronic device are shown, and components such as a bus, an input / output interface, etc. are omitted. In addition, according to specific application situations, the electronic device may further include any other appropriate components.

[0183] In addition to the above-mentioned methods and devices, an embodiment of the present disclosure may also be a computer program product, which includes computer program instructions, which, when executed by a processor, enable the processor to execute the steps of the point cloud coloring processing method according to various embodiments of the present disclosure described in the above part of this specification.

[0184] The computer program product may be written in any combination of one or more programming languages ​​to write program code for performing the operations of the disclosed embodiments, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0185] In addition, an embodiment of the present disclosure may also be a computer-readable storage medium on which computer program instructions are stored. When the computer program instructions are executed by a processor, the processor executes the steps of the point cloud coloring processing method according to various embodiments of the present disclosure described in the above part of this specification.

[0186] The computer readable storage medium can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can include, for example, but is not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0187] A person of ordinary skill in the art can understand that all or part of the steps of implementing the above method embodiment can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above method embodiment; and the aforementioned storage medium includes: ROM, RAM, magnetic disk or optical disk, etc., various media that can store program codes.

[0188] The basic principles of the present disclosure are described above in conjunction with specific embodiments. However, it should be noted that the advantages, strengths, effects, etc. mentioned in the present disclosure are only examples and not limitations, and it cannot be considered that these advantages, strengths, effects, etc. are required by each embodiment of the present disclosure. In addition, the specific details disclosed above are only for the purpose of illustration and ease of understanding, and are not limitations. The above details do not limit the present disclosure to the necessity of adopting the above specific details to be implemented.

[0189] Each embodiment in this specification is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the system embodiment, since it basically corresponds to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0190] The block diagrams of the devices, apparatuses, equipment, and systems involved in this disclosure are only illustrative examples and are not intended to require or imply that they must be connected, arranged, and configured in the manner shown in the block diagrams. As will be appreciated by those skilled in the art, these devices, apparatuses, equipment, and systems can be connected, arranged, and configured in any manner. Words such as "including," "comprising," "having," and the like are open words, referring to "including but not limited to," and can be used interchangeably therewith. The words "or" and "and" used herein refer to the words "and / or," and can be used interchangeably therewith, unless the context clearly indicates otherwise. The word "such as" used herein refers to the phrase "such as but not limited to," and can be used interchangeably therewith.

[0191] The method and apparatus of the present disclosure may be implemented in many ways. For example, the method and apparatus of the present disclosure may be implemented by software, hardware, firmware, or any combination of software, hardware, and firmware. The above order of steps for the method is for illustration only, and the steps of the method of the present disclosure are not limited to the order specifically described above, unless otherwise specifically stated. In addition, in some embodiments, the present disclosure may also be implemented as a program recorded in a recording medium, which includes machine-readable instructions for implementing the method according to the present disclosure. Therefore, the present disclosure also covers a recording medium storing a program for executing the method according to the present disclosure.

[0192] It should also be noted that in the apparatus, device and method of the present disclosure, each component or each step can be decomposed and / or recombined. Such decomposition and / or recombination should be regarded as equivalent solutions of the present disclosure.

[0193] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of the present disclosure. Therefore, the present disclosure is not intended to be limited to the aspects shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

[0194] The above description has been given for the purpose of illustration and description. In addition, this description is not intended to limit the embodiments of the present disclosure to the forms disclosed herein. Although multiple example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, changes, additions and sub-combinations thereof.

Claims

1. A point cloud coloring processing method, characterized in that: Applied to a laser scanning device, the laser scanning device is equipped with a laser radar and a camera, the laser radar is used to generate a point cloud for a target scene, the camera is used to collect images for the target scene, and the clocks between the laser radar and the camera are synchronized; the method comprises: For each frame of image captured by the camera, based on the capture time of the image, obtain first pose data of the laser radar at the capture time, where the first pose data is used to characterize the position and pose of the laser radar in the world coordinate system corresponding to the target scene; Based on the first pose data and a first external parameter of the camera relative to the laser radar, obtaining second pose data of the camera at the acquisition time, wherein the second pose data is used to characterize the position and posture of the camera in the world coordinate system, and the first external parameter is used to characterize the relative position and relative posture of the camera relative to the laser radar in the world coordinate system; Based on the second pose data and the first coordinate of at least one point in the point cloud in the world coordinate system, performing coordinate transformation on the at least one point to obtain a second coordinate of the at least one point in the image plane coordinate system corresponding to the image; The at least one point is colored based on the color value at the second coordinate in the image.

2. The method according to claim 1, characterized in that The laser scanning device is also equipped with an independent clock. For each frame of image captured by the camera, based on the acquisition time of the image, before obtaining the first pose data of the laser radar at the acquisition time, the method includes: Capturing images through the camera to generate an image data stream, and performing laser scanning through the laser radar to generate a point cloud data stream; Based on the preset clock synchronization method, the independent clock is used to synchronize the internal clock of the laser radar and the internal clock of the camera, so as to obtain the acquisition time of each frame of the image in the image data stream and the generation time of each frame of the point cloud in the point cloud data stream.

3. The method according to claim 2, characterized in that The method of acquiring the first position data of the laser radar at the acquisition time for each frame of image acquired by the camera based on the acquisition time of the image comprises: For each frame of image in the image data stream, comparing the acquisition time of the image with the timestamp of each first position data in the first position list; In response to the existence of a target timestamp that is the same as the acquisition time, determining the first position data corresponding to the target timestamp as the first position data of the laser radar at the acquisition time; In response to the absence of the target timestamp, an interpolation calculation is performed based on the first position data corresponding to at least two groups of timestamps adjacent to the acquisition time to obtain the first position data of the laser radar at the acquisition time.

4. The method according to any one of claims 1 to 3, characterized in that: The step of performing coordinate transformation on the at least one point based on the second pose data and the first coordinate of at least one point in the point cloud in the world coordinate system to obtain the second coordinate of the at least one point in the image plane coordinate system corresponding to the image includes: Based on the second posture data and the first coordinates, performing a first coordinate transformation on the at least one point to obtain a third coordinate of the at least one point in a camera coordinate system corresponding to the camera; Based on the third coordinate and the camera intrinsic parameter of the camera, a second coordinate transformation is performed on the at least one point to obtain the second coordinate, wherein the camera intrinsic parameter is used to characterize a mapping relationship between the camera coordinate system and the image plane coordinate system.

5. The method according to claim 4, characterized in that The performing a first coordinate transformation on the at least one point based on the camera pose data and the first coordinate to obtain a third coordinate of the at least one point in a camera coordinate system corresponding to the camera includes: A matrix multiplication operation is performed on the first coordinate using a second external parameter of the laser radar relative to the camera, second posture data of the camera at the generation time, and second posture data of the camera at the acquisition time to obtain the third coordinate of the at least one point generated at the generation time in the camera coordinate system at the acquisition time, wherein the second external parameter is used to characterize the relative position and relative posture of the laser radar relative to the camera in the world coordinate system.

6. The method according to claim 4, characterized in that The performing a second coordinate transformation on the at least one point based on the third coordinate and the camera intrinsic parameter of the camera to obtain the second coordinate includes: A matrix operation is performed on the third coordinate of the at least one point using a parameter matrix corresponding to the camera intrinsic parameter to obtain the second coordinate, wherein the elements in the parameter matrix corresponding to the camera intrinsic parameter include a focal length in the coordinate axis direction of the image plane coordinate system and an offset of the origin of the image plane coordinate system relative to the origin of the camera coordinate system.

7. The method according to any one of claims 1 to 3, characterized in that: The acquiring, based on the first posture data and a first external parameter of the camera relative to the laser radar, second posture data of the camera at the acquisition time comprises: Performing a matrix multiplication operation based on the first posture data and the first external parameter to obtain the second posture data; Among them, the parameter matrix corresponding to the first posture data is composed of a translation matrix used to represent the position of the laser radar in the world coordinate system and a rotation matrix representing the orientation, and the parameter matrix corresponding to the first external parameter is composed of a translation matrix used to represent the relative position of the camera relative to the laser radar in the world coordinate system and a rotation matrix representing the relative orientation.

8. The method according to any one of claims 1 to 3, characterized in that: The step of performing coordinate transformation on the at least one point based on the second pose data and the first coordinate of at least one point in the point cloud in the world coordinate system to obtain the second coordinate of the at least one point in the image plane coordinate system corresponding to the image includes: Based on the second position data of the camera at the acquisition time corresponding to at least two frames of target images and the first coordinates of the at least one point, coordinate transformation is performed on the at least one point to obtain the second coordinates of the at least one point in the image plane coordinate system of the at least two frames of target images, wherein the target image is an image whose acquisition time is earlier than or equal to the generation time of the at least one point; The coloring process of the at least one point based on the color value at the second coordinate in the image includes: The at least one point is subjected to coloring processing based on the color value of the at least one point at the second coordinates in the at least two frames of target images.

9. The method according to claim 8, characterized in that The coloring process of the at least one point based on the color value of the at least one point at the second coordinate in the at least two frames of target images includes: Taking the average of the color values ​​of the at least one point at the second coordinates in the at least two frames of target images as the actual color value of the at least one point, and performing coloring processing on the at least one point; or, The at least two frames of target images are traversed in a preset order, and the uncolored points in the at least one point are colored using the color value of the uncolored point at the second coordinate in the current target image.

10. A laser scanning device, characterized in that: include: An inertial measurement unit, a laser radar, a camera and a point cloud coloring processing device; the laser radar is used to generate a point cloud for a target scene, the camera is used to collect images for the target scene, and the clocks between the laser radar and the camera are synchronized; The point cloud coloring processing device is used to: For each frame of image captured by the camera, based on the capture time of the image, obtain first pose data of the laser radar at the capture time, where the first pose data is used to characterize the position and pose of the laser radar in the world coordinate system corresponding to the target scene; Based on the first pose data and a first external parameter of the camera relative to the laser radar, obtaining second pose data of the camera at the acquisition time, wherein the second pose data is used to characterize the position and posture of the camera in the world coordinate system, and the first external parameter is used to characterize the relative position and relative posture of the camera relative to the laser radar in the world coordinate system; Based on the second pose data and the first coordinate of at least one point in the point cloud in the world coordinate system, performing coordinate transformation on the at least one point to obtain a second coordinate of the at least one point in the image plane coordinate system corresponding to the image; The at least one point is colored based on the color value at the second coordinate in the image.

Images