A method and device for calibrating laser point cloud coordinates for automatic tower crane driving

Abstract

The embodiment of the present invention provides a method and device for calibrating laser point cloud coordinates for automatic tower crane driving, the method comprising: projecting each point of the laser point cloud to the pixel coordinate system of a virtual camera to obtain the pixel coordinates of each point; obtaining pixel images of multiple feature maps in the pixel plane of the virtual camera according to the value and pixel coordinates of each point; identifying several feature maps from the pixel images; obtaining the laser radar calibration matrix from laser coordinates to world coordinates according to the pixel coordinates and world coordinates of the feature points in the several feature maps. The technical solution of the present invention converts the laser radar point cloud into an image through a virtual camera, solving the problem of difficulty in obtaining feature points from the laser point cloud, thereby automatically and quickly obtaining the above-mentioned calibration matrix of the laser radar, which is used to obtain its world coordinates using the laser coordinates in the tower crane environment according to the calibration matrix, so as to calculate the obstacle contour during automatic tower crane driving.

Description

Technical Field

[0001] The present invention relates to the field of intelligent control, and in particular to a method and device for calibrating laser point cloud coordinates for automatic driving of a tower crane. Background Art

[0002] Many target detection algorithms have emerged based on the data processing of a single sensor. The most commonly used sensors are based on cameras or lidars. Cameras can obtain rich two-dimensional visual information, but monocular cameras cannot obtain depth information of the environment. Therefore, they lack distance information of the target and it is difficult to accurately locate the target. In addition, visual imaging based solely on cameras is often unstable due to interference from light.

[0003] LiDAR can accurately provide the depth and distance information of the target. Its sensing method is not affected by light and bad weather, and the data collection is relatively stable. However, LiDAR only obtains point cloud data, lacking visual information such as color texture. Therefore, it is difficult to accurately identify and detect targets that are far away and have few point clouds based on radar data alone. In addition, the automatic external parameter calibration of LiDAR often requires a specific experimental environment, specific markers, etc., which is undoubtedly much more complicated than the external parameter calibration of cameras. Summary of the invention

[0004] In view of this, an embodiment of the present invention provides a method and device for laser point cloud coordinate calibration for automatic driving of a tower crane.

[0005] In the first aspect, an embodiment of the present invention provides a method for calibrating laser point cloud coordinates for automatic tower crane driving, comprising: projecting each point in the laser point cloud to the pixel coordinate system of a virtual camera to obtain the pixel coordinates of the points, wherein the laser point cloud is obtained by scanning a tower crane environment including multiple feature maps by a laser radar; obtaining pixel images of the multiple feature maps in the pixel plane of the virtual camera according to the values ​​and pixel coordinates of the points; identifying several feature maps in the multiple feature maps from the pixel images; obtaining a calibration matrix of the laser radar from world coordinates to laser coordinates according to the pixel coordinates and world coordinates of the feature points in the several feature maps, and obtaining its world coordinates using the laser coordinates in the tower crane environment according to the calibration matrix, so as to calculate the obstacle contour during automatic tower crane driving.

[0006] From the above, by projecting each point in the laser point cloud to the pixel coordinate system of the virtual camera, the pixel image on the pixel plane of the virtual camera is obtained, and a number of feature points are identified from the pixel image, so as to use the pixel coordinates and world coordinates of the identified feature points to obtain the calibration matrix of the lidar from the world coordinates to the laser coordinates, which solves the problem that it is difficult to obtain feature points directly from the laser point cloud. The obtained calibration matrix is ​​used to obtain the world coordinates according to the laser coordinates in the tower crane environment, so as to calculate the obstacle contour and plan and correct the obstacle avoidance path during the automatic driving of the tower crane, thereby realizing the automatic driving of the tower crane.

[0007] In a possible implementation of the first aspect, the spatial geometric ratio between the camera coordinates and the laser coordinates of the virtual camera remains unchanged and there is no translation, the optical axis direction of the virtual camera is consistent with the depth direction of the laser radar, and the projecting of each point in the laser point cloud onto the pixel coordinate system of the virtual camera to obtain the pixel coordinates of the points includes: setting the resolution of the pixel plane of the virtual camera according to the half field angle of the width and height of the laser point cloud, and determining the intrinsic parameter matrix of the virtual camera accordingly; and obtaining the pixel coordinates of the points according to the intrinsic parameter matrix of the virtual camera using the laser coordinates of the points.

[0008] From the above, by setting the spatial geometric ratio between the camera coordinates of the virtual camera and the laser coordinates unchanged and without translation, the optical axis direction of the virtual camera is consistent with the depth direction of the laser radar to obtain the resolution and intrinsic parameter matrix of the pixel plane of the virtual camera, and accordingly project each point in the laser point cloud to the pixel coordinate system of the virtual camera to obtain the pixel coordinates of each point in the laser point cloud, so that the feature points in the pixel image of the virtual camera formed by the laser point cloud projection are easy to identify.

[0009] In a possible implementation of the first aspect, the method of obtaining the pixel coordinates of each point using the laser coordinates of each point according to the intrinsic parameter matrix of the virtual camera includes: obtaining the homogeneous coordinates of each point in the laser point cloud in the virtual camera using the laser coordinates of each point in the laser point cloud, wherein the homogeneous coordinates are the normalization of the camera coordinates of each point in the laser point cloud in the depth direction; and obtaining the pixel coordinates of each point in the laser point cloud using the intrinsic parameter matrix of the virtual camera and the homogeneous coordinates of each point in the laser point cloud in the virtual camera.

[0010] From the above, the homogeneous coordinates of each point in the laser point cloud in the virtual camera are obtained through the laser coordinates of each point in the laser point cloud, which simplifies the process of projecting each point in the laser point cloud to the pixel coordinate system of the virtual camera, making it easier to obtain the pixel coordinates of each point in the laser point cloud.

[0011] In a possible implementation of the first aspect, the method of obtaining pixel images of the multiple feature maps in the pixel plane of the virtual camera according to the values ​​and pixel coordinates of the points includes: obtaining the grayscale of the points in the pixel plane according to the values ​​and pixel coordinates of the points; and performing grayscale interpolation difference on blank points on the pixel plane of the virtual camera according to the pixel coordinates of the points and the grayscale in the pixel plane to obtain the pixel image.

[0012] From the above, the grayscale of each point of the pixel image of the virtual camera is obtained by the reflectivity of each point of the laser point cloud, so that the pixel image retains the characteristics of the feature map and is easy to be recognized.

[0013] In a possible implementation of the first aspect, the obtaining of the calibration matrix of the lidar from world coordinates to laser coordinates according to the pixel coordinates and world coordinates of the feature points in the several feature maps includes: obtaining the calibration matrix from world coordinates to camera coordinates according to the pixel coordinates and world coordinates of the feature points in the several feature maps and the intrinsic parameter matrix of the virtual camera; obtaining the calibration matrix from world coordinates to laser coordinates of the lidar according to the calibration matrix from world coordinates to camera coordinates and the calibration matrix from camera coordinates to laser coordinates of the virtual camera.

[0014] From the above, the calibration matrix of the lidar from world coordinates to laser coordinates is obtained by identifying the pixel coordinates and world coordinates of the feature points, thereby solving the problem that the image texture of the laser point cloud is not rich and is easily affected by weather and light, making the feature map difficult to identify.

[0015] In a possible implementation manner of the first aspect, a calibration matrix from camera coordinates to laser coordinates of the virtual camera is obtained according to a rotation and a translation from camera coordinates to laser coordinates of the virtual camera.

[0016] From the above, the rotation and translation of the virtual camera from the camera coordinates to the laser coordinates can be set according to the requirements, so that the calibration matrix of the virtual camera from the camera coordinates to the laser coordinates is easy to obtain and convenient to calculate.

[0017] In a possible implementation manner of the first aspect, the feature map includes at least one of the following: a checkerboard with known size and position, and a QR code.

[0018] From the above, the checkerboard or QR code is used to make the feature points of the laser point cloud projected on the pixel image of the virtual camera redundantly identified.

[0019] In the second aspect, an embodiment of the present invention provides a device for calibrating laser point cloud coordinates for automatic tower crane driving, comprising: a virtual projection module, used to project each point in the laser point cloud to the pixel coordinate system of a virtual camera to obtain the pixel coordinates of the points, wherein the laser point cloud is obtained by scanning a tower crane environment including multiple feature maps by a laser radar; an image acquisition module, used to obtain pixel images of the multiple feature maps in the pixel plane of the virtual camera according to the values ​​and pixel coordinates of the points; a feature point recognition module, used to identify several feature maps in the multiple feature maps from the pixel images; a calibration acquisition module, used to obtain a calibration matrix of the laser radar from world coordinates to laser coordinates according to the pixel coordinates and world coordinates of the feature points in the several feature maps, and used to obtain its world coordinates using the laser coordinates in the tower crane environment according to the calibration matrix, so as to calculate the obstacle contour during automatic tower crane driving.

[0020] From the above, by projecting each point in the laser point cloud to the pixel coordinate system of the virtual camera, a pixel image on the pixel plane of the virtual camera is obtained, and a number of feature points are identified from the pixel image, so as to use the pixel coordinates and world coordinates of the identified feature points to obtain the calibration matrix of the laser radar from the world coordinates to the laser coordinates. The technical solution of the embodiment of the present invention solves the problem that it is difficult to obtain feature points directly from the laser point cloud, and uses the obtained calibration matrix to obtain its world coordinates according to the laser coordinates in the tower crane environment, so as to calculate the obstacle contour during the automatic driving of the tower crane or to plan and correct the obstacle avoidance path during the automatic driving of the tower crane, so as to realize the automatic driving of the tower crane.

[0021] In a possible implementation of the second aspect, the spatial geometric ratio between the camera coordinates and the laser coordinates of the virtual camera remains unchanged and there is no translation, the optical axis direction of the virtual camera is consistent with the depth direction of the laser radar, and the virtual projection module is specifically used to include: setting the resolution of the virtual camera pixel plane according to the half field angle of the width and height of the laser point cloud, and determining the intrinsic parameter matrix of the virtual camera accordingly; obtaining the pixel coordinates of each point according to the intrinsic parameter matrix of the virtual camera using the laser coordinates of each point.

[0022] From the above, by setting the spatial geometric ratio between the camera coordinates of the virtual camera and the laser coordinates unchanged and without translation, the optical axis direction of the virtual camera is consistent with the depth direction of the laser radar to obtain the resolution and intrinsic parameter matrix of the pixel plane of the virtual camera, and accordingly project each point in the laser point cloud to the pixel coordinate system of the virtual camera to obtain the pixel coordinates of each point in the laser point cloud, so that the feature points in the pixel image of the virtual camera formed by the laser point cloud projection are easy to identify.

[0023] In a possible implementation of the second aspect, when the virtual projection module is specifically used to obtain the pixel coordinates of each point using the laser coordinates of each point according to the intrinsic parameter matrix of the virtual camera, the laser coordinates of each point in the laser point cloud are used to obtain the homogeneous coordinates of each point in the laser point cloud in the virtual camera, and the homogeneous coordinates are the normalization of the camera coordinates of each point in the laser point cloud in the depth direction; the pixel coordinates of each point in the laser point cloud are obtained using the intrinsic parameter matrix of the virtual camera and the homogeneous coordinates of each point in the laser point cloud in the virtual camera.

[0024] From the above, the homogeneous coordinates of each point in the laser point cloud in the virtual camera are obtained through the laser coordinates of each point in the laser point cloud, which simplifies the process of projecting each point in the laser point cloud to the pixel coordinate system of the virtual camera, making it easier to obtain the pixel coordinates of each point in the laser point cloud.

[0025] In a possible implementation of the second aspect, the image acquisition module is specifically used to: obtain the grayscale of each point in the pixel plane according to the value of each point and the pixel coordinates; perform grayscale interpolation on the blank points on the pixel plane of the virtual camera according to the pixel coordinates of each point and the grayscale in the pixel plane to obtain the pixel image.

[0026] From the above, the grayscale of each point of the pixel image of the virtual camera is obtained by the reflectivity of each point of the laser point cloud, so that the pixel image retains the characteristics of the feature map and is easy to be recognized.

[0027] In a possible implementation of the second aspect, the calibration acquisition module is specifically used to include: obtaining a calibration matrix from world coordinates to camera coordinates based on the pixel coordinates and world coordinates of the feature points in the several feature maps and the intrinsic parameter matrix of the virtual camera; obtaining a calibration matrix from world coordinates to laser coordinates of the laser radar based on the calibration matrix from world coordinates to camera coordinates and the calibration matrix from camera coordinates to laser coordinates of the virtual camera.

[0028] From the above, the calibration matrix of the lidar from world coordinates to laser coordinates is obtained by identifying the pixel coordinates and world coordinates of the feature points, thereby solving the problem that the image texture of the laser point cloud is not rich and is easily affected by weather and light, making the feature map difficult to identify.

[0029] In a possible implementation manner of the second aspect, a calibration matrix from camera coordinates to laser coordinates of the virtual camera is obtained according to a rotation and a translation from camera coordinates to laser coordinates of the virtual camera.

[0030] From the above, the rotation and translation of the virtual camera from the camera coordinates to the laser coordinates can be set according to the requirements, so that the calibration matrix of the virtual camera from the camera coordinates to the laser coordinates is easy to obtain and convenient to calculate.

[0031] In a possible implementation manner of the second aspect, the feature map includes at least one of the following: a checkerboard with known size and position, and a QR code.

[0032] From the above, the checkerboard or QR code is used to make the feature points of the laser point cloud projected on the pixel image of the virtual camera redundantly identified.

[0033] In a third aspect, an embodiment of the present invention provides a tower crane controller, comprising the device described in any embodiment of the second aspect.

[0034] In a fourth aspect, an embodiment of the present invention provides a tower crane, comprising: the tower crane controller, camera and laser radar described in the third aspect.

[0035] In a fifth aspect, an embodiment of the present invention provides a computing device, comprising a bus; a communication interface connected to the bus; at least one processor connected to the bus; and at least one memory connected to the bus and storing program instructions, wherein when the program instructions are executed by the at least one processor, the at least one processor executes the method described in any embodiment of the first aspect.

[0036] In a sixth aspect, an embodiment of the present invention provides a computer-readable storage medium having program instructions stored thereon, wherein when the program instructions are executed by a computer, the computer executes the method described in any one of the embodiments of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 It is a flow chart of a method for laser point cloud coordinate calibration for automatic driving of a tower crane according to a first embodiment of the present invention;

[0038] Figure 2A It is a flow chart of a second embodiment of a method for laser point cloud coordinate calibration for automatic driving of a tower crane according to the present invention;

[0039] Figure 2B It is a schematic diagram of the direction relationship of each coordinate system in Embodiment 2 of a method for laser point cloud coordinate calibration for automatic driving of a tower crane of the present invention;

[0040] Figure 3 It is a structural schematic diagram of a first embodiment of a device for laser point cloud coordinate calibration for automatic driving of a tower crane according to the present invention;

[0041] Figure 4 It is a structural schematic diagram of a second embodiment of a device for laser point cloud coordinate calibration for automatic driving of a tower crane according to the present invention;

[0042] Figure 5 Schematic diagram of the structure of the computing device according to various embodiments of the present invention. DETAILED DESCRIPTION

[0043] In the following description, reference is made to “some embodiments”, which describe a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0044] In the following description, the terms "first\second\third, etc." or module A, module B, module C, etc. are only used to distinguish similar objects, or to distinguish different embodiments, and do not represent a specific ordering of the objects. It can be understood that the specific order or sequence can be interchanged where permitted, so that the embodiments of the present invention described here can be implemented in an order other than that illustrated or described here.

[0045] In the following description, the numbers representing the steps, such as S110, S120, etc., do not necessarily mean that the steps must be executed in this manner. If permitted, the order of the previous and next steps can be interchanged, or they can be executed simultaneously.

[0046] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which the present invention belongs. The terms used herein are only for the purpose of describing the embodiments of the present invention and are not intended to limit the present invention.

[0047] The present invention is used in the automatic driving scene of a tower crane. Only a laser radar is installed on the tower crane. The tower crane is relatively high, and installing a camera on it is an aerial operation, which is inconvenient. At the same time, during the driving process of the tower crane, the position of the laser radar changes with the driving process, and the laser coordinate system of the laser point cloud obtained by it also changes with the position of the laser radar during the driving process. The directions of the origin and the coordinate axes will change, and the calibration matrix of the laser radar from the world coordinates to the laser coordinates (also called the external parameters of the laser radar) will also change. In the process of unmanned automatic driving of the tower crane, it is necessary to automatically and quickly obtain the calibration matrix of the laser radar from the world coordinates to the laser coordinates.

[0048] In the calibration method of the embodiment of the present invention, during the tower crane driving process, a virtual camera is constructed, and the laser point cloud in the tower crane environment is projected onto the virtual camera to obtain a pixel image; a number of feature maps (at least 5) in the pixel image are identified, and the pixel coordinates and world coordinates of the feature points in the identified feature map are obtained; according to the pixel coordinates and world coordinates of the feature points, the calibration matrix of the laser radar from the world coordinates to the laser coordinates is automatically obtained, that is, the external parameter calibration of the laser radar is completed. The calibration method of the embodiment of the present invention can obtain the calibration matrix of the laser radar from the world coordinates to the laser coordinates at any position of the tower crane driving.

[0049] The tower crane controller controls the tower crane driving process. During the tower crane driving process, the above-mentioned calibration matrix currently obtained is used to obtain the world coordinates in the tower crane environment according to the laser coordinates in the current tower crane environment. Then, the outlines of obstacles around the tower crane are identified based on this, and the world coordinates of the contour points of each obstacle and the world coordinates of the contour points of the hoisted object are obtained. The obstacle avoidance path of the tower crane is planned or corrected based on this, and the current tower crane driving trajectory points are planned; then the world coordinates of the current tower crane driving trajectory points are converted into the axis coordinates of the movement of each axis of the tower crane to control each axis of the tower crane, thereby realizing automatic driving of the tower crane without human intervention.

[0050] The following first describes various embodiments of the present invention in conjunction with the accompanying drawings.

[0051] Figure 1 The flowchart of a first embodiment of a method for laser point cloud coordinate calibration for automatic tower crane driving is shown, including steps S110 to S140.

[0052] S110: Projecting the laser point cloud in the tower crane environment to the pixel coordinate system of the virtual camera to obtain the pixel coordinates of each point of the laser point cloud.

[0053] The laser point cloud is obtained by scanning the crane environment including multiple feature maps by the laser radar, and the value of each point is obtained based on the reflected laser of the point. The laser point cloud is distributed in a dotted manner and is easily affected by weather, so it is difficult to directly identify the feature map. The laser coordinate system of the laser point cloud changes with its mapping position, and its mapping position changes with the position of the laser radar.

[0054] In some embodiments, the feature map includes at least one of the following: a checkerboard with known size and position, and a two-dimensional code map, wherein the two-dimensional code may be an AprilTag code.

[0055] In some embodiments, a calibration matrix from camera coordinates to laser coordinates of the virtual camera and an internal parameter matrix of the virtual camera are set, and then the pixel coordinates of the laser point cloud are obtained using the laser coordinates of each point of the laser point cloud according to the two matrices.

[0056] In some embodiments, the camera coordinates of the virtual camera and the laser coordinates have no proportional change and translation, the optical axis direction of the virtual camera is consistent with the depth direction of the laser radar, and the virtual camera has no distortion; because the camera coordinates of the virtual camera and the laser coordinates have no proportional change, the resolution of the virtual camera pixel plane is set according to the half field of view of the width and height of the laser point cloud, wherein the width half field of view of the laser point cloud is the inverse tangent of the ratio of half the height of the laser point cloud to the depth, and the width half field of view of the laser point cloud is the inverse tangent of the ratio of half the width of the laser point cloud to the depth; because the optical axis direction of the virtual camera is also consistent with the depth direction of the laser radar, the virtual camera has no distortion, and the intrinsic parameter matrix of the virtual camera is determined according to the resolution and the half field of view of the bandwidth and height; finally, the pixel coordinates of the laser point cloud are obtained according to the laser coordinates of each point in the laser point cloud according to the intrinsic parameter matrix of the virtual camera.

[0057] In some of the embodiments, when the pixel coordinates of the laser point cloud are obtained using the laser coordinates of each point in the laser point cloud according to the intrinsic parameter matrix of the virtual camera, because the camera coordinates of the virtual camera and the laser coordinates have no proportional change and translation, and the optical axis direction of the virtual camera is consistent with the depth direction of the laser radar, the calibration matrix from the camera coordinates to the laser coordinates of the virtual camera is first determined based on this; then, the pixel coordinates of the laser point cloud are obtained using the laser coordinates of each point in the laser point cloud according to the calibration matrix from the camera coordinates to the laser coordinates of the virtual camera and the intrinsic parameter matrix of the virtual camera.

[0058] In some other embodiments, because the camera coordinates of the virtual camera and the laser coordinates have no proportional change or translation, and the optical axis direction of the virtual camera is consistent with the depth direction of the laser radar, the laser coordinates of each point in the laser point cloud are first used to obtain the homogeneous coordinates of each point in the laser point cloud in the virtual camera, and the homogeneous coordinates are the normalized camera coordinates of each point in the laser point cloud in the depth direction; finally, the pixel coordinates of each point in the laser point cloud are obtained using the intrinsic parameter matrix of the virtual camera and the homogeneous coordinates of each point in the laser point cloud in the virtual camera.

[0059] S120: Obtain a pixel image of each feature map on the pixel plane of the virtual camera according to the value and pixel coordinates of each point in the laser point cloud.

[0060] The pixel image is the imaging of each feature map on the pixel plane of the virtual camera, which is a grayscale image.

[0061] In some embodiments, the value of each point in the laser point cloud is used as the grayscale of each point in the pixel plane. The value of each point in the laser point cloud is obtained based on the reflected laser of the point during laser scanning, which can be the reflectivity, reflection intensity, etc. of the laser.

[0062] In some embodiments, due to the point distribution of the laser point cloud, the points of the projected image on the pixel plane are not rich enough, and the feature map cannot be identified. In addition, according to the determined resolution of the pixel plane, the pixel coordinates of each point in the laser point cloud and the grayscale in the pixel plane are used to perform grayscale interpolation on the blank points on the pixel plane of the virtual camera to obtain a pixel image.

[0063] S130: Identify several feature maps from the plurality of feature maps from the pixel image.

[0064] Among them, the recognition methods include image processing, machine learning or deep learning.

[0065] In some embodiments, before identifying some feature maps among the multiple feature maps, the pixel image is corrected according to the pixel image and the intrinsic parameter matrix and distortion matrix of the virtual camera to obtain a second image of the multiple feature maps in the pixel plane of the virtual camera; and then some feature maps are identified from the second image.

[0066] In some embodiments, at least five feature maps are identified from the pixel image to improve the accuracy of the calibration.

[0067] From the above, the dry feature map is identified from the pixel image, which solves the difficulty of directly identifying the feature map from the laser point cloud, thereby realizing the calibration from the world coordinates to the laser coordinates of this embodiment by using the identified feature points.

[0068] S140: Obtaining a calibration matrix of the laser radar from laser coordinates to world coordinates according to the pixel coordinates and world coordinates of the feature points in the identified feature images, which is used to obtain its world coordinates using the laser coordinates in the tower crane environment according to the calibration matrix to perform automatic tower crane driving.

[0069] Among them, obtaining the laser radar's calibration matrix from laser coordinates to world coordinates is also called the laser radar's external parameter calibration.

[0070] In some embodiments, the pixel number in the pixel plane of the virtual camera is obtained according to the pixel coordinates of the feature point, and the corresponding number in the laser point cloud is obtained according to the pixel number, thereby obtaining its laser coordinates; and then, a calibration matrix from the laser coordinates to the world coordinates is obtained according to the laser coordinates and world coordinates corresponding to the feature points in the identified several feature maps.

[0071] In some embodiments, a calibration matrix from world coordinates to camera coordinates is obtained based on the pixel coordinates and world coordinates of the feature points in several identified feature maps and the intrinsic parameter matrix of the virtual camera; and then a calibration matrix from world coordinates to laser coordinates of the laser radar is obtained based on the product of the calibration matrix from world coordinates to camera coordinates of the virtual camera and the calibration matrix from camera coordinates to laser coordinates.

[0072] In some embodiments, the laser radar calibration matrix from world coordinates to laser coordinates is also used to obtain the world coordinates of the laser in the tower crane environment, and then the obstacle contour is calculated based on this to obtain its world coordinates. In other embodiments, the laser point cloud is used to identify the obstacle contour, and the laser radar calibration matrix from world coordinates to laser coordinates is also used to obtain the world coordinates of the obstacle contour based on the laser coordinates. The obstacle avoidance path of the tower crane is planned and corrected based on the world coordinates of the obstacle contour obtained to achieve automatic tower crane driving.

[0073] In summary, a method for calibrating laser point cloud coordinates for automatic tower crane driving, embodiment 1, converts the laser radar point cloud into an image through a virtual camera, and identifies feature points therefrom, thereby solving the problem that it is difficult to directly obtain feature points from the laser point cloud, and obtains the laser radar calibration matrix from world coordinates to laser coordinates based on the pixel coordinates and world coordinates of the identified feature points. The external parameter calibration of the laser radar is quickly identified during the tower crane driving process, and the current calibration matrix is ​​used to obtain its world coordinates based on the laser coordinates in the tower crane environment, so as to calculate the obstacle contour and plan and correct the obstacle avoidance path during the automatic tower crane driving, thereby realizing the automatic tower crane driving.

[0074] A second embodiment of a method for laser point cloud coordinate calibration for automatic driving of a tower crane is a specific implementation method of the first embodiment of a method for laser point cloud coordinate calibration for automatic driving of a tower crane, and has all the advantages of the first embodiment of a method for laser point cloud coordinate calibration for automatic driving of a tower crane.

[0075] Figure 2A The flow chart of a second embodiment of a method for calibrating laser point cloud coordinates for automatic driving of a tower crane is shown, which is used for calibrating the laser radar of the tower crane and the world coordinates, and includes steps S210 to S290.

[0076] S210: Obtain a laser point cloud including multiple AprilTag code images through a laser radar.

[0077] Among them, the laser radar is a semi-solid or solid-state laser radar. When it is stationary, it scans the tower crane environment field containing multiple AprilTag code images, and obtains the laser point cloud through a sufficiently long integration time. The value of each point is obtained according to the laser reflectivity of the point, and the reflectivity is mapped to the [0,255] interval. The origin Q coordinate of the laser coordinate system is (0, 0, 0).

[0078] The world coordinates of the four corners and the center point of each AprilTag code image are known, and the posture of the AprilTag code image is not limited.

[0079] The number of the plurality of AprilTag code images is at least 9, so as to improve the calibration accuracy.

[0080] S220: Setting the resolution of the virtual camera pixel plane according to the half field angle of the width and height of the laser point cloud.

[0081] First, a virtual camera is constructed. The external parameters of the virtual camera include the transformation matrix from the camera coordinates to the laser coordinates and the translation vector. There is no proportional change and translation between the camera coordinates of the virtual camera and the laser coordinates. The optical axis direction of the virtual camera is consistent with the depth direction of the laser radar. The translation vector is (0, 0, 0).

[0082] Among them, the laser coordinates of any point P in the laser point cloud are (xl, yl, zl), and its camera coordinates in the virtual camera are (xc, yc, zc). According to the conversion matrix from camera coordinates to laser radar coordinates, the camera coordinates of point P can be expressed as (-yl, -zl, xl), and the virtual optical axis direction in the zc direction is also the xl direction of the laser coordinate system. The camera coordinates of the projection point of the origin Q of the laser coordinate system in the camera coordinate system of the virtual camera are also (0, 0, 0).

[0083] Among them, the point P of the laser point cloud projected onto the XQY plane of the laser coordinate system is point U, and the point projected onto the XQZ plane of the laser coordinate system is point V. The angle between the vector QU and the vector (1,0,0) direction, that is, the direction of the optical axis of the virtual camera, is recorded as AngU, and AngU can be solved using the cosine theorem. The angle between the vector QV and the vector (1,0,0) direction, that is, the direction of the optical axis of the virtual camera, is recorded as AngV, and AngV can also be solved using the cosine theorem.

[0084] Among them, the maximum values ​​of AngU and AngV are recorded as AngUmax and AngVmax, which are the half-field angles of the width and height of the laser point cloud respectively, and tan(AngVmax)Tan(AngUmax) is the ratio of the width to the height of the laser point cloud.

[0085] Assuming that the horizontal resolution of the image of the virtual camera is 2M, the vertical resolution 2N is calculated using formula (1).

[0086] 2N=2M*tan (AngVmax) / Tan(AngUmax) (1)

[0087] Figure 2B A schematic diagram of the direction relationship of each coordinate system in Embodiment 2 of a method for laser point cloud coordinate calibration for automatic driving of a tower crane is shown.

[0088] Among them, (xl, yl, zl) represents the laser coordinates of the laser point cloud, (xc, yc, zc) represents the camera coordinates of the virtual camera, (u, v) represents the pixel coordinates of the virtual camera, there is no proportional change and translation between the camera coordinates of the virtual camera and the laser coordinates, and the optical axis direction of the virtual camera is consistent with the depth direction of the laser radar. (xw, yw, zw) represents the world coordinates, and each coordinate system is a right-handed coordinate system. Among them, the laser coordinate system changes with the position of the laser radar during the tower crane driving process, while the world coordinates generally remain unchanged.

[0089] S230: Project the laser point cloud to the pixel coordinate system of the virtual camera to obtain the pixel coordinates of each point of the laser point cloud.

[0090] Among them, the laser coordinates of any point P in the laser point cloud can be mapped to the homogeneous coordinates of the virtual camera coordinate system (±tan(AngU), ±tan(AngV), 1), the ± sign of tan(AngU) is consistent with the sign of the Y coordinate of the laser coordinate, and the ± sign of tan(AngV) is consistent with the sign of the Z coordinate of the laser coordinate. The homogeneous coordinates are the camera coordinates of the virtual camera normalized to the coordinates in the direction of the optical axis of the virtual camera. Because the homogeneous coordinates are already the coordinates normalized to the coordinates in the direction of the optical axis of the virtual camera, they can be directly obtained based on the half field angle of the width and height of the laser point cloud and the laser coordinates of each point in the laser point cloud, without calculating complex depth coordinates.

[0091] In order to describe the aspect, the homogeneous coordinate is represented by (xt, yt, 1), which is converted into the pixel coordinate of the pixel plane of the virtual camera as (K*xt+M, K*yt+N), where K=M / (tan(AngUmax), and the pixel coordinate is represented by (u, v). The above process can be completed by formula (2).

[0092]

[0093] in, is the intrinsic parameter matrix of the virtual camera. It should be emphasized that the pixel coordinates (u, v) are also rounded to correspond to the serial number of the pixel plane of the virtual camera.

[0094] S240: Obtain a pixel image of each feature map on the pixel plane of the virtual camera according to the value and pixel coordinates of each point in the laser point cloud.

[0095] The pixel image is the imaging of each feature map on the pixel plane of the virtual camera.

[0096] Among them, the value of each point in the laser point cloud is first used as the grayscale of each point in the pixel plane of the laser point cloud; and the grayscale interpolation of the blank points on the pixel plane of the virtual camera is performed according to the pixel coordinates of each point in the laser point cloud and the grayscale in the pixel plane to obtain a pixel image, so as to solve the problem that the texture of the laser point cloud is not rich and it is easily affected by weather factors, resulting in the feature map being difficult to identify.

[0097] S250: Identify some AprilTag code images among the plurality of AprilTag code images from the pixel image, and obtain feature points in the identified AprilTag code images.

[0098] The method for AprilTag code pattern recognition can be machine learning, deep learning, image processing, etc., or a combination of the two. The four corners and the center point of each AprilTag code pattern recognized are feature points.

[0099] Among them, at least 5 AprilTag code images are recognized to improve the accuracy of calibration.

[0100] S260: Obtain a calibration matrix from the world coordinates to the camera coordinates of the virtual camera according to the pixel coordinates and the world coordinates of the feature points in the identified AprilTag code image.

[0101] In the case where the virtual camera is distortion-free, the conversion from the world coordinates to the pixel coordinates of the virtual camera using the intrinsic parameter matrix of the virtual camera can be described by formula (3).

[0102]

[0103] Among them, RT is the calibration matrix from the world coordinates to the camera coordinates of the virtual camera, which is a 4*4 matrix.

[0104] Among them, according to the pixel coordinates and world coordinates of the feature points in the identified several AprilTag code images, multiple equations can be used to form a system of equations about each element of RT using formula (3), and then the system of equations is solved by the numerical optimization method to obtain the calibration matrix RT from the world coordinates to the camera coordinates of the virtual camera. It should be emphasized that the number of identified AprilTag code images must satisfy the solution of the above multiple equations. For example, if RT is a 4*4 matrix, at least (4*4 / 3) rounded up 6 feature points are required to be identified, and at least (6 / 5) rounded up 2 AprilTag code images are required. In order to improve the calibration accuracy, it is generally necessary to identify 5 AprilTag code images.

[0105] S270: Obtain the current calibration matrix of the laser radar from world coordinates to laser coordinates according to the calibration matrix of the virtual camera from world coordinates to camera coordinates and the calibration matrix from camera coordinates to laser coordinates.

[0106] The calibration matrix of the laser radar from world coordinates to laser coordinates can be described by formula (4).

[0107]

[0108] Among them, TT is the calibration matrix of the lidar from world coordinates to laser coordinates, also called the extrinsic parameter matrix, which is a 4*4 matrix, RcToL is the 3*3 transformation matrix from camera coordinates to laser coordinates constructed in step S220, and RcToLW is the calibration matrix of the virtual camera from camera coordinates to laser coordinates, which is obtained by synthesizing a 4*4 matrix of RcToL and the translation vector (0, 0, 0).

[0109] It should be emphasized that during the automatic driving of the tower crane, the position of the lidar changes with the position of the tower crane, and the calibration matrix of the lidar from the world coordinates to the laser coordinates also changes, so it is necessary to automatically obtain the current calibration matrix of the lidar from the world coordinates to the laser coordinates.

[0110] S280: Project the laser point cloud into the world coordinate system according to the current calibration matrix of the laser radar from the world coordinate to the laser coordinate, and obtain the world coordinates of each point in the current tower crane environment.

[0111] Among them, the current calibration matrix of the lidar from world coordinates to laser coordinates changes with the movement position of the tower crane, so its current value is used to obtain the world coordinates of each point in the tower crane environment.

[0112] S290: Obtain the obstacle outline of the tower crane during automatic driving according to the world coordinates of each point in the current tower crane environment, so as to perform automatic driving of the tower crane.

[0113] The points in the tower crane environment include not only the contour points of obstacles during the tower crane driving process, but also the contour points of the hook or the hoisted object. In some embodiments, obstacles are identified and the world coordinates of the contour points of obstacles are determined based on the world coordinates of the points in the tower crane environment; in other embodiments, the positions of the contour points of obstacles are identified based on the laser point cloud of the tower crane environment, and the world coordinates of the contour points of obstacles are obtained.

[0114] Among them, an obstacle avoidance path is designed according to the world coordinates of each contour point of the obstacle during the tower crane driving process and the world coordinates of the contour points of the hook or the hoisted object to perform automatic tower crane driving.

[0115] Among them, after the obstacle avoidance path design is completed, the obstacle avoidance path is also corrected according to new obstacles appearing in the tower crane environment during the tower crane automatic driving process.

[0116] Among them, after the obstacle avoidance path correction is completed during the tower crane driving process, the world coordinates of the obstacle avoidance path are converted into tower crane column coordinates to control the driving of each axis of the tower crane.

[0117] Combine the following Figure 3 to Figure 4 Various embodiments of a device for laser point cloud coordinate calibration for automatic tower crane driving of the present invention are introduced.

[0118] A device for calibrating laser point cloud coordinates for automatic driving of a tower crane. Embodiment 1 is used for calibrating the laser radar and world coordinates of a tower crane. A method for calibrating laser point cloud coordinates for automatic driving of a tower crane. The method of embodiment 1 has all its advantages.

[0119] Figure 3 The structure of a first embodiment of a device for laser point cloud coordinate calibration for automatic tower crane driving is shown, including: a virtual projection module 310, an image acquisition module 320, a feature point recognition module 330 and a calibration acquisition module 340.

[0120] The virtual projection module 310 is used to project the laser point cloud in the tower crane environment to the pixel coordinate system of the virtual camera to obtain the pixel coordinates of each point in the laser point cloud. For its working principle and advantages, please refer to step S110 of the first embodiment of a method for calibrating laser point cloud coordinates for tower crane automatic driving.

[0121] The image acquisition module 320 is used to obtain the pixel image of each feature map on the pixel plane of the virtual camera according to the value and pixel coordinates of each point of the laser point cloud. For its working principle and advantages, please refer to step S120 of the first embodiment of a method for laser point cloud coordinate calibration for tower crane automatic driving.

[0122] The feature point recognition module 330 is used to recognize several feature maps from the plurality of feature maps from the pixel image. For its working principle and advantages, please refer to step S130 of the first embodiment of a method for laser point cloud coordinate calibration for automatic driving of a tower crane.

[0123] The calibration acquisition module 340 is used to obtain a calibration matrix from laser coordinates to world coordinates according to the pixel coordinates and world coordinates of the feature points in the identified feature maps, and is used to obtain its world coordinates according to the laser coordinates in the tower crane environment to perform automatic tower crane driving. For its working principle and advantages, please refer to step S140 of the first embodiment of a method for calibrating laser point cloud coordinates for automatic tower crane driving.

[0124] Embodiment 2 of a device for calibrating laser point cloud coordinates for automatic tower crane driving is used for calibrating the laser radar and world coordinates of a tower crane, and the method of embodiment 2 of a method for calibrating laser point cloud coordinates for automatic tower crane driving has all its advantages.

[0125] Figure 4 The structure of a second embodiment of a device for laser point cloud coordinate calibration for tower crane automatic driving is shown, including: a point cloud acquisition module 410, a resolution determination module 420, a virtual projection module 430, an image acquisition module 440, a feature point recognition module 450, a camera calibration acquisition module 460, a calibration acquisition module 470, a world coordinate acquisition module 480 and a tower crane automatic driving module 490.

[0126] The point cloud acquisition module 410 is used to obtain a laser point cloud containing multiple AprilTag code images through a laser radar. For its working principle and advantages, please refer to step S210 of the second embodiment of a method for laser point cloud coordinate calibration for automatic tower crane driving.

[0127] The resolution determination module 420 is used to set the resolution of the virtual camera pixel plane according to the half field angle of the width and height of the laser point cloud. For its working principle and advantages, please refer to step S220 of the second embodiment of a method for laser point cloud coordinate calibration for tower crane automatic driving.

[0128] The virtual projection module 430 is used to project the laser point cloud to the pixel coordinate system of the virtual camera to obtain the pixel coordinates of each point of the laser point cloud. For its working principle and advantages, please refer to step S230 of the second embodiment of a method for calibrating laser point cloud coordinates for automatic tower crane driving.

[0129] The image acquisition module 440 is used to obtain the pixel image of each feature map on the pixel plane of the virtual camera according to the value and pixel coordinates of each point of the laser point cloud. For its working principle and advantages, please refer to step S240 of the second embodiment of a method for laser point cloud coordinate calibration for tower crane automatic driving.

[0130] The feature point recognition module 450 is used to recognize several AprilTag code images in the plurality of AprilTag code images from the pixel image, and obtain the feature points in the recognized AprilTag code images. For its working principle and advantages, please refer to step S250 of the second embodiment of a method for laser point cloud coordinate calibration for automatic tower crane driving.

[0131] The camera calibration acquisition module 460 is used to obtain a calibration matrix from the world coordinates to the camera coordinates of the virtual camera according to the pixel coordinates and world coordinates of the feature points in the identified AprilTag code map. For its working principle and advantages, please refer to step S260 of the second embodiment of a method for laser point cloud coordinate calibration for tower crane automatic driving.

[0132] The calibration acquisition module 470 is used to obtain the current calibration matrix of the laser radar from world coordinates to laser coordinates according to the calibration matrix of the virtual camera from world coordinates to camera coordinates and the calibration matrix from camera coordinates to laser coordinates. For its working principle and advantages, please refer to step S270 of the second embodiment of a method for laser point cloud coordinate calibration for tower crane automatic driving.

[0133] The world coordinate acquisition module 480 is used to project the laser point cloud into the world coordinate system according to the current calibration matrix from the world coordinate to the laser coordinate of the laser radar, and obtain the world coordinates of each point in the current tower crane environment. For its working principle and advantages, please refer to step S280 of the second embodiment of a method for calibrating laser point cloud coordinates for automatic tower crane driving.

[0134] The tower crane automatic driving module 490 is used to obtain the obstacle outline of the tower crane automatic driving according to the world coordinates of each point in the current tower crane environment, so as to perform the tower crane automatic driving. For its working principle and advantages, please refer to step S290 of the second embodiment of a method for laser point cloud coordinate calibration for tower crane automatic driving.

[0135] An embodiment of the present invention provides a tower crane controller, including a device described in Embodiment 1 of a device for calibrating laser point cloud coordinates for automatic driving of a tower crane or a device described in Embodiment 2 of a device for calibrating laser point cloud coordinates for automatic driving of a tower crane.

[0136] The present invention provides a tower crane embodiment, comprising: a tower crane controller as described in a tower crane controller embodiment, a camera and a laser radar.

[0137] The embodiment of the present invention further provides a computing device, Figure 5 Detailed introduction.

[0138] The computing device 500 includes a processor 510 , a memory 520 , a communication interface 530 , and a bus 540 .

[0139] It should be understood that the communication interface 530 in the computing device 500 shown in this figure can be used to communicate with other devices.

[0140] The processor 510 may be connected to a memory 520. The memory 520 may be used to store the program code and data. Therefore, the memory 520 may be a storage unit inside the processor 510, or an external storage unit independent of the processor 510, or a component including a storage unit inside the processor 510 and an external storage unit independent of the processor 510.

[0141] Optionally, the computing device 500 may further include a bus 540. The memory 520 and the communication interface 530 may be connected to the processor 510 via the bus 540. The bus 540 may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. The bus 550 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, the figure is represented by only one line, but it does not mean that there is only one bus or one type of bus.

[0142] It should be understood that in the embodiment of the present invention, the processor 510 may adopt a central processing unit (CPU). The processor may also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. Alternatively, the processor 510 may adopt one or more integrated circuits to execute relevant programs to implement the technical solutions provided in the embodiment of the present invention.

[0143] The memory 520 may include a read-only memory and a random access memory, and provides instructions and data to the processor 510. A portion of the processor 510 may also include a nonvolatile random access memory. For example, the processor 510 may also store information on the device type.

[0144] When the computing device 500 is running, the processor 510 executes the computer-executable instructions in the memory 520 to perform the operating steps of the method embodiment.

[0145] It should be understood that the computing device 500 according to an embodiment of the present invention can correspond to the corresponding subjects in the methods according to the various embodiments of the present invention, and the above-mentioned and other operations and / or functions of each module in the computing device 500 are respectively for implementing the corresponding processes of each method of the present embodiment. For the sake of brevity, they will not be repeated here.

[0146] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0147] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0148] In the several embodiments provided by the present invention, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0149] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0150] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0151] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present invention. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0152] An embodiment of the present invention further provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, it is used to execute the operating steps of the method embodiment.

[0153] The computer storage medium of the embodiment of the present invention can adopt any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples (non-exhaustive table) of computer-readable storage media include, an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, a computer-readable storage medium can be any tangible medium containing or storing a program, which can be used by an instruction execution system, device or device or used in combination with it.

[0154] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, which carry computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0155] The program code embodied on the computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0156] Computer program code for performing the operations of the present invention may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0157] Note that the above are only preferred embodiments of the present invention and the technical principles used. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in more detail through the above embodiments, the present invention is not limited to the above embodiments, and may also include more other equivalent embodiments without departing from the concept of the present invention, all of which belong to the scope of protection of the present invention.

Claims

1. A method for calibrating laser point cloud coordinates for automatic tower crane driving, characterized in that: include: Projecting each point in the laser point cloud to the pixel coordinate system of the virtual camera to obtain the pixel coordinates of each point, wherein the laser point cloud is obtained by scanning a tower crane environment including multiple feature maps by a laser radar; Obtain pixel images of the plurality of feature maps on a pixel plane of a virtual camera according to the values ​​and pixel coordinates of the points; identifying a number of feature maps from the plurality of feature maps from the pixel image; Obtaining a calibration matrix of the laser radar from world coordinates to laser coordinates according to the pixel coordinates and world coordinates of the feature points in the plurality of feature images, and using the laser coordinates in the tower crane environment to obtain its world coordinates according to the calibration matrix, so as to calculate the obstacle contour during automatic driving of the tower crane; The spatial geometric ratio between the camera coordinates and the laser coordinates of the virtual camera remains unchanged and has no translation, the optical axis direction of the virtual camera is consistent with the depth direction of the laser radar, and the pixel coordinates of each point in the laser point cloud are projected to the pixel coordinate system of the virtual camera to obtain the pixel coordinates of each point, including: The resolution of the virtual camera pixel plane is set according to the half field of view of the width and height of the laser point cloud, and the intrinsic parameter matrix of the virtual camera is determined accordingly; the pixel coordinates of each point are obtained using the laser coordinates of each point according to the intrinsic parameter matrix of the virtual camera.

2. The method according to claim 1, characterized in that: The step of obtaining the pixel coordinates of each point by using the laser coordinates of each point according to the intrinsic parameter matrix of the virtual camera includes: The laser coordinates of each point in the laser point cloud are used to obtain the homogeneous coordinates of each point in the laser point cloud in the virtual camera, and the homogeneous coordinates are the normalization of the camera coordinates of each point in the laser point cloud in its depth direction; The pixel coordinates of each point in the laser point cloud are obtained by using the intrinsic parameter matrix of the virtual camera and the homogeneous coordinates of each point in the laser point cloud in the virtual camera.

3. The method according to claim 1, characterized in that: The step of obtaining pixel images of the plurality of feature maps on the pixel plane of the virtual camera according to the values ​​and pixel coordinates of the points comprises: Obtaining the grayscale of each point in the pixel plane according to the value and pixel coordinates of each point; Grayscale interpolation is performed on blank points on the pixel plane of the virtual camera according to the pixel coordinates of each point and the grayscale in the pixel plane to obtain the pixel image.

4. The method according to claim 1, characterized in that: The step of obtaining a calibration matrix of the laser radar from world coordinates to laser coordinates according to the pixel coordinates and world coordinates of the feature points in the plurality of feature maps comprises: Obtaining a calibration matrix from world coordinates to camera coordinates according to pixel coordinates and world coordinates of feature points in the plurality of feature maps and an intrinsic parameter matrix of the virtual camera; The calibration matrix from world coordinates to laser coordinates of the laser radar is obtained according to the product of the calibration matrix from world coordinates to camera coordinates and the calibration matrix from camera coordinates to laser coordinates of the virtual camera.

5. The method according to claim 4, characterized in that: The calibration matrix from the camera coordinates to the laser coordinates of the virtual camera is obtained according to the rotation and translation from the camera coordinates to the laser coordinates of the virtual camera.

6. The method according to claim 1, characterized in that: The feature map includes at least one of the following: a chessboard with known size and position, and a QR code.

7. A device for laser point cloud coordinate calibration for automatic tower crane driving, characterized in that: include: A virtual projection module, used to project each point in the laser point cloud to the pixel coordinate system of the virtual camera to obtain the pixel coordinates of each point, wherein the laser point cloud is obtained by scanning a tower crane environment including multiple feature maps by a laser radar; An image acquisition module, used for acquiring pixel images of the plurality of feature maps on a pixel plane of a virtual camera according to the values ​​and pixel coordinates of the points; A feature point recognition module, used for recognizing several feature maps from the plurality of feature maps from the pixel image; A calibration acquisition module, used for obtaining a calibration matrix from world coordinates to laser coordinates of the laser radar according to the pixel coordinates and world coordinates of the feature points in the plurality of feature images, and used for obtaining its world coordinates using the laser coordinates in the tower crane environment according to the calibration matrix, so as to calculate the obstacle contour during automatic driving of the tower crane; The spatial geometric ratio between the camera coordinates and the laser coordinates of the virtual camera remains unchanged and has no translation, the optical axis direction of the virtual camera is consistent with the depth direction of the laser radar, and the pixel coordinates of each point in the laser point cloud are projected to the pixel coordinate system of the virtual camera to obtain the pixel coordinates of each point, including: The resolution of the virtual camera pixel plane is set according to the half field of view of the width and height of the laser point cloud, and the intrinsic parameter matrix of the virtual camera is determined accordingly; the pixel coordinates of each point are obtained using the laser coordinates of each point according to the intrinsic parameter matrix of the virtual camera.

8. A computing device, characterized in that include, bus; A communication interface connected to the bus; at least one processor connected to the bus; as well as At least one memory is connected to the bus and stores program instructions, and when the program instructions are executed by the at least one processor, the at least one processor executes the method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: Program instructions are stored thereon, and when the program instructions are executed by a computer, the computer is caused to execute the method according to any one of claims 1 to 6.

Images