Automatic driving method, device, equipment and storage medium for tower crane

Abstract

The present invention provides an automatic driving method, device, equipment and storage medium for a tower crane. Its technical solution includes: obtaining a first three-dimensional environmental cloud map around the tower crane based on sensor data, the first three-dimensional environmental cloud map including the distribution and outline of objects around the tower crane; obtaining a planned trajectory for the hook's movement based on the first three-dimensional environmental cloud map, the tower crane's lifting point and the hook drop point; controlling the hook to move point by point along the planned trajectory, and detecting whether there are obstacles around the hook based on newly acquired sensor data during the hook's movement; when an obstacle exists, adding the obstacle to the first three-dimensional environmental cloud map, and replanning the remaining planned trajectory of the hook, so as to control the hook to avoid the obstacle during continued movement. This method embodiment continuously iterates the planned trajectory based on surrounding obstacles during the tower crane's movement, reasonably and efficiently avoiding obstacles, thereby improving the operating efficiency and safety of the tower crane and reducing the labor intensity of the tower crane operator.

Description

Technical Field

[0001] The present invention belongs to the field of safe production, and in particular relates to an automatic driving method, device, equipment and storage medium for a tower crane. Background Art

[0002] There are two main ways to drive a tower crane:

[0003] 1) In the tower crane cab, the operator manually operates the crane. Furthermore, there is a method that combines hook visualization, using a camera to track and capture the hook position image, which is provided to the operator in real time to assist the operator in obstacle avoidance operations and complete the lifting task;

[0004] 2) In remote driving mode, combined with auxiliary path planning, the camera installed on the tower provides video information of the hook to the remote cockpit and provides it to the tower operator as the tower operator's operating field of view. The tower operator then performs remote operation to complete the lifting task.

[0005] In the first method, the operator operates the crane from the cab, which is labor-intensive and has high operational risks. In the second method, although the crane can be remotely operated, it is difficult to determine whether there are obstacles based on the image remotely, and the path still needs to be changed manually if there are obstacles, so the operational risk is still high.

[0006] Comparative document CN202110727448.3 discloses a method and system for automatically planning a tower crane hoisting path. This system creates a three-dimensional model of the construction site according to a preset modeling strategy. Obstacle points are removed based on the coordinates of the hoisted objects or hooks. This system then plans an aerial hoisting path based on a feasible set of points within the construction site, and operates on the hoisted objects. This technology, which plans the path based on a pre-established three-dimensional model of the construction site, cannot address the issue of newly-emerging obstacles around the tower crane. Summary of the Invention

[0007] In view of this, an embodiment of the present invention provides an automatic driving method, device, equipment and storage medium for a tower crane, the technical solution of which includes: obtaining a first three-dimensional environmental cloud map around the tower crane based on sensor data, the first three-dimensional environmental cloud map including the distribution and outline of objects around the tower crane; obtaining a planned trajectory for the movement of the hook based on the first three-dimensional environmental cloud map, the lifting point and the hook drop point of the tower crane; controlling the hook to move point by point according to the planned trajectory, and detecting whether there are obstacles around the hook based on the newly acquired sensor data during the movement of the hook; when an obstacle exists, the obstacle is added to the first three-dimensional environmental cloud map, and the remaining planned trajectory of the hook that has not moved is re-planned to control the hook to avoid the obstacle during the continued movement. This method embodiment continuously iterates the planned trajectory based on the surrounding obstacles during the movement of the tower crane, reasonably and efficiently avoids obstacles, improves the operating efficiency and safety of the tower crane, and reduces the labor intensity of the tower crane operator.

[0008] In a first aspect, an embodiment of the present invention provides an automatic driving method for a tower crane, comprising: obtaining a first three-dimensional environmental cloud map around the tower crane based on sensor data, the first three-dimensional environmental cloud map including the distribution and outlines of objects around the tower crane; obtaining a planned trajectory for the movement of the tower crane hook based on the first three-dimensional environmental cloud map, the lifting point, and the hook drop point of the tower crane; controlling the hook to move point by point along the planned trajectory, and detecting whether there are obstacles around the hook during the movement of the hook based on newly acquired sensor data, wherein the distance between the obstacle and the planned trajectory is less than a safety distance; when an obstacle exists around the hook, the obstacle is added to the first three-dimensional point cloud map, and the remaining planned trajectory of the hook that has not moved is replanned to control the hook to avoid the obstacle during continued movement. In some embodiments, the replanned trajectory maintains the continuity of the velocity and acceleration of each point.

[0009] As described above, the above method is used to introduce autonomous driving technology into the movement of the tower crane, continuously iteratively plan the trajectory according to the surrounding obstacles, and reasonably and efficiently avoid obstacles, thereby improving the operating efficiency and safety of the tower crane and reducing the labor intensity of the tower crane operators.

[0010] In a possible implementation of the first aspect, obtaining a first three-dimensional environmental cloud map around the tower crane based on sensor data includes: obtaining video data and / or radar data around the tower crane through a sensor; processing the video data and / or the radar data to identify the positions and contours of objects around the tower crane, and obtaining a first three-dimensional environmental cloud map.

[0011] As described above, the above method is used to construct a three-dimensional environment cloud map in advance through the data obtained by the sensors, which makes it easier to identify obstacles in the environment and enable the planned trajectory to avoid these obstacles.

[0012] In a possible implementation of the first aspect, the method of detecting whether there are obstacles around the hook based on sensor data during the movement of the hook includes: obtaining video data and / or radar data around the hook through sensors when reaching each trajectory point; processing the video data and / or the radar data to identify the positions and contours of objects around the hook, and obtaining a second three-dimensional environmental cloud map around the hook, the second three-dimensional environmental cloud map including the distribution and contours of objects around the hook; and determining whether each object in the second three-dimensional environmental cloud map is an obstacle.

[0013] As described above, the above method is used to construct a three-dimensional environment cloud map using the data obtained by the sensors during the movement of the tower crane, which is convenient for identifying new obstacles in the environment during the movement.

[0014] In a possible implementation of the first aspect, the re-planning of the remaining unmoved planned trajectory of the hook includes: adding an obstacle to the first three-dimensional environmental cloud map; obtaining a safety point of the obstacle, the safety point being the first point on the planned trajectory whose outline distance from the obstacle is equal to the safety distance; using the safety point as the new lifting point, and obtaining a new planned trajectory for the movement of the tower crane hook based on the new first three-dimensional environmental cloud map and the hook drop point.

[0015] Based on the above, the above method is used to replan the remaining movement trajectory according to the new obstacles identified during the tower crane movement to avoid the obstacles and realize automatic driving of the automatic tower crane.

[0016] In a possible implementation of the first aspect, the distance between an object and the planned trajectory of the hook without movement is the minimum distance between each point on the contour of the object and each point on the planned trajectory of the hook without movement.

[0017] From the above, the distance between the object and the unplanned trajectory of the hook is obtained according to the above method, so as to accurately judge the obstacle.

[0018] In a possible implementation of the first aspect, the hook is controlled to avoid the obstacle while continuing to move, including: controlling the hook to decelerate from the trajectory point where the obstacle is discovered to the safety point; and controlling the hook to move according to the new planned trajectory from the safety point to the hook landing point.

[0019] From the above, the above method is used to control the remaining movement of the tower crane to avoid obstacles and realize automatic driving of the automatic tower crane.

[0020] In a possible implementation of the first aspect, the sensor includes at least one of the following: a camera at the boom lifting axis, a laser radar at the boom lifting axis, a camera at the boom luffing axis, a laser radar at the boom luffing axis, and a camera at the front end of the boom, and each sensor is aimed at the hook.

[0021] From the above, the above sensors are used to facilitate comprehensive identification of obstacles in the environment.

[0022] In one possible implementation of the first aspect, the processing of the video data and / or the radar data includes at least: converting the data from each sensor into three-dimensional point cloud data, and performing point cloud stitching, point cloud denoising, and point cloud clustering; and obtaining a target bounding box for the point cloud data of each cluster after the point cloud clustering to obtain the outline and position of each object in the point cloud;

[0023] From the above, the above point cloud processing method is used to combine the data of different sensors to identify objects and locations in the environment, thereby constructing an environmental point cloud map that can be used for automatic driving of tower cranes.

[0024] In the second aspect, an embodiment of the present invention provides an automatic driving device for a tower crane, comprising: a trajectory planning module, a motion controller and a trajectory controller; the trajectory planning module is used to obtain a first three-dimensional environmental cloud map around the tower crane, and obtain a planned trajectory for the movement of the tower crane hook based on the first three-dimensional environmental cloud map, the lifting point and the hook dropping point of the tower crane, the first three-dimensional environmental cloud map including the distribution and contours of objects around the tower crane; the motion controller is used to control the hook to move point by point according to the planned trajectory; the trajectory controller is used to detect whether there are obstacles around the hook according to newly acquired sensor data during the movement of the hook, and when there are obstacles around the hook, add the obstacles to the first three-dimensional environmental cloud map, and re-plan the unmoved planned trajectory of the hook to control the hook to avoid the obstacle during continued movement, and the distance between the obstacle and the unmoved planned trajectory of the hook is less than the safety distance.

[0025] From the above, the above-mentioned device is used to introduce automatic driving technology during the movement of the tower crane, and the trajectory is continuously iteratively planned according to the surrounding obstacles, so that obstacles can be avoided reasonably and efficiently, thereby improving the operating efficiency and safety of the tower crane and reducing the labor intensity of the tower crane operators.

[0026] In a possible implementation of the second aspect, when obtaining the first three-dimensional environmental cloud map, the trajectory planning module is specifically used to: obtain video data and / or radar data around the tower crane through sensors; process the video data and / or the radar data to identify the position and outline of objects around the tower crane, and obtain the first three-dimensional environmental cloud map.

[0027] From the above, the data obtained by the above device through the sensor is used to build a three-dimensional environment cloud map in advance, which is convenient for identifying obstacles in the environment and allowing the planned trajectory to avoid these obstacles.

[0028] In a possible implementation of the second aspect, the trajectory controller is specifically used to identify obstacles, including: obtaining video data and / or radar data around the hook through sensors when reaching each trajectory point; processing the video data and / or the radar data to identify the positions and contours of objects around the hook, and obtaining a second three-dimensional environmental cloud map around the hook, the second three-dimensional environmental cloud map including the distribution and contours of objects around the hook; and determining whether each object in the second three-dimensional environmental cloud map is an obstacle.

[0029] As described above, the data obtained by the sensors during the movement of the tower crane by the above-mentioned device is used to construct a three-dimensional environmental cloud map, which is convenient for identifying new obstacles that appear in the environment during the movement.

[0030] In a possible implementation of the second aspect, the trajectory replanning module is specifically used to, when replanning the motion trajectory, include: obtaining the safety point of the obstacle, the safety point being the first point on the planned trajectory whose contour distance from the obstacle is equal to the safety distance; using the safety point as the new lifting point, according to the new first three-dimensional environmental cloud map and the hook drop point, obtaining a new planned trajectory for the tower crane hook movement.

[0031] Based on the above, the above method is used to replan the remaining movement trajectory according to the new obstacles identified during the tower crane movement to avoid the obstacles and realize automatic driving of the automatic tower crane.

[0032] In a possible implementation of the second aspect, the distance between an object and the planned trajectory of the hook without movement is the minimum distance between each point on the contour of the object and each point on the planned trajectory of the hook without movement.

[0033] From the above, the distance between the object and the unplanned trajectory of the hook is obtained according to the above device, so as to accurately judge the obstacle.

[0034] In a possible implementation of the second aspect, the motion controller is also used to control the hook to decelerate from the trajectory point where the obstacle is discovered to the safety point, and to control the hook to move according to the new planned trajectory from the safety point to the hook landing point.

[0035] From the above, the above device is used to control the remaining movement of the tower crane to avoid obstacles and realize automatic driving of the automatic tower crane.

[0036] In a possible implementation of the second aspect, the sensor includes at least one of the following: a camera at the boom lifting axis, a laser radar at the boom lifting axis, a camera at the boom luffing axis, a laser radar at the boom luffing axis, and a camera at the front end of the boom, and each sensor is aimed at the hook.

[0037] From the above, the above sensors are used to facilitate comprehensive identification of obstacles in the environment.

[0038] In a possible implementation of the second aspect, the processing of the video data and / or the radar data at least includes: converting the data from each sensor into three-dimensional point cloud data, and performing point cloud stitching, point cloud denoising, and point cloud clustering; and obtaining a target bounding box for the point cloud data of each cluster after the point cloud clustering to obtain the outline and position of each object in the point cloud;

[0039] As described above, the above-mentioned point cloud processing function is used to combine data from different sensors to identify objects and locations in the environment, thereby constructing an environmental point cloud map that can be used for autonomous driving of tower cranes.

[0040] In a third aspect, an embodiment of the present invention provides an automatic driving system for a tower crane, comprising: a sensor for acquiring environmental data surrounding the tower crane; and a controller for executing the method described in any embodiment of the first aspect.

[0041] In a fourth aspect, an embodiment of the present invention provides a computing device, comprising a bus; a communication interface connected to the bus; at least one processor connected to the bus; and at least one memory connected to the bus and storing program instructions, wherein the program instructions, when executed by the at least one processor, enable the at least one processor to execute the method described in any embodiment of the first aspect.

[0042] In a fifth aspect, an embodiment of the present invention provides a computer-readable storage medium having program instructions stored thereon, wherein the program instructions, when executed by a computer, cause the computer to execute the method described in any one of the implementation methods of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 This is a flow chart of a first embodiment of an automatic driving method for a tower crane according to the present invention;

[0044] Figure 2 This is a schematic diagram of the distribution of sensors on a tower crane in Example 2 of an automatic driving method for a tower crane of the present invention;

[0045] Figure 3 This is a flow chart of a second embodiment of an automatic driving method for a tower crane according to the present invention;

[0046] Figure 4 This is a structural schematic diagram of a first embodiment of an automatic driving device for a tower crane according to the present invention;

[0047] Figure 5 This is a structural schematic diagram of a second embodiment of an automatic driving device for a tower crane of the present invention.

[0048] Figure 6 Schematic diagram of the structure of a computing device embodiment of the present invention.

[0049] in, Figure 2 The numbers in the figure are as follows: 1, boom luffing axis; 2, laser radar at the boom lifting axis; 3, camera at the boom lifting axis; 4, laser radar at the boom luffing axis; 5, camera at the luffing axis; 6, camera at the front end of the boom; 7, hook. DETAILED DESCRIPTION

[0050] In the following description, reference is made to “some embodiments”, which describes a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0051] In the following description, the terms "first\second\third, etc." or module A, module B, module C, etc. are only used to distinguish similar objects, or to distinguish different embodiments, and do not represent a specific ordering of the objects. It can be understood that the specific order or sequence can be interchanged where permitted, so that the embodiments of the present invention described herein can be implemented in an order other than that illustrated or described herein.

[0052] In the following description, the numbers representing the steps, such as S110, S120, etc., do not necessarily mean that the steps must be executed in this manner. If permitted, the order of the steps can be interchanged or they can be executed simultaneously.

[0053] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which the present invention pertains. The terms used herein are for the purpose of describing embodiments of the present invention only and are not intended to limit the present invention.

[0054] An embodiment of the present invention provides an automatic driving method, device, equipment and storage medium for a tower crane, the technical solution of which includes: obtaining a first three-dimensional environmental cloud map around the tower crane based on sensor data, the first three-dimensional environmental cloud map including the distribution and outline of objects around the tower crane; obtaining a planned trajectory for the movement of the hook based on the first three-dimensional environmental cloud map, the lifting point and the hook drop point of the tower crane; controlling the hook to move point by point according to the planned trajectory, and detecting whether there are obstacles around the hook based on the newly acquired sensor data during the movement of the hook; when an obstacle exists, the obstacle is added to the first three-dimensional environmental cloud map, and the remaining planned trajectory of the hook that has not moved is replanned to control the hook to avoid the obstacle during the continued movement. This method embodiment continuously iterates the planned trajectory based on the surrounding obstacles during the movement of the tower crane, reasonably and efficiently avoids obstacles, improves the operating efficiency and safety of the tower crane, and reduces the labor intensity of the tower crane operator.

[0055] The following describes various embodiments of the present invention with reference to the accompanying drawings.

[0056] The following first describes the tower crane system used in various embodiments of the present invention. The system comprises a tower crane, a sensor, and a controller. The sensor is used to acquire video and / or laser point clouds around the crane, and may include at least one of the following sensors: a camera, a lidar, an IMU, an ultrasonic radar, or a millimeter-wave radar. The controller is used to plan the trajectory of the hook and control its movement based on the planned trajectory.

[0057] In some embodiments, the tower crane includes three axes: a tower crane arm rotation axis, a tower crane arm lifting axis, and a hook luffing axis. In some embodiments, each sensor includes at least one of the following: a camera at the tower crane arm lifting axis, a laser radar at the tower crane arm lifting axis, a camera at the tower crane arm luffing axis, a laser radar at the tower crane arm luffing axis, and a camera at the front end of the tower crane, with each sensor aimed at the tower crane hook.

[0058] The following combination Figure 1 A first embodiment of an automatic driving method for a tower crane is introduced.

[0059] An embodiment of an automatic driving method for a tower crane is implemented in a controller of a tower crane system. The controller uses sensors to collect information about the surroundings of the tower crane in advance to establish a three-dimensional environmental point cloud map. During the movement of the hook, the controller uses the environmental information around the hook collected by the sensors to identify obstacles on the running trajectory, and dynamically adds the obstacle information to the three-dimensional environmental point cloud map and replans a new trajectory, running along the new path to avoid obstacles.

[0060] Figure 1 The flowchart of a first embodiment of an automatic driving method for a tower crane is shown, which includes steps S110 to S150.

[0061] S110: Obtain a first three-dimensional environmental cloud map around the tower crane according to the sensor data, where the first three-dimensional environmental cloud map includes the distribution and outlines of objects around the tower crane.

[0062] The sensor can be installed on the tower crane or elsewhere, as long as it can cover the surrounding environment of the tower crane. The sensor data includes video data and radar data.

[0063] When obtaining a first three-dimensional environmental cloud map around the tower crane based on sensor data, the positions and contours of objects around the hook are identified through point cloud data processing, thereby obtaining the first three-dimensional environmental cloud map. The point cloud data processing includes at least one of the following: a clustering method using machine learning, image recognition using deep learning, or image segmentation and recognition.

[0064] The machine learning clustering method includes: converting the video data and radar data of each sensor into three-dimensional point cloud data, and performing point cloud stitching, point cloud denoising, and point cloud clustering; obtaining the target bounding box of the point cloud data of each class after point cloud clustering to obtain the outline and position of each object in the point cloud; and forming the first three-dimensional environment cloud map from the outline and position of each object in the point cloud.

[0065] S120: Obtaining a planned trajectory of the tower crane hook movement according to the first three-dimensional environment cloud map, the lifting point and the hook drop point of the tower crane.

[0066] The obtained planning trajectory avoids each object in the first three-dimensional environment cloud image and maintains a safe distance from the outline of each object.

[0067] The obtained planned trajectory includes the motion path of the hook and the speed of each point on each path, and the speed and acceleration at each point remain continuous.

[0068] S130: Control the tower crane hook to move point by point along the planned trajectory.

[0069] Among them, the various axes of the tower crane are controlled to move according to the coordinates and speeds in the planned trajectory, so that the hook can move point by point along the planned trajectory.

[0070] S140: During the movement of the hook, detect whether there are obstacles around the hook according to the newly acquired sensor data, and the obstacles do not belong to the first three-dimensional environment cloud map and the distance from the planned trajectory is less than the safety distance.

[0071] Among them, during the movement of the hook, the position and outline of the objects around the hook are identified based on the newly acquired sensor data. This process is the same as the process of obtaining the first point cloud map.

[0072] After obtaining the positions and contours of objects around the hook, the distances between each object around the hook and the unmoving planned trajectory are calculated, where the distance between an object and the unmoving planned trajectory is the minimum value of the distances between each point on the contour of the object and each point on the unmoving planned trajectory. When the distance between an object around the hook and the unmoving planned trajectory is less than the safety distance, the object is an obstacle.

[0073] S150: When there is an obstacle around the hook, the obstacle is added to the first three-dimensional point cloud image, and the unmoved planned trajectory is replanned to control the hook to avoid the obstacle during the continued movement.

[0074] In some embodiments, the re-planned trajectory maintains the continuity of velocity and acceleration at each point.

[0075] In summary, a first embodiment of an autonomous driving method for a tower crane utilizes environmental information collected by sensors around the hook to identify obstacles on its trajectory during hook movement, and then replans the remaining trajectory to avoid the obstacles. This method introduces an autonomous driving solution to the field of tower crane operation, rationally and efficiently avoiding obstacles, improving the crane's operating efficiency and safety while reducing the labor intensity of crane operators.

[0076] The following combination Figure 2 and Figure 3 A second embodiment of an automatic driving method for a tower crane is introduced.

[0077] Example 2 of an automatic driving method for a tower crane is a detailed implementation method of Example 1 of an automatic driving method for a tower crane. It runs in the controller of the tower crane system, and its integrated controller uses sensors to collect information about the surroundings of the tower crane in advance to establish a first three-dimensional environmental point cloud map. During the movement of the hook, the controller uses the environmental information around the hook collected by the sensors to construct a second three-dimensional environmental point cloud map, and identifies obstacles on the motion trajectory based on the two point cloud maps and the planned trajectory, and adds the obstacle information to the three-dimensional environmental point cloud map and re-plans a new trajectory, moving along the new trajectory to avoid obstacles.

[0078] In a second embodiment of an automatic driving method for a tower crane, the tower crane includes three axes: a boom rotation axis, a boom lifting axis and a boom luffing axis. The boom rotation axis and the boom lifting axis are together and located at the junction of the boom and the tower crane tower body. As the boom rises or lowers, the boom luffing axis moves the hook in a direction parallel to the boom.

[0079] Figure 2 The diagram shows the sensors installed on the tower crane, including: a camera at the boom lifting axis, a laser radar at the boom lifting axis, a camera at the boom luffing axis, a laser radar at the boom luffing axis, and a camera at the front end of the boom. Each sensor is aimed at the hook.

[0080] The entire working environment of the tower crane is scanned by rotating the boom and adjusting the boom length, and the three-dimensional information of the working environment is collected by the camera and lidar.

[0081] Figure 3 The flowchart of a second embodiment of an automatic driving method for a tower crane is shown, which includes steps S310 to S370.

[0082] S310: Obtain a first three-dimensional environmental cloud map around the tower crane using the data from each sensor through a cloud processing method.

[0083] This step includes the following sub-steps:

[0084] (1) Obtain video data and radar data around the tower crane through sensors.

[0085] The entire working environment of the tower crane is scanned by rotating the boom and adjusting the boom length, and the three-dimensional information of the working environment is collected by the camera and lidar.

[0086] (2) Identify the positions and outlines of objects around the tower crane through point metadata processing based on the video data and the radar data, and obtain a first three-dimensional environmental cloud map.

[0087] Among them, point cloud data processing includes the following processes:

[0088] 1) Convert the data from each sensor into 3D point cloud data, and perform point cloud stitching, point cloud denoising, and point cloud clustering;

[0089] 2) After point cloud clustering, the target bounding box of each class of point cloud data is obtained to obtain the outline and position of each object in the point cloud.

[0090] S320: Obtain a planned trajectory of the tower crane hook movement according to the first three-dimensional environment cloud map, the lifting point and the hook drop point of the tower crane, where the planned trajectory is a trajectory in the tower crane axis space.

[0091] The planned trajectory includes the points and velocities of each point during the hook's motion. To facilitate control, the coordinates and velocities of each point on the planned trajectory are converted into coordinates and velocities in the three dimensions of the crane's axis space: the boom's rotation axis, the boom's lifting axis, and the hook's luffing axis.

[0092] For example, the planned trajectory point set is [A,...,m,...n,...B]; A is the lifting point, B is the hook drop point, and m and n are two points in the middle.

[0093] S330: The three axes of the tower crane are controlled to drive the hook to move point by point along the planned trajectory.

[0094] Among them, the planned trajectory not only includes the position of each trajectory in the tower crane axis space, but also the speed on the three axes of the tower crane. During movement, the position and speed of the three axes of the tower crane refer to the planned trajectory.

[0095] For example, the trajectory point set [A,...m,...n,...B] is input into the motion control module of the tower crane controller, and the operation of the rotating motor, variable amplitude motor, and lifting motor is controlled according to the position and speed of each point of the planned trajectory, so that the hook runs according to the planned trajectory.

[0096] S340: During the movement of the tower crane hook, the newly acquired data from each sensor is used to detect whether there are obstacles around the hook. The obstacles do not belong to the first three-dimensional environment cloud map and the distance from the planned trajectory is less than the safety distance.

[0097] When an obstacle exists, step S350 is executed; otherwise, step S370 is executed.

[0098] This step includes the following sub-steps:

[0099] (1) Obtain video data and / or radar data around the hook through sensors.

[0100] (2) Identifying the positions and contours of objects around the hook through point metadata processing based on the video data and / or the radar data, and obtaining a second three-dimensional environmental cloud map around the hook, the second three-dimensional environmental cloud map including the distribution and contours of objects around the hook. The point cloud processing method is the same as the point cloud data processing method in step S310.

[0101] (3) Determine whether each object in the second three-dimensional environment cloud map is an obstacle.

[0102] If there is an object in the second three-dimensional environment cloud image and the distance between it and the planned trajectory is less than the safety distance, then the object is an obstacle to the motion trajectory.

[0103] Among them, after obtaining the position and contour of objects around the hook, the distance between each object around the hook and the unmoving planned trajectory is calculated. The distance between an object and the unmoving planned trajectory is the minimum value of the distance between each point on the contour of the object and each point on the unmoving planned trajectory; when the distance between an object around the hook and the unmoving planned trajectory is less than the safety distance, the object is an obstacle.

[0104] S350: Add the obstacle to the first three-dimensional environment cloud map, and re-plan the remaining movement trajectory of the tower crane hook based on the obstacle to control the tower crane hook to avoid the obstacle during the continued movement. In some embodiments, the re-planned trajectory maintains the continuity of the velocity and acceleration of each point.

[0105] This step includes the following sub-steps:

[0106] (1) Obtaining a safe point of the obstacle from the updated first three-dimensional environment cloud map, the safe point being the first point on the unmoved planned trajectory whose distance from the outline of the obstacle is equal to the safe distance;

[0107] (2) obtaining a new planned trajectory of the tower crane hook movement based on the new first three-dimensional environment cloud map, the safety point and the hook drop point;

[0108] For example, obstacle C is found at point m of the trajectory [A,...m,...n,...B], and the first point n in the non-moving planned trajectory that is a safe distance d from obstacle C is determined. Point n is a safe point, and point n is set as the starting point of the changed path; then starting from point m, point n is used as the new lifting point, keeping the hook point B, and inputting n, B and the three-dimensional environment point cloud map with obstacle C added into the trajectory planning module to solve the new planned trajectory from n to B.

[0109] S360: Control the tower crane hook to decelerate from the trajectory point where the obstacle is found, and move along the new planned trajectory after reaching the safety point corresponding to the obstacle.

[0110] This step includes the following sub-steps:

[0111] (1) Control the tower crane hook to decelerate from the trajectory point where the obstacle is found to the safety point. The speed at which the hook reaches the safety point is not limited and can be decelerated to 0.

[0112] (2) Control the tower crane hook to move from a safe point to the hook drop point according to the newly planned trajectory.

[0113] For example, the hook decelerates from point m to point n. The speed at point n can be set according to the actual trajectory and can be decelerated to 0; the new path is input into the trajectory control module, and the hook moves from point n to the hook drop point B according to the new planned trajectory.

[0114] S370: Determine whether the vehicle has reached the hook drop point.

[0115] If the crane reaches the hook drop point, the crane movement ends; otherwise, the process returns to step S330.

[0116] In summary, a second embodiment of an automatic driving method for a tower crane runs in the controller of the tower crane system, and its integrated controller uses sensors to collect information about the surroundings of the tower crane in advance to establish a first three-dimensional environmental point cloud map. During the movement of the hook, the controller uses the environmental information around the hook collected by the sensors to construct a second three-dimensional environmental point cloud map. Through the two point cloud maps and the planned trajectory, obstacles on the running trajectory can be more accurately identified, and the obstacle information is added to the three-dimensional environmental point cloud map and a new trajectory is replanned to avoid obstacles in the remaining movement.

[0117] The following combination Figure 4 and Figure 5 An embodiment of an automatic driving device for a tower crane is introduced.

[0118] Figure 4 The structure of an embodiment of an automatic driving device for a tower crane is shown, including: a trajectory planning module 410, a motion controller 420 and a trajectory controller 430, wherein the trajectory planning module 410 includes an environment point cloud acquisition module 4110 and an initial planning module 4120, the motion controller 420 includes a motion control module 4130, and the trajectory controller 430 includes: an obstacle recognition module 4140 and a trajectory replanning module 4150.

[0119] The environmental point cloud acquisition module 4110 is used to obtain a first three-dimensional environmental cloud map around the tower crane. The first three-dimensional environmental cloud map includes the distribution and outlines of objects around the tower crane. For its principles and advantages, please refer to step S110 of the first embodiment of the automatic driving method for a tower crane.

[0120] The initial planning module 4120 is used to obtain a planned trajectory of the tower crane hook movement based on the first three-dimensional environment cloud map, the tower crane's lifting point, and the hook drop point. For its principles and advantages, please refer to step S120 of the first embodiment of a tower crane automatic driving method.

[0121] The motion control module 4130 is used to control the hook to move point by point along the planned trajectory. Its principles and advantages are described in step S130 of the first embodiment of an automatic driving method for a tower crane.

[0122] Obstacle identification module 4140 is used to detect obstacles around the hook during its movement. Obstacles that are not part of the first three-dimensional environment cloud map and are less than the safe distance from the planned trajectory are not detected. For its principles and advantages, please refer to step S140 of Example 1 of a method for autonomous driving a tower crane.

[0123] When obstacles are present near the hook, trajectory replanning module 4150 adds them to the first 3D point cloud image and replans the hook's trajectory. This allows the trajectory control module to control the hook to avoid the obstacle during continued movement. For details on its principles and advantages, please refer to step S150 in Example 1 of a method for autonomous driving a tower crane.

[0124] Figure 5 The structure of a first embodiment of an automatic driving device for a tower crane is shown, including: a trajectory planning module 510 , a motion controller 520 and a trajectory controller 530 .

[0125] Among them, the trajectory planning module 510 includes an environment point cloud acquisition module 5110 and an initial planning module 5120, the motion controller 520 includes: a motion control module 5130 and a motion re-control module 5160, and the trajectory controller 530 includes an obstacle recognition module 5140, a trajectory re-planning module 5150 and a hook point judgment module 5170.

[0126] The trajectory planning module 510 includes an environment point cloud acquisition module 5110 and an initial planning module 5120 .

[0127] The environmental point cloud acquisition module 5110 is used to obtain a first three-dimensional environmental cloud map around the tower crane using the data from each sensor through cloud processing. For its principles and advantages, please refer to step S310 of the second embodiment of the automatic driving method for a tower crane.

[0128] The initial planning module 5120 is used to obtain a planned trajectory for the tower crane hook's motion based on the first 3D environment cloud map, the crane's lifting point, and the crane's hook drop point. This planned trajectory is the trajectory of the crane's axis space. For its principles and advantages, please refer to step S320 of Example 2 of a method for autonomous driving a tower crane.

[0129] The motion controller 520 includes: a motion control module 5130 , a motion re-control module 5160 and a hook drop point determination module 5170 .

[0130] The motion control module 5130 is used to control the three axes of the tower crane to drive the hook to move point by point along the planned trajectory. For its principles and advantages, please refer to step S330 of the second embodiment of the automatic driving method for a tower crane.

[0131] The motion control module 5160 is also used to control the tower crane hook to decelerate from the trajectory point where the obstacle is detected, and then move along the newly planned trajectory after reaching the safety point corresponding to the obstacle. For its principles and advantages, please refer to step S360 of the second embodiment of a tower crane automatic driving method.

[0132] The hook drop point determination module 5170 is used to determine whether the vehicle has reached the hook drop point. For its principles and advantages, please refer to step S370 of the second embodiment of the automatic driving method for a tower crane.

[0133] The trajectory controller 530 includes an obstacle recognition module 5140 and a trajectory replanning module 5150 .

[0134] Among them, the obstacle recognition module 5140 is used to use the newly acquired data from each sensor to detect whether there are obstacles around the tower crane hook during the movement of the hook. The obstacles do not belong to the first three-dimensional environmental cloud map and the distance from the planned trajectory is less than the safety distance. Its principles and advantages please refer to step S340 of embodiment 2 of a method for automatic driving of a tower crane. Among them, the trajectory replanning module 5150 is used to add the obstacles to the first three-dimensional environmental cloud map when there are obstacles around the hook, and replan the movement trajectory of the hook, so as to control the hook to avoid the obstacle during the continued movement through the trajectory control module. Its principles and advantages please refer to step S350 of embodiment 2 of a method for automatic driving of a tower crane.

[0135] In some embodiments, a possible implementation of the trajectory controller 530 includes the following process, where (1) and (2) are implemented in the obstacle recognition module 5140 , and (3) and (4) are implemented in the trajectory replanning module 5150 .

[0136] (1) During the movement of the hook, the trajectory controller 530 uses sensors to collect relevant environmental and tower crane information, and performs digital modeling to dynamically discover object targets around the hook, including the position and outline of the object.

[0137] (2) Calculate the intersection of the dynamically discovered object target and the planned trajectory. If an intersection is found, the object target is defined as an obstacle. The intersection is when the distance between the discovered object target and the planned trajectory is equal to 0 and the two overlap (i.e., the safety distance is 0).

[0138] (3) For dynamically discovered obstacles, the bounding box information of the obstacle (i.e., position and contour information) is added to the tower crane environment digital model dataset, which is the first three-dimensional environment cloud map.

[0139] (4) Assume that obstacle C is found at point m, and point n, which is a safe distance d from C, is determined to be safe. Point n is set as the starting point of the changed trajectory, and the original hook point B is maintained. n, B, and the environmental digital model data set with obstacle C added are input into the called trajectory planning module to solve the new planned trajectory from n to B.

[0140] The embodiment of the present invention also provides a computing device, Figure 6 Detailed introduction.

[0141] The computing device 600 includes a processor 610 , a memory 620 , a communication interface 630 , and a bus 640 .

[0142] It should be understood that the communication interface 630 in the computing device 600 shown in this figure can be used to communicate with other devices.

[0143] The processor 610 may be connected to a memory 620. The memory 620 may be used to store the program code and data. Therefore, the memory 620 may be a storage unit within the processor 610, an external storage unit independent of the processor 610, or a component including both a storage unit within the processor 610 and an external storage unit independent of the processor 610.

[0144] Optionally, computing device 600 may further include a bus 640. Memory 620 and communication interface 630 may be connected to processor 610 via bus 640. Bus 640 may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. Bus 650 may be classified as an address bus, a data bus, a control bus, and the like. For ease of illustration, the figure shows only one line, but this does not imply that there is only one bus or only one type of bus.

[0145] It should be understood that in the embodiment of the present invention, the processor 610 may be a central processing unit (CPU). The processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor. Alternatively, the processor 610 may be one or more integrated circuits for executing relevant programs to implement the technical solutions provided in the embodiment of the present invention.

[0146] The memory 620 may include a read-only memory and a random access memory, and provides instructions and data to the processor 610. A portion of the processor 610 may also include a non-volatile random access memory. For example, the processor 610 may also store information about the device type.

[0147] When the computing device 600 is running, the processor 610 executes the computer-executable instructions in the memory 620 to perform the operating steps of the method embodiment.

[0148] It should be understood that the computing device 600 according to an embodiment of the present invention can correspond to the corresponding subjects in executing the methods according to various embodiments of the present invention, and the above-mentioned and other operations and / or functions of each module in the computing device 600 are respectively for implementing the corresponding processes of each method of this embodiment. For the sake of brevity, they will not be repeated here.

[0149] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present invention.

[0150] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0151] In the several embodiments provided by the present invention, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0152] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0153] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0154] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0155] An embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, which is used to execute the operating steps of the method embodiment when the program is executed by a processor.

[0156] The computer storage medium of the embodiment of the present invention can adopt any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or component, or any combination of the above. More specific examples (non-exhaustive list) of computer-readable storage media include, an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, a computer-readable storage medium can be any tangible medium containing or storing a program, which can be used by an instruction execution system, device or device or used in combination with it.

[0157] A computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0158] Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0159] Computer program code for performing the operations of the present invention may be written in one or more programming languages, or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0160] Note that the above are only preferred embodiments of the present invention and the technical principles employed. Those skilled in the art will appreciate that the present invention is not limited to the specific embodiments described herein, and that various obvious changes, readjustments, and substitutions can be made by those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in detail through the above embodiments, the present invention is not limited to the above embodiments and may include many other equivalent embodiments without departing from the scope of protection of the present invention.

Claims

1. A tower crane automatic driving method, characterized in that: include: obtaining a first three-dimensional environmental cloud map around the tower crane according to the sensor data, wherein the first three-dimensional environmental cloud map includes the distribution and contours of objects around the tower crane; Obtaining a planned trajectory for the tower crane hook movement based on the first three-dimensional environmental cloud image, the lifting point, and the hook drop point of the tower crane, wherein the obtained planned trajectory avoids objects in the first three-dimensional environmental cloud image and maintains a safe distance from the outlines of the objects; The hook is controlled to move point by point along the planned trajectory, and during the movement of the hook, whether there is an obstacle around the hook is detected based on the newly acquired sensor data, wherein the distance between the obstacle and the planned trajectory of the hook is less than the safety distance, and the obstacle does not belong to the first three-dimensional environment cloud map, wherein the distance between an object and the planned trajectory of the hook is the minimum distance between each point on the contour of the object and each point on the planned trajectory of the hook; the obtained planned trajectory includes the movement path of the hook and the speed of each point on each path, and the speed and acceleration at each point are continuous; When there is an obstacle around the hook, the obstacle is added to the first three-dimensional environment cloud map, and the unmoved planned trajectory of the hook is replanned to control the hook to avoid the obstacle during the continued movement, and the replanned trajectory maintains the continuity of speed and acceleration at each point; The method of detecting whether there are obstacles around the hook according to the newly acquired sensor data during the movement of the hook includes: obtaining video data and / or radar data around the hook through the sensor when reaching each trajectory point; identifying the positions and contours of objects around the hook through point cloud data processing based on the video data and / or the radar data to obtain a second three-dimensional environmental cloud map around the hook; and determining whether each object in the second three-dimensional environmental cloud map is an obstacle; The replanning of the unmoved planned trajectory of the hook includes: obtaining a safety point of the obstacle, the safety point being the first trajectory point on the unmoved planned trajectory of the tower crane at a distance from the outline of the obstacle equal to the safety distance; using the safety point as a new lifting point, and obtaining a new planned trajectory for the remaining movement of the hook based on the new first three-dimensional environment cloud map and the hook drop point; Among them, the controlling hook to avoid the obstacle during the continued movement includes: controlling the hook to decelerate from the trajectory point where the obstacle is discovered to the safety point, and the speed at the safety point is set according to the actual trajectory; controlling the hook to move according to the new planned trajectory from the safety point to the hook drop point.

2. The method according to claim 1, characterized in that The step of obtaining a first three-dimensional environmental cloud map around the tower crane according to the sensor data includes: Obtain video data and / or radar data around the tower crane through sensors; The positions and contours of objects around the tower crane are identified by processing point cloud data based on the video data and / or the radar data to obtain a first three-dimensional environmental cloud map.

3. The method according to claim 1 or 2, characterized in that The point cloud data processing at least includes: Convert the data from each sensor into 3D point cloud data, and perform point cloud stitching, point cloud denoising and point cloud clustering; After point cloud clustering, the target bounding box of each class of point cloud data is obtained to obtain the outline and position of each object in the point cloud.

4. An automatic driving device for a tower crane, characterized in that: Includes: trajectory planning module, motion controller and trajectory controller; A trajectory planning module is configured to obtain a first three-dimensional environmental cloud map surrounding the tower crane, and to obtain a planned trajectory for the movement of the tower crane hook based on the first three-dimensional environmental cloud map, the lifting point, and the hook drop point of the tower crane, wherein the first three-dimensional environmental cloud map includes the distribution and outlines of objects surrounding the tower crane, wherein the obtained planned trajectory avoids each object in the first three-dimensional environmental cloud map and maintains a safe distance from the outlines of each object; the obtained planned trajectory includes the movement path of the hook and the speed of each point on each path, and the speed and acceleration at each point are continuous; A motion controller for controlling the hook to move point by point along the planned trajectory; a trajectory controller for detecting whether there are obstacles around the hook based on newly acquired sensor data during the movement of the hook, and when an obstacle is present around the hook, adding the obstacle to the first three-dimensional environmental cloud map and replanning the unmoved planned trajectory of the hook to control the hook to avoid the obstacle during continued movement, wherein the replanned trajectory maintains the continuity of velocity and acceleration of each point, the distance between the obstacle and the unmoved planned trajectory of the hook is less than a safety distance, and the obstacle does not belong to the first three-dimensional environmental cloud map; wherein the distance between an object and the unmoved planned trajectory of the hook is the minimum distance between each point on the outline of the object and each point on the unmoved planned trajectory of the hook; The trajectory controller is specifically configured to, when identifying obstacles, include: obtaining video data and / or radar data around the hook through a sensor when reaching each trajectory point; identifying the positions and contours of objects around the hook through point cloud data processing based on the video data and / or the radar data, to obtain a second three-dimensional environmental cloud map around the hook; and determining whether each object in the second three-dimensional environmental cloud map is an obstacle; The replanning of the unmoved planned trajectory of the hook includes: obtaining a safety point of the obstacle, the safety point being the first trajectory point on the unmoved planned trajectory of the tower crane at a distance from the outline of the obstacle equal to the safety distance; using the safety point as a new lifting point, and obtaining a new planned trajectory for the remaining movement of the hook based on the new first three-dimensional environment cloud map and the hook drop point; Among them, the controlling hook to avoid the obstacle during the continued movement includes: controlling the hook to decelerate from the trajectory point where the obstacle is discovered to the safety point, and the speed at the safety point is set according to the actual trajectory; controlling the hook to move according to the new planned trajectory from the safety point to the hook drop point.

5. A computing device, characterized in that include, bus; a communication interface connected to the bus; at least one processor connected to the bus; as well as At least one memory is connected to the bus and stores program instructions, and when the program instructions are executed by the at least one processor, the at least one processor executes the method according to any one of claims 1 to 3.

6. A computer-readable storage medium having program instructions stored thereon, characterized in that: When the program instructions are executed by a computer, the computer is caused to perform the method according to any one of claims 1 to 3.

Images