Path planning method and device of engineering machinery, computing equipment and storage medium

Abstract

The embodiment of the invention relates to the technical field of artificial intelligence, and relates to a path planning method and device for engineering machinery, computing equipment and a storage medium. According to the scheme, the method comprises the following steps: reconstructing a dynamic obstacle in a working area of the engineering machinery in the operation process of the engineering machinery, and extracting a bounding box of the dynamic obstacle; based on the bounding box of the dynamic obstacle, judging whether the planned path of the engineering machinery is intersected with the dynamic obstacle; wherein the planned path is obtained by performing path planning on the engineering machinery based on a static obstacle in advance; and under the condition that the planned path of the engineering machinery intersects with the dynamic obstacle, path planning is carried out on the engineering machinery again, so that the moving trajectory of the engineering machinery avoids the dynamic obstacle. According to the embodiment of the invention, the dynamic obstacles possibly appearing on the planned path are monitored in real time in the operation process of the engineering machinery, so that the engineering machinery can accurately avoid the obstacles, and the safety and reliability of the engineering machinery are guaranteed.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a path planning method, device, computing equipment and storage medium for engineering machinery. Background Art

[0002] Path planning refers to determining the trajectory path of an object's movement in engineering machinery so that the object can complete a specific task along the specified path. In actual engineering machinery design, path planning can avoid collisions between objects and other objects, and achieve smooth, efficient and safe movement. For example, traditional tower cranes are manually driven by crane drivers during operation. The blind spots caused by obstacles cause great difficulties for crane drivers' driving operations. For example, when hoisting objects across obstacles, since the crane driver cannot see the objects and hooks, he must rely on other personnel to direct on-site and assist the crane driver in completing the driving operation. It can be seen that how to carry out path planning for engineering machinery so that engineering machinery can accurately avoid obstacles during operation and ensure the safety and reliability of engineering machinery driving is a problem that needs to be solved urgently. Summary of the invention

[0003] In view of the above problems in the prior art, the embodiments of the present application provide a path planning method, device, computing device and storage medium for engineering machinery. In the operation process of the engineering machinery, the embodiments of the present application monitor the dynamic obstacles that may appear on the planned path in real time, so that the engineering machinery can accurately avoid the obstacles and ensure the safety and reliability of the engineering machinery.

[0004] To achieve the above objectives, the present application provides a first aspect of a path planning method for an engineering machine, comprising:

[0005] During the operation of the construction machinery, dynamic obstacles in the working area of ​​the construction machinery are reconstructed and the bounding box of the dynamic obstacles is extracted;

[0006] Based on the bounding box of the dynamic obstacle, determining whether the planned path of the engineering machinery intersects with the dynamic obstacle; wherein the planned path is obtained by pre-planning the path of the engineering machinery based on the static obstacle;

[0007] When the planned path of the engineering machine intersects with the dynamic obstacle, the path of the engineering machine is replanned so that the running track of the engineering machine avoids the dynamic obstacle.

[0008] As a possible implementation manner of the first aspect, the engineering machinery includes a tower crane; and the reconstructing a dynamic obstacle in a working area of ​​the engineering machinery and extracting a bounding box of the dynamic obstacle includes:

[0009] Use lidar to create a point cloud map of the working area of the tower crane;

[0010] Extract non-obstacle point cloud information from the point cloud map; wherein, the non-obstacle point cloud information includes the point cloud information of the hook, the suspended load, and the suspension rope of the tower crane, and the suspension rope is the suspension rope connecting the hook and the suspended load;

[0011] Remove the non-obstacle point cloud information from the point cloud map to obtain a non-obstacle point cloud map;

[0012] Extract the bounding box of the dynamic obstacle from the non-obstacle point cloud map.

[0013] As a possible implementation of the first aspect, the path planning is set as follows: in the direction of the lifting axis, rise from the current point to the lifting height value; keep the coordinates of the lifting axis unchanged, and run to directly above the end point in the directions of the slewing axis and the luffing axis; in the direction of the lifting axis, descend to reach the position of the end point.

[0014] As a possible implementation of the first aspect, the lidar includes a first lidar and a second lidar; the first lidar is arranged below the boom of the tower crane and at the position of the crane root; the second lidar is arranged on the trolley of the tower crane; the method further includes:

[0015] Extract the first bounding box of the dynamic obstacle from the point cloud map established by the first lidar; based on the first bounding box, re-plan the path in the directions of the slewing axis and the luffing axis of the tower crane; and / or,

[0016] Extract the second bounding box of the dynamic obstacle from the point cloud map established by the second lidar; based on the second bounding box, re-plan the path in the direction of the lifting axis of the tower crane.

[0017] As a possible implementation of the first aspect, the method further includes:

[0018] During the operation of the tower crane in the direction of the lifting axis, monitor whether the suspended load swings abnormally based on the point cloud map established by the second lidar;

[0019] In the case of monitoring that the suspended load swings abnormally, execute the corresponding preset strategy.

[0020] As a possible implementation of the first aspect, the judging whether the planned path of the construction machinery intersects with the dynamic obstacle based on the bounding box of the dynamic obstacle includes:

[0021] Expand the planned path of the construction machinery into a planned channel, and the planned channel is a channel with the planned path as the center line and a preset cross-sectional shape;

[0022] Determine whether the planned path intersects with the dynamic obstacle.

[0023] As a possible implementation of the first aspect, the construction machinery includes a tower crane; the method further includes:

[0024] When the lifted object of the tower crane is descending along the lifting axis and the distance between the lifted object and the end point of the planned path is a preset distance, switch the control mode of the tower crane to make the tower crane slowly run to the end point.

[0025] The second aspect of the present application provides a path planning device for construction machinery, including:

[0026] An extraction unit, configured to: during the operation of the construction machinery, reconstruct the dynamic obstacles in the working area of the construction machinery and extract the bounding box of the dynamic obstacles;

[0027] A judgment unit, configured to: based on the bounding box of the dynamic obstacles, judge whether the planned path of the construction machinery intersects with the dynamic obstacles; wherein, the planned path is obtained by pre-planning the path of the construction machinery based on static obstacles;

[0028] A planning unit, configured to: when the planned path of the construction machinery intersects with the dynamic obstacles, re-plan the path of the construction machinery so that the running trajectory of the construction machinery avoids the dynamic obstacles.

[0029] As a possible implementation of the second aspect, the construction machinery includes a tower crane; the extraction unit is configured to:

[0030] Use a lidar to establish a point cloud map of the working area of the tower crane;

[0031] Extract non-obstacle point cloud information from the point cloud map; wherein, the non-obstacle point cloud information includes the point cloud information of the hook, the lifted object and the lifting rope of the tower crane, and the lifting rope is the lifting rope connecting the hook and the lifted object;

[0032] Remove the non-obstacle point cloud information from the point cloud map to obtain a non-obstacle point cloud map;

[0033] Extract the bounding box of the dynamic obstacles from the non-obstacle point cloud map.

[0034] As a possible implementation of the second aspect, the path planning is set as: in the direction of the lifting axis, rise from the current point to the lifting height value; keep the coordinates of the lifting axis unchanged and run to directly above the end point in the directions of the slewing axis and the luffing axis; in the direction of the lifting axis, descend to reach the position of the end point.

[0035] As a possible implementation of the second aspect, the lidar includes a first lidar and a second lidar; the first lidar is arranged below the jib of the tower crane and is located at the root of the tower crane jib; the second lidar is arranged on the trolley of the tower crane;

[0036] The extraction unit is configured to: extract a first bounding box of the dynamic obstacle by using the point cloud map established by the first lidar; the determination unit and the planning unit are configured to: re-plan the path in the directions of the tower crane's slewing axis and luffing axis based on the first bounding box; and / or,

[0037] The extraction unit is configured to: extract a second bounding box of the dynamic obstacle by using the point cloud map established by the second lidar; the determination unit and the planning unit are configured to: re-plan the path in the direction of the tower crane's hoisting axis based on the second bounding box.

[0038] As a possible implementation of the second aspect, the device further includes a monitoring unit, and the monitoring unit is configured to:

[0039] During the operation of the tower crane in the hoisting axis direction, monitor whether the suspended load swings abnormally based on the point cloud map established by the second lidar;

[0040] In the case of monitoring that the suspended load swings abnormally, execute a corresponding preset strategy.

[0041] As a possible implementation of the second aspect, the determination unit is configured to:

[0042] Expand the planned path of the construction machinery into a planned channel, where the planned channel is a channel with the planned path as the center line and a preset cross-sectional shape;

[0043] Determine whether the planned channel intersects with the dynamic obstacle.

[0044] As a possible implementation of the second aspect, the construction machinery includes a tower crane; the device further includes a control unit, and the control unit is configured to:

[0045] When the suspended load of the tower crane is descending in the hoisting axis direction and the distance between the suspended load and the end point of the planned path is a preset distance, switch the control mode of the tower crane to make the tower crane slowly run to the end point.

[0046] A third aspect of the present application provides a computing device, including:

[0047] A communication interface;

[0048] At least one processor, which is connected to the communication interface; and

[0049] At least one memory, which is connected to the processor and stores program instructions, and when the program instructions are executed by the at least one processor, the at least one processor is caused to execute the method according to any one of the above first aspects.

[0050] A fourth aspect of the present application provides a computer-readable storage medium, on which program instructions are stored, and when the program instructions are executed by a computer, the computer is caused to execute the method according to any one of the above first aspects.

[0051] These and other aspects of the present invention will become more readily apparent in the following description of the (one or more) embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] The following further describes each feature of the present invention and the relationship between each feature with reference to the drawings. The drawings are all exemplary. Some features are not shown in actual proportion, and in some drawings, conventional features in the field related to the present application that are not necessary for the present application may be omitted, or features that are not necessary for the present application may be additionally shown. The combination of the features shown in the drawings is not used to limit the present application. In addition, throughout the present specification, the content referred to by the same reference numerals is also the same. The specific description of the drawings is as follows:

[0053] Figure 1A It is a schematic structural diagram of the application scenario of each embodiment of the present application;

[0054] Figure 1B It is a schematic structural diagram of the tower crane applied in each embodiment of the present application;

[0055] Figure 2 It is a schematic diagram of an embodiment of the path planning method for construction machinery provided by an embodiment of the present application;

[0056] Figure 3 It is a schematic diagram of an embodiment of the path planning method for construction machinery provided by an embodiment of the present application;

[0057] Figure 4 It is a schematic diagram of an embodiment of the path planning device for construction machinery provided by an embodiment of the present application;

[0058] Figure 5 It is a schematic diagram of an embodiment of the path planning device for construction machinery provided by an embodiment of the present application;

[0059] Figure 6 It is a schematic diagram of the computing device provided by an embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0060] The terms "first", "second", "third", etc. or similar terms such as Module A, Module B, Module C, etc. in the specification and claims are only used to distinguish similar objects and do not represent a specific order for the objects. Understandably, the specific order or sequence can be interchanged where permitted so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0061] In the following description, the reference numerals representing steps, such as S110, S120, etc., do not necessarily mean that the steps will be executed in this order. The order of the steps can be interchanged where permitted, or the steps can be executed simultaneously.

[0062] The term "comprising" used in the specification and claims should not be construed as being limited to the content listed thereafter; it does not exclude other elements or steps. Therefore, it should be interpreted as specifying the presence of the stated features, wholes, steps or components, but does not exclude the presence or addition of one or more other features, wholes, steps or components and their groups. Thus, the expression "a device comprising device A and B" should not be limited to a device consisting only of components A and B.

[0063] The "one embodiment" or "embodiment" mentioned in this specification means that the specific features, structures or characteristics described in connection with the embodiment are included in at least one embodiment of the present invention. Therefore, the phrases "in one embodiment" or "in an embodiment" that appear throughout this specification do not necessarily all refer to the same embodiment, but may refer to the same embodiment. In addition, in one or more embodiments, the various specific features, structures or characteristics can be combined in any suitable manner, as will be apparent to those of ordinary skill in the art from this disclosure.

[0064] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the technical field to which this application belongs. In case of inconsistency, the meaning set forth in this specification or the meaning derived from the content recorded in this specification shall prevail. Additionally, the terms used herein are for the purpose of describing the embodiments of this application only and are not intended to limit this application. For the purpose of accurately describing the technical content in this application and for accurately understanding the present invention, the following explanations or definitions of the terms used in this specification are given before describing the specific embodiments:

[0065] 1) SLAM (Simultaneous Localization and Mapping): Also known as CML (Concurrent Mapping and Localization). The SLAM problem can be described as follows: A robot starts moving from an unknown location in an unknown environment. During the movement, it locates itself based on its position and the map, and at the same time, builds an incremental map based on its self-localization to achieve the robot's autonomous localization and navigation.

[0066] 2) RTK (Real Time Kinematic): A real-time differential GPS (Global Positioning System) technology based on carrier phase observations. It is a breakthrough in the development of surveying technology and consists of three parts: a reference station receiver, a data link, and a rover receiver. One receiver is installed at the reference station as the reference station, continuously observing the satellites, and transmitting its observation data and station information to the rover in real time through a radio transmission device. While receiving GPS satellite signals, the rover GPS receiver receives the data transmitted from the reference station through a wireless receiving device, and then, based on the principle of relative positioning, calculates the three-dimensional coordinates of the rover and its accuracy in real time.

[0067] 3) IMU (Inertial Measurement Unit): A sensor mainly used to detect and measure acceleration and rotational motion. Its principle is based on the law of inertia. These sensors range from ultra-small MEMS (Micro-Electro-Mechanical Systems) sensors to laser gyroscopes with very high measurement accuracy. Whether it is an MEMS sensor with a size of only a few millimeters or an optical fiber device with a diameter of nearly half a meter, the same principle is adopted.

[0068] 4) Variable-frequency Drive (VFD): It is a power control device that uses frequency conversion technology and microelectronics technology to control AC motors by changing the frequency of the motor's working power supply. The inverter is mainly composed of rectification (AC to DC), filtering, inversion (DC to AC), braking unit, drive unit, detection unit, microprocessor unit, etc. The inverter adjusts the voltage and frequency of the output power supply by the opening and closing of the internal IGBT (Insulated Gate Bipolar Transistor), and provides the required power supply voltage according to the actual needs of the motor, thereby achieving the purpose of energy saving and speed regulation. In addition, the inverter has many protection functions, such as over-current, over-voltage, overload protection, etc.

[0069] 5) TCP (Transmission Control Protocol): is a connection-oriented, reliable, byte-stream-based transport layer communication protocol. TCP is designed to adapt to a layered protocol hierarchy that supports multiple network applications. Pairs of processes in host computers connected to different but interconnected computer communication networks rely on TCP to provide reliable communication services. TCP assumes that it can obtain simple, possibly unreliable datagram services from lower-level protocols. In principle, TCP should be able to operate on a variety of communication systems from hard-wired connections to packet switching or circuit switching networks.

[0070] The following first introduces the existing method, and then introduces the technical solution of the present application in detail.

[0071] When traditional construction machinery is planning its path, the blind spots caused by obstacles will cause great difficulties in operation. Take tower cranes as an example. Traditional tower cranes are manually driven by crane drivers during operation. The blind spots caused by obstacles cause great difficulties for crane drivers in driving operations. For example, when hoisting objects across obstacles, the crane driver cannot see the objects and hooks, so he must rely on other personnel to direct on-site and assist the crane driver in completing the driving operation. It can be seen that how to ensure the safety and reliability of construction machinery driving is a problem that needs to be solved urgently.

[0072] The existing technology has the following defects: the construction machinery cannot accurately avoid obstacles during operation, and the safety and reliability of the construction machinery driving cannot be guaranteed.

[0073] Based on the technical problems existing in the above-mentioned prior art, the embodiments of the present application provide a path planning method, device, equipment and medium for construction machinery. During the operation of the construction machinery, the embodiments of the present application monitor in real time the dynamic obstacles that may appear on the planned path, enabling the construction machinery to accurately avoid obstacles and ensuring the safety and reliability of the construction machinery, thereby solving the technical problems in the prior art that the construction machinery cannot accurately avoid obstacles during operation and cannot ensure the safety and reliability of the operation of the construction machinery.

[0074] The present application provides various method embodiments, device embodiments, computing device embodiments and storage medium embodiments for path planning of construction machinery. The construction machinery in the embodiments of the present application may be heavy construction machinery and equipment such as tower cranes and tower hoists. Taking a tower crane as an example, the application scenarios of the embodiments of the present application will be described below in conjunction with Figure 1A and Figure 1B introduce the application scenarios of the embodiments of the present application.

[0075] Figure 1A The control system of the tower crane of the present application is shown, which includes a video device, a controller, an inverter, a motor and an encoder.

[0076] Among them, the video device is used to obtain in advance the reference points during the operation of the tower crane hook. The reference points include the starting position, target position of the tower crane hook and the passable position in front of the obstacle to be avoided. The coordinates of the reference points obtained by the video device are the coordinates of the user coordinate system. The video device communicates with the controller through the Modubus TCP protocol.

[0077] Among them, the motor is used to drive the movement of the tower crane hook. The tower crane includes a number of axes, that is, a number of adjustment directions. Each axis is driven by a motor.

[0078] Among them, the inverter is used to receive the planned speed output by the controller and control the rotation speed of the motor accordingly. One axis direction corresponds to the use of one inverter for control.

[0079] Among them, the encoder is used to collect the actual position of the tower crane hook during the movement to help the tower crane hook avoid obstacles according to the planned trajectory and reach the target position. The encoder communicates with the controller through the Modubus 485 protocol.

[0080] Among them, the controller is used to generate the planned trajectory of the tower crane hook according to the reference points, determine the planned speed of each trajectory point, and drive the inverter of the tower crane to control the hook to bypass the obstacle according to the planned trajectory and accurately reach the target position.

[0081] Among them, the method embodiments of the present application run in the controller, and the device embodiments are deployed in the controller.

[0082] Figure 1B The figure shows a tower crane to which the embodiments of the present application are applied. The tower crane includes three axes: Lift, Turn, and Swing. Among them, Lift is also called hoisting, Turn is also called slewing, and Swing is also called luffing.

[0083] Among them, Figure 1A the video device obtains the coordinates of the tower crane hook in the user coordinate system, which can be converted into Figure 1B the coordinates in the Cartesian coordinate system through the transformation matrix.

[0084] Figure 1B The structure of... is an example of a tower crane. In an actual scenario, a tower crane may include other numbers of axes, such as axes that move horizontally and / or longitudinally along a guide rail.

[0085] Taking the three axes of Lift, Turn, and Swing as an example, during the operation of the tower crane hook, the three axes respectively drive the AC motors to perform movements through three frequency converters, and the current positions of each axis are collected through absolute value encoders. The vision device transmits the main path point positions to the controller through ModbusTCP according to the monitored obstacle positions. The controller sends motion instructions to the three frequency converters through the Modbus485 bus, and at the same time collects the position information of the three encoders.

[0086] Figure 2 This is a schematic diagram of an embodiment of the path planning method for construction machinery provided by the embodiments of the present application. As Figure 2 shown, the method may specifically include:

[0087] Step S110, during the operation of the construction machinery, reconstruct the dynamic obstacles in the working area of the construction machinery, and extract the bounding boxes of the dynamic obstacles;

[0088] Step S120, based on the bounding boxes of the dynamic obstacles, determine whether the planned path of the construction machinery intersects with the dynamic obstacles; wherein, the planned path is obtained by pre-planning the path of the construction machinery based on static obstacles;

[0089] Step S130, in the case where the planned path of the construction machinery intersects with the dynamic obstacles, re-plan the path of the construction machinery so that the running trajectory of the construction machinery avoids the dynamic obstacles.

[0090] Taking a tower crane as an example, during the unmanned operation of the tower crane, it is necessary to accurately avoid obstacles on the running trajectory. All obstacles in the working area of the tower crane may be obstacles that need to be avoided. These obstacles include static obstacles and dynamic obstacles.

[0091] Among them, a dynamic obstacle refers to an obstacle that is detected in real time during the operation of a tower crane, including objects that may move their positions at any time, such as obstacle vehicles, animals, pedestrians, etc.; in addition, dynamic obstacles also include all obstacles within the working area of the tower crane detected in real time. The information of dynamic obstacles will change at any time as the object moves.

[0092] Among them, a static obstacle refers to an obstacle that is detected when the tower crane is about to start running and formulating a path plan. A static obstacle can be an object that remains stationary, such as a building, railing, tree, etc.; in addition, static obstacles also include objects that are temporarily stationary or in motion when formulating a path plan, such as obstacle vehicles, animals or pedestrians, etc. For an object that is in motion when formulating a path plan, it will be detected as a static obstacle when formulating a path plan and will become a dynamic obstacle due to position changes during the operation of the tower crane.

[0093] When formulating a path plan for a tower crane, not only should static obstacles be considered to be avoided, but also dynamic obstacles should be considered to be avoided. Thus, a lidar installed on the tower crane can be used to identify static and dynamic obstacles, enabling the tower crane to accurately avoid obstacles.

[0094] In step S110, during the operation of the construction machinery, a point cloud map of the working area of the construction machinery is established using a lidar, the dynamic obstacles in the working area of the construction machinery are reconstructed based on the point cloud map, and the bounding box of the dynamic obstacles is extracted. Specifically, the point cloud map can be denoised, and the point cloud map can be segmented using the bounding box. The area where the bounding box is located is considered to be an obstacle. Among them, the bounding box refers to a cube that completely encloses an object. When calculating the bounding box, it is first necessary to perform image segmentation on the target object to separate it from the background. Then, by analyzing the contour of the target object, the minimum circumscribed cube of the object is calculated, thereby obtaining the bounding box.

[0095] When the construction machinery is about to start running, a path plan for the construction machinery can be pre-formulated based on static obstacles. For example, the static obstacles in the working area of the construction machinery can be reconstructed based on the point cloud map, and the bounding box of the static obstacles can be extracted; the lifting height value is determined based on the bounding box of the static obstacles, so that the lifting height value is higher than the height value of the highest point of all the bounding boxes of the static obstacles. Then, in the planned path, the lifted object is lifted to the lifting height value, and then the lifted object is controlled to move to directly above the end point in the slewing and luffing directions, and finally lowered to the end point position, which can enable the construction machinery to accurately avoid static obstacles.

[0096] The above planned path is generated based on the point cloud map when the construction machinery is about to start running. In the actual scenario, the positions of dynamic obstacles are dynamically changing, and the point cloud map is also changing in real time. During the actual operation of the construction machinery, it is possible that a dynamic obstacle just moves onto the planned path, posing a safety hazard to the operation of the construction machinery. Therefore, in step S120, based on the bounding boxes of the dynamic obstacles extracted in real time in step S110, it is determined whether the planned path of the construction machinery intersects with the dynamic obstacles. In step S130, in the case where the planned path of the construction machinery intersects with the dynamic obstacles, the path planning of the construction machinery is re-performed so that the actual running trajectory of the construction machinery avoids static obstacles and dynamic obstacles.

[0097] In the embodiment of the present application, during the operation of the construction machinery, the dynamic obstacles that may appear on the planned path are monitored in real time, so that the construction machinery can accurately avoid the obstacles, ensuring the safety and reliability of the unmanned construction machinery.

[0098] In one implementation manner, the construction machinery includes a tower crane; the method further includes:

[0099] A combined sensor is pre-installed below the boom of the tower crane and at the position of the root of the tower crane boom; wherein, the combined sensor includes an inertial measurement unit, a first lidar, and a camera.

[0100] Calibrate the external parameters of the combined sensor in the world coordinate system.

[0101] Use the first lidar for simultaneous localization and mapping to establish a point cloud map of the working area of the tower crane.

[0102] Perform image segmentation on the obstacles in the point cloud map to obtain the bounding boxes of the static obstacles.

[0103] Before performing the path planning of the unmanned tower crane, it is necessary to pre-install the hardware equipment for SLAM mapping and perform the necessary parameter calibration and preprocessing steps. In one example, the specific equipment installation and preprocessing steps are as follows:

[0104] Step 1. Establish the tower crane coordinate system: Take the center point of the tower base on the ground as the coordinate origin, the due north direction as the positive x-axis direction, the due west direction as the positive y-axis direction, and the vertically upward direction as the positive z-axis direction. The RTK coordinates of the coordinate origin are (WE0, SN0, H0).

[0105] Step 2. Install the combined sensor: The sensors in the combined sensor include an IMU, a first lidar, and a camera. First, fix the IMU, the first lidar, and the camera on the mounting frame, then calibrate the internal parameters and distortion parameters of the camera, secondly calibrate the external parameters of the IMU and the lidar, and finally calibrate the external parameters of the lidar and the camera.

[0106] Among them, the mounting frame is installed under the tower crane's jib and at the position of the tower crane's jib root, that is, at the intersection of the tower base and the jib. The mounting frame and the combined sensor can move together with the jib in the slewing direction.

[0107] Among them, the vertical field of view angle of the first lidar is greater than 90 degrees.

[0108] Among them, the internal parameters of the camera are parameters related to the characteristics of the camera itself, such as the focal length and pixel size of the camera.

[0109] Among them, each sensor in the combined sensor has its own coordinate system. The external parameters refer to the relative relationship of the conversion between coordinate systems. Taking one coordinate system as the original coordinate system, the conversion matrix of another coordinate system relative to it is called the external parameter. In the combined sensor, the positions of the first lidar, the camera, and the IMU are fixed, and there are external parameters between them. In step 2, the external parameters between the IMU, the first lidar, and the camera are calibrated.

[0110] Step 3. Calibrate the external parameters between the first lidar and the tower crane coordinate system:

[0111] In the combined sensor, there are mutual external parameters between the sensors, and each sensor also has external parameters relative to the tower crane. For example, the external parameters may include: the external parameters of the camera relative to the tower crane coordinate system, the external parameters of the camera relative to the first lidar, the external parameters of the first lidar relative to the camera, and the external parameters of the first lidar relative to the tower crane coordinate system.

[0112] Before the jib rotates, calibrate the external parameters of the combined sensor in the world coordinate system. The external parameters of the combined sensor in the world coordinate system include all the external parameter relationships between the sensors and between the sensors and the world coordinate system. In the combined sensor, taking the first lidar as the main body, there are external parameters between the camera and the first lidar, between the IMU and the first lidar, and between the first lidar and the tower crane coordinate system. Among them, the world coordinate system is the tower crane coordinate system.

[0113] Specifically, in the embodiments of the present application, it is necessary to use the first lidar for SLAM mapping. Before the jib rotates, it is necessary to calibrate the starting position of the SLAM mapping, that is, to calibrate the external parameters between the first lidar and the tower crane coordinate system.

[0114] Step 4. Laser SLAM mapping: When the tower crane's jib rotates one week, use the first lidar SLAM to reconstruct the three-dimensional point cloud map of the tower crane's working area. During the SLAM mapping process, the first lidar can be used for real-time positioning and map construction, and the IMU can also be used for auxiliary positioning.

[0115] Step 5. Point cloud map processing: Using the external parameter matrix calibrated in Step 2, the point cloud map can be transformed into the world coordinate system. Noise reduction processing is performed on the point cloud map, and the point cloud map is segmented using a bounding box. The area where the bounding box is located is considered a static obstacle. Among them, the bounding box refers to a cube that completely encloses an object. When calculating the bounding box, it is first necessary to perform image segmentation on the target object to separate it from the background. Then, by analyzing the contour of the target object, the minimum circumscribed cube of the object is calculated to obtain the bounding box.

[0116] Step 6. Obtain the starting point and ending point of the tower crane hook in the desired operating trajectory:

[0117] The starting point is determined to be (Xs, Ys, Hs) in the tower crane coordinate system through conversion of the data obtained from the encoder on the tower crane, and is converted into cylindrical coordinates as (Rs, Thetas, Zs). Among them, the conversion coefficient for converting the encoder data needs to be calibrated in advance, and then the data conversion is performed.

[0118] The ending point is given by the RTK point coordinates in the following manner: The longitude and latitude parts of the RTK coordinates (WE1, SN1, H1) are converted into (Xe, Ye) in the tower crane coordinate system; since the RTK coordinates of the coordinate origin are (WE0, SN0, H0), H1 is converted into the z-direction coordinate as (H1 - H0). In summary, the tower crane coordinate system coordinates of the ending point are (Xe, Ye, H1 - H0), and are converted into cylindrical coordinates as (Re, Thetae, Ze).

[0119] Step 7. Install a second lidar on the tower crane trolley, and use the second lidar to scan downward to measure the dimensions of the suspended object, hook, and connecting piece at the moment before path planning. Among them, the connecting piece is the connecting component between the suspended object and the hook. Since the combined sensor is installed at the root of the tower crane arm, the first lidar in the combined sensor may not be able to see the hook and the suspended object due to occlusion. Therefore, it is necessary to install a second lidar on the tower crane trolley to measure the component dimensions using the second lidar.

[0120] In one implementation, the method further includes:

[0121] Pre-install a second lidar on the tower crane trolley;

[0122] Use the second lidar to measure the distance from the bottom to the top of the object hanging on the tower crane hook.

[0123] Pre-measuring the distance from the bottom to the top of the object hanging on the tower crane hook can consider the influence of this distance on path planning during path planning. Based on this distance, the tower crane path is planned, which can avoid collisions between the hook and the suspended object and obstacles, thus ensuring the safety and reliability of tower crane unmanned driving.

[0124] Specifically, the following assumptions can be made: The overall combination of the lifted object, the hook, and the connecting member is considered to be in the shape of a cuboid. Obtain the dimensions (l, d, Hw) of the overall cuboid. Here, l, d, and Hw are the length, width, and height of the cuboid respectively; Hw is the distance from the bottom of the object hanging on the hook to the top of the hook.

[0125] Taking a tower crane as an example, after the equipment installation and preprocessing steps, when the tower crane is about to operate, the path of the unmanned tower crane can be planned based on static obstacles, so as to achieve static obstacle avoidance first. The specific steps for realizing the path planning of static obstacle avoidance are as Figure 3 shown, and the method may specifically include:

[0126] Step S210, obtain the coverage area of the projection of the expected operating trajectory of the tower crane on the ground; wherein, the range of the polar radius coordinates of all the trajectory points of the expected operating trajectory is used as the polar radius range of the coverage area, and the range of the polar angle coordinates of all the trajectory points of the expected operating trajectory is used as the polar angle range of the coverage area;

[0127] Step S220, obtain the maximum value of the polar radius of all the trajectory points in the coverage area; determine the corresponding sector area currently based on the maximum value of the polar radius; wherein, taking the value less than or equal to the maximum value of the polar radius as the polar radius range of the sector area, and taking the polar angle range of the coverage area as the polar angle range of the sector area;

[0128] Step S230, determine the height value of the highest point in the path planning of the tower crane based on the minimum value of the pitch angles of the bounding boxes of all the static obstacles in the currently corresponding sector area; wherein, the pitch angle of the bounding box is the minimum value of the pitch angles of all the points in the bounding box;

[0129] Step S240, perform path planning for the tower crane based on the height value of the highest point.

[0130] During the unmanned operation of the tower crane, it is necessary to accurately avoid obstacles in the planned trajectory. If the lifting height in the planned trajectory is insufficient, for example, the lifting height is lower than the height of the obstacle, the tower crane may hit the obstacle during operation. To accurately avoid obstacles, it is first necessary to identify the obstacles that the tower crane may encounter during operation within the range where the expected operating trajectory of the tower crane is located. Specifically, a lidar installed at the root of the tower crane boom, that is, below the intersection of the tower base and the jib, can be used to build a map and perform image segmentation on the obstacles to obtain the bounding boxes of the static obstacles.

[0131] In step S210, first, obtain the coverage area of the projected ground of the expected operating trajectory of the tower crane. In subsequent steps, it is necessary to identify the static obstacles within this coverage area. In the embodiments of the present application, a cylindrical coordinate system is used to locate the position of the coverage area. The cylindrical coordinate system refers to a coordinate system that uses planar polar coordinates and the distance in the z - direction to define the spatial coordinates of an object. Taking the center point of the tower base on the ground as the origin of the planar polar coordinates, and the vertically upward direction as the positive direction of the z - axis. Project the expected operating trajectory of the tower crane onto the ground. The range of the polar radius coordinates of all the trajectory points of the expected operating trajectory is used as the polar radius range of the coverage area; the range of the polar angle coordinates of all the trajectory points of the expected operating trajectory is used as the polar angle range of the coverage area. For example, the minimum value of the polar radius coordinates of all the trajectory points of the expected operating trajectory is 10m, and the maximum value is 20m; the minimum value of the polar angle coordinates of all the trajectory points of the expected operating trajectory is 20°, and the maximum value is 50°; then the coverage area is a part of an annular region, and the polar radius range of this region is from 10m to 20m, and the polar angle range is from 20° to 50°.

[0132] The recognition range of the lidar will have a blind area due to the occlusion of nearby obstacles. Therefore, it is necessary not only to identify the obstacles within the coverage area, but also to identify the obstacles in the area outside the coverage area and closer to the tower body. In step S220, obtain the maximum value of the polar radius of all the trajectory points in the coverage area obtained in step S210. Determine the corresponding fan - shaped area based on the maximum value of the polar radius. Among them, the range of the polar radius less than or equal to the maximum value of the polar radius is used as the polar radius range of the fan - shaped area, and the polar angle range of the coverage area is used as the polar angle range of the fan - shaped area. In the above example, the maximum value of the polar radius of all the trajectory points in the coverage area is 20m, then the currently corresponding fan - shaped area is an area with a radius of 20m and a polar angle range of 20° to 50°. This fan - shaped area includes the coverage area and the area outside the coverage area and closer to the tower body. In subsequent steps, it is necessary to identify the obstacles within this fan - shaped area.

[0133] In step S230, first, identify the static obstacles within the fan - shaped area determined in step S220. Then, solve the pitch angle of the bounding box of all the static obstacles in this fan - shaped area, and take the minimum value from all the pitch angles to obtain the minimum pitch angle. Among them, the pitch angle of a certain point in space is the angle between the line connecting the lidar and this point and the line where the jib is located. Under the condition of the same polar radius, the smaller the pitch angle, the higher the height of this point. Therefore, based on the minimum value of the pitch angles of the bounding boxes of all the obstacles in the local area, the height value of the highest point in the tower - crane path planning can be determined. Operating above this height can accurately avoid obstacles. In step S230, for the currently corresponding fan - shaped area, solve the minimum value of the pitch angles of the bounding boxes of all the obstacles in this fan - shaped area, and determine the height value of the highest point in the tower - crane path planning based on this minimum value.

[0134] In step S240, the path planning of the tower crane is performed based on the height value of the highest point determined in step S230. For example, in the planned path, it can first rise above the highest point in the lifting direction. Since this height is higher than the height of the obstacle, moving in the slewing direction and the luffing direction at this height can accurately avoid the obstacle.

[0135] In summary, in the embodiment of the present application, by defining the solution method of the highest point, the height value of the highest point in the path planning is determined based on the pitch angle of the bounding box of the obstacle, and the path planning of the tower crane is performed based on the height value of the highest point. Among them, by searching for the bounding boxes of all obstacles in the coverage area of the ground projection of the expected running trajectory of the tower crane and selecting the bounding box with the smallest pitch angle from all the bounding boxes, and determining the height value of the highest point in the path planning on this basis, the tower crane can accurately avoid obstacles during operation, thus ensuring the safety and reliability of the driverless tower crane.

[0136] After the path of the driverless tower crane is planned based on static obstacles, during the operation of the tower crane, the path of the driverless tower crane can be planned in real time based on dynamic obstacles, so as to further achieve dynamic obstacle avoidance. The method shown in Figure 2 can be executed to implement the path planning of dynamic obstacle avoidance. In one example, the specific steps to achieve dynamic obstacle avoidance are as follows:

[0137] Step a. Install a second lidar on the trolley. The emitting end of the second lidar is vertically downward, and the field of view angle is not less than 40 degrees both horizontally and vertically.

[0138] Step b. During the ascending and descending processes of the second lidar on the trolley, point clouds are obtained in real time, and the point cloud information of the hook, the sling, and the lifted object is extracted, and the point cloud information of the hook, the sling, and the lifted object is removed. Among the remaining point cloud information after removal, the bounding box extracted is considered as the bounding box of the obstacle.

[0139] In one implementation manner, the reconstruction of the dynamic obstacles in the working area of the tower crane and the extraction of the bounding boxes of the dynamic obstacles include:

[0140] Establish a point cloud map of the working area of the tower crane using a lidar;

[0141] Extract non-obstacle point cloud information from the point cloud map; wherein, the non-obstacle point cloud information includes the point cloud information of the hook, the lifted object, and the sling of the tower crane, and the sling is the sling connecting the hook and the lifted object;

[0142] Remove the non-obstacle point cloud information from the point cloud map to obtain a non-obstacle point cloud map;

[0143] Extract the bounding box of the dynamic obstacle from the non-obstacle point cloud map.

[0144] Among them, the installation position of the lidar on the trolley is fixed. The approximate positions of the hook, the lifted object, and the lifting rope can be obtained through the encoder. Then, using the method of point cloud segmentation, the areas where the hook, the lifted object, and the lifting rope are located are directly cut out and removed from the point cloud map, that is, the non-obstacle point cloud information is removed from the point cloud map.

[0145] Step c. Determine whether the trajectory line of the planned path will intersect with the dynamic obstacle. Among them, the trajectory line of the planned path needs to be expanded into a corresponding shape according to the situation, such as a cylinder, a cuboid, etc.

[0146] In one implementation, determining whether the planned path of the construction machinery intersects with the dynamic obstacle based on the bounding box of the dynamic obstacle includes:

[0147] Expand the planned path of the construction machinery into a planned channel, where the planned channel is a channel with the planned path as the center line and a preset cross-sectional shape;

[0148] Determine whether the planned channel intersects with the dynamic obstacle.

[0149] The trajectory line of the planned path is just a line, and the trajectory line is only the movement trajectory of a certain point on the hook. However, the hook, the lifted object, and the lifting rope as a whole are actually an object with volume. Therefore, this whole needs to be expanded into a geometric body, and then it is determined whether this whole intersects with the obstacle. In an example, the size of the cross-section of the expanded planned channel can be set, and this size can be used as the expansion parameter. With the planned path as the center line, each point on the center line is expanded into a cross-section with a preset size. The center line and all the cross-sections form the planned channel. The planned path can be expanded into a planned channel with a preset shape according to the size, shape, assembly method, etc. of the lifted object. For example, for a flat and long lifted object such as a wooden board, it can be expanded into a cuboid; for a triangular lifting rope, it can be expanded into a cuboid; for a lifted object that is lifted from the middle, may swing left and right, and has no triangular fixation, it can be expanded into a cylinder; for a rotating lifted object, it can be expanded into a cylinder.

[0150] In one implementation, the path planning is set as follows: 1) In the direction of the lifting axis, rise from the current point to the lifting height value; this section of the trajectory is called the first section of the trajectory; 2) Keep the coordinates of the lifting axis unchanged and run to directly above the end point in the directions of the rotation axis and the luffing axis; this section of the trajectory is called the second section of the trajectory; 3) In the direction of the lifting axis, descend to reach the position of the end point; this section of the trajectory is called the third section of the trajectory.

[0151] Step d. When the planned path of the lifting upward intersects with a dynamic obstacle in the first section of the trajectory, the path of the lifting upward needs to be changed to avoid the obstacle. Similarly, when the planned path of the lifting downward intersects with a dynamic obstacle in the third section of the trajectory, the path of the lifting downward needs to be changed to avoid the obstacle. The changed path can be a path that is as close as possible to the original planned point. That is to say, the suspended load is bypassed beside the obstacle, not too far from the original planned point, and the stable operation of the tower crane is controlled as much as possible. A safety distance in the horizontal direction can be preset, and when changing the path, the distance between the obstacle and the suspended load is made greater than the safety distance to further ensure safety.

[0152] In one embodiment, the lidar includes a second lidar; the second lidar is arranged on the trolley of the tower crane; the method further includes:

[0153] Using the point cloud map established by the second lidar, the second bounding box of the dynamic obstacle is extracted; based on the second bounding box, the path in the direction of the lifting axis of the tower crane is re-planned.

[0154] Among them, during the lifting movement, the point cloud obtained by the second lidar installed on the trolley is used for dynamic obstacle avoidance. When re-planning the path based on the dynamic obstacle, in the direction of the lifting axis, the second bounding box of the dynamic obstacle is extracted based on the point cloud map established by the second lidar; based on the second bounding box, it is judged whether the planned path of the tower crane intersects with the dynamic obstacle; in the case where the planned path of the tower crane intersects with the dynamic obstacle, the path in the direction of the lifting axis of the tower crane is re-planned.

[0155] Step e. When the planned path during the slewing and luffing processes intersects with a dynamic obstacle in the second section of the trajectory, the paths of the slewing and luffing need to be changed accordingly to avoid the obstacle. Similarly, the changed path can also be a path that is as close as possible to the original planned point. A safety distance in the vertical direction can be preset, and when changing the path, the distance between the obstacle and the suspended load is made greater than the safety distance to further ensure safety.

[0156] In one embodiment, the lidar includes a first lidar; the first lidar is arranged below the boom of the tower crane and at the position of the root of the tower crane boom; the method further includes:

[0157] Using the point cloud map established by the first lidar, the first bounding box of the dynamic obstacle is extracted; based on the first bounding box, the paths in the directions of the slewing axis and the luffing axis of the tower crane are re-planned.

[0158] Since the lidar at the root of the boom is too far away and there is a situation where it is blocked and cannot be seen, the point cloud obtained by the lidar on the trolley is used for dynamic obstacle avoidance during the upward and downward processes, and the point cloud obtained by the lidar at the root of the boom is used for dynamic obstacle avoidance during the slewing and luffing processes.

[0159] When re-planning the path based on dynamic obstacles, in the direction of the slewing axis and the luffing axis, based on the point cloud map established by the first lidar, the first bounding box of the dynamic obstacle is extracted. Similar to the obstacle avoidance process of the lifting axis, the process of extracting the first bounding box also includes removing the point cloud information of the hook, the suspended load, and the lifting rope from the point cloud map. After removal, the first bounding box of the dynamic obstacle is extracted from the remaining point cloud map. Then, based on the first bounding box, it is determined whether the planned path of the tower crane intersects with the dynamic obstacle. Among them, the trajectory line of the planned path also needs to be expanded into a corresponding shape according to the situation. When the planned path of the tower crane intersects with the dynamic obstacle, the path in the direction of the slewing axis and the luffing axis of the tower crane is re-planned.

[0160] In one implementation, the method further includes:

[0161] During the operation of the tower crane in the lifting axis direction, based on the point cloud map established by the second lidar, it is monitored whether the suspended load swings abnormally;

[0162] When it is monitored that the suspended load swings abnormally, the corresponding preset strategy is executed.

[0163] After path planning, the controller drives the frequency converter of the tower crane to control the motor so that the hook runs according to the planned path. In the first and third trajectories of the operation process, the lidar on the trolley acquires the point cloud, identifies the hook and the lifting rope, monitors whether the motion control runs according to the plan, and monitors the swinging conditions of the lifting rope and the suspended load. According to the operation conditions and the swinging conditions, abnormal conditions such as control abnormality or severe swaying are judged, and corresponding strategies are made.

[0164] Since the lidar at the base of the boom is too far away and there may be occlusion and invisibility, the lidar on the trolley is used to measure the swinging amplitude of the suspended load during the ascending and descending processes, and the lidar at the base of the boom is used to measure the swinging amplitude of the suspended load during the slewing and luffing processes. By adopting this method, all processes of the tower crane operation can be monitored to implement comprehensive motion control and control monitoring.

[0165] When it is monitored that the suspended load swings abnormally, the preset strategies that can be executed may include deceleration, stopping the motion, or moving in the same direction. For example, sometimes due to strong wind or the heavy object not being properly hung, the suspended load may keep swinging. In this case, the motion can be decelerated or stopped, and after the heavy object is re-hung or when the wind becomes smaller, the motion can be restarted. Another example is that when it is monitored that the swinging amplitude of the suspended load is too large, the tower crane can be controlled to move in the same direction as the swinging direction of the suspended load, so that the tower crane moves with the suspended load. Executing this strategy can reduce the swing of the suspended load, convert the swing into translational motion, and enable the tower crane to operate smoothly.

[0166] In one implementation, the construction machinery includes a tower crane; the method further includes:

[0167] When the lifted object of the tower crane is descending along the lifting axis and the distance between the lifted object and the end point of the planned path is a preset distance, switch the control mode of the tower crane to make the tower crane slowly run to the end point.

[0168] In one example, the preset distance can be set to 2 m. For example, when the tower crane is automatically controlled to run to a position 2 m away from the target point, that is, when the distance between the bottom of the lifted object and the target point is 2 m, switch to the "ant speed" control mode, and control the three axes of lifting, luffing, and slewing to move slowly, and lower the lifted object until it is convenient for the operator to unload the lifted object. Another example is when the tower crane is automatically controlled to run directly above the target point at a height of 2 m, switch to the "ant speed" control mode, and control the lifting axis to move slowly, and lower the lifted object until it is convenient for the operator to unload the lifted object.

[0169] When the lifted object is approaching the end point of the planned path, if the running speed of the tower crane is too fast, it may collide with the object at the end point. Therefore, when the lifted object is approaching the end point of the planned path, control the tower crane to run slowly. The remote controller can be used to operate and control to make the lifted object move slowly to the target position, and then lower the lifted object. During the slow movement process, it is possible that the three axes are all moving, or only two axes are moving, or only one axis is moving.

[0170] In summary, in the embodiment of the present application, a static obstacle is first established, and the following principles are defined for path planning: 1) Except for the ascending and descending paths, the planned path runs outside the reconstructed obstacles in other areas; 2) First lift to the highest point of this operation, then control the slewing and luffing to directly above the target end point, and finally lift and lower to the target point. During the unmanned operation stage of the construction machinery, path planning is performed based on the static obstacle when about to run to avoid obstacles; the dynamic obstacles are reconstructed and extracted in real time, and it is judged whether the obstacle is on the running path, so as to make a decision on whether to pause or re-plan the route to bypass the obstacle. By adopting this method, the unmanned operation of the construction machinery can be completed while ensuring safety.

[0171] As Figure 4 shown, the present application also provides an embodiment of a path planning device for a construction machinery. Regarding the beneficial effects or technical problems solved by the device, reference can be made to the descriptions in the methods corresponding to the respective devices, or the descriptions in the summary of the invention, which will not be elaborated here one by one.

[0172] In the embodiment of the path planning device for the construction machinery, the device includes:

[0173] Extraction unit 100, configured to: during the operation of construction machinery, reconstruct dynamic obstacles in the working area of the construction machinery, and extract the bounding boxes of the dynamic obstacles;

[0174] Judgment unit 200, configured to: based on the bounding boxes of the dynamic obstacles, judge whether the planned path of the construction machinery intersects with the dynamic obstacles; wherein, the planned path is obtained by pre-planning the path of the construction machinery based on static obstacles;

[0175] Planning unit 300, configured to: in the case where the planned path of the construction machinery intersects with the dynamic obstacles, re-plan the path of the construction machinery so that the running trajectory of the construction machinery avoids the dynamic obstacles.

[0176] In one implementation, the construction machinery includes a tower crane; the extraction unit 100 is configured to:

[0177] Use a lidar to establish a point cloud map of the working area of the tower crane;

[0178] Extract non-obstacle point cloud information from the point cloud map; wherein, the non-obstacle point cloud information includes the point cloud information of the hook, the suspended load, and the suspension rope of the tower crane, and the suspension rope is the suspension rope connecting the hook and the suspended load;

[0179] Remove the non-obstacle point cloud information from the point cloud map to obtain a non-obstacle point cloud map;

[0180] Extract the bounding boxes of the dynamic obstacles from the non-obstacle point cloud map.

[0181] In one implementation, the path planning is set as: in the direction of the lifting axis, rise from the current point to the lifting height value; keep the coordinates of the lifting axis unchanged, and run to directly above the end point in the directions of the slewing axis and the luffing axis; in the direction of the lifting axis, descend to reach the position of the end point.

[0182] In one implementation, the lidar includes a first lidar and a second lidar; the first lidar is arranged below the boom of the tower crane and at the position of the tower crane arm root; the second lidar is arranged on the trolley of the tower crane;

[0183] The extraction unit 100 is configured to: extract a first bounding box of the dynamic obstacle by using the point cloud map established by the first lidar; the judgment unit 200 and the planning unit 300 are configured to: re-plan the path in the directions of the tower crane's slewing axis and luffing axis based on the first bounding box; wherein, the judgment unit 200 is configured to: judge whether the planned path of the tower crane intersects with the dynamic obstacle based on the first bounding box; the planning unit 300 is configured to: re-plan the path in the directions of the tower crane's slewing axis and luffing axis when the planned path of the tower crane intersects with the dynamic obstacle; and / or,

[0184] The extraction unit 100 is configured to: extract a second bounding box of the dynamic obstacle by using the point cloud map established by the second lidar; the judgment unit 200 and the planning unit 300 are configured to: re-plan the path in the direction of the tower crane's hoisting axis based on the second bounding box; wherein, the judgment unit 200 is configured to: judge whether the planned path of the tower crane intersects with the dynamic obstacle based on the second bounding box; the planning unit 300 is configured to: re-plan the path in the directions of the tower crane's slewing axis and luffing axis when the planned path of the tower crane intersects with the dynamic obstacle.

[0185] As Figure 5 shown, in one implementation, the device further includes a monitoring unit 400, and the monitoring unit 400 is configured to:

[0186] During the operation of the tower crane in the hoisting axis direction, monitor whether the suspended load swings abnormally based on the point cloud map established by the second lidar;

[0187] When it is monitored that the suspended load swings abnormally, execute the corresponding preset strategy.

[0188] In one implementation, the judgment unit is configured to:

[0189] Expand the planned path of the construction machinery into a planned channel, where the planned channel is a channel with the planned path as the center line and a preset cross-sectional shape;

[0190] Judge whether the planned channel intersects with the dynamic obstacle.

[0191] As Figure 5 shown, in one implementation, the construction machinery includes a tower crane; the device further includes a control unit 500, and the control unit 500 is configured to:

[0192] When the lifted object of the tower crane is moving downward along the lifting axis and the distance between the lifted object and the end point of the planned path is a preset distance, switch the control mode of the tower crane to make the tower crane slowly move to the end point.

[0193] Figure 6 FIG. 4 is a schematic structural diagram of a computing device 900 provided by an embodiment of the present application. The computing device 900 includes: a processor 910, a memory 920, and a communication interface 930.

[0194] It should be understood that Figure 6 the communication interface 930 in the computing device 900 shown in FIG. 4 can be used for communication with other devices.

[0195] Wherein, the processor 910 can be connected to the memory 920. The memory 920 can be used to store the program code and data. Therefore, the memory 920 can be an internal storage unit of the processor 910, an external storage unit independent of the processor 910, or a component including an internal storage unit of the processor 910 and an external storage unit independent of the processor 910.

[0196] Optionally, the computing device 900 may further include a bus. Wherein, the memory 920 and the communication interface 930 can be connected to the processor 910 through the bus. The bus can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc.

[0197] It should be understood that in the embodiment of the present application, the processor 910 can be a central processing unit (CPU). The processor can also be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. Or the processor 910 adopts one or more integrated circuits to execute relevant programs to implement the technical solutions provided by the embodiments of the present application.

[0198] The memory 920 may include a read-only memory and a random access memory, and provide instructions and data to the processor 910. A part of the processor 910 may also include a non-volatile random access memory. For example, the processor 910 may also store information about the device type.

[0199] When the computing device 900 is running, the processor 910 executes the computer-executable instructions in the memory 920 to perform the operation steps of the above method.

[0200] It should be understood that the computing device 900 according to the embodiments of the present application may correspond to the corresponding subject that executes the methods according to the various embodiments of the present application, and the above and other operations and / or functions of each module in the computing device 900 respectively implement the corresponding processes of the methods of each embodiment. For the sake of brevity, they will not be described in detail here.

[0201] Those of ordinary skill in the art can realize that the units and algorithm steps of the examples described in conjunction with the embodiments disclosed herein can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present application.

[0202] Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working processes of the systems, devices, and units described above can refer to the corresponding processes in the foregoing method embodiments, and will not be described in detail here.

[0203] In several embodiments provided by the present application, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed mutual coupling or direct coupling or communication connection may be through some interfaces. The indirect coupling or communication connection of the devices or units may be in an electrical, mechanical, or other form.

[0204] The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0205] In addition, in each embodiment of the present application, each functional unit can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit.

[0206] If the above-mentioned functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in each embodiment of the present application. The foregoing storage medium includes: various media that can store program codes, such as USB flash drives, mobile hard disks, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical discs.

[0207] The embodiments of the present application also provide a computer-readable storage medium, on which a computer program is stored. When the program is executed by a processor, it is used to execute a path planning method for construction machinery, and the method includes at least one of the solutions described in the above-mentioned various embodiments.

[0208] The computer storage medium of the embodiments of the present application can adopt any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples (non-exhaustive list) of the computer-readable storage medium include: an electrical connection having one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, the computer-readable storage medium can be any tangible medium that contains or stores a program, and the program can be used by or combined with an instruction execution system, apparatus, or device.

[0209] A computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which computer-readable program code is carried. Such a propagated data signal may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the foregoing. The computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device.

[0210] The program code contained on a computer-readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wire, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0211] The computer program code for performing the operations of this application may be written in one or more programming languages or combinations thereof. The programming languages include object-oriented programming languages such as Java, Smalltalk, C++, and also include conventional procedural programming languages such as the "C" language or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, executed as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., by using an Internet service provider to connect through the Internet).

[0212] Note that the above is only the preferred embodiment of this application and the technical principles applied. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein. Various obvious changes, re-adjustments, and substitutions can be made by those skilled in the art without departing from the protection scope of the present invention. Therefore, although this application has been described in more detail through the above embodiments, the present invention is not limited to the above embodiments. Without departing from the concept of the present invention, more other equivalent embodiments may be included, all of which fall within the protection scope of the present invention.

Claims

1. A path planning method for construction machinery, characterized in that, it includes: During the operation of the construction machinery, reconstruct the dynamic obstacles in the working area of the construction machinery, and extract the bounding boxes of the dynamic obstacles; Based on the bounding boxes of the dynamic obstacles, determine whether the planned path of the construction machinery intersects with the dynamic obstacles; wherein, the planned path is obtained by pre-planning the path of the construction machinery based on static obstacles; In the case where the planned path of the construction machinery intersects with the dynamic obstacles, re-plan the path of the construction machinery so that the running trajectory of the construction machinery avoids the dynamic obstacles.

2. The method according to claim 1, characterized in that, the construction machinery includes a tower crane; the reconstruction of the dynamic obstacles in the working area of the construction machinery and the extraction of the bounding boxes of the dynamic obstacles include: Using a lidar to establish a point cloud map of the working area of the tower crane; Extract non-obstacle point cloud information from the point cloud map; wherein, the non-obstacle point cloud information includes the point cloud information of the hook, the suspended load and the suspension rope of the tower crane, and the suspension rope is the suspension rope connecting the hook and the suspended load; Remove the non-obstacle point cloud information from the point cloud map to obtain a non-obstacle point cloud map; Extract the bounding boxes of the dynamic obstacles from the non-obstacle point cloud map.

3. The method according to claim 2, characterized in that, The path planning is set as: in the direction of the lifting axis, rise from the current point to the lifting height value; keep the coordinates of the lifting axis unchanged, and run to directly above the end point in the directions of the slewing axis and the luffing axis; in the direction of the lifting axis, descend to reach the position of the end point.

4. The method according to claim 3, characterized in that, The lidar includes a first lidar and a second lidar; the first lidar is arranged below the boom of the tower crane and at the position of the root of the tower crane boom; the second lidar is arranged on the trolley of the tower crane; the method further includes: Using the point cloud map established by the first lidar, extract the first bounding box of the dynamic obstacles; based on the first bounding box, re-plan the path in the directions of the slewing axis and the luffing axis of the tower crane; and / or, Using the point cloud map established by the second lidar, extract the second bounding box of the dynamic obstacles; based on the second bounding box, re-plan the path in the direction of the lifting axis of the tower crane.

5. The method according to claim 4, characterized in that, The method further includes: During the operation of the tower crane in the direction of the lifting axis, monitor whether the suspended load swings abnormally based on the point cloud map established by the second lidar; In the case where it is monitored that the suspended load swings abnormally, execute the corresponding preset strategy.

6. The method according to any one of claims 1 to 5, characterized in that, The determination of whether the planned path of the construction machinery intersects with the dynamic obstacles based on the bounding boxes of the dynamic obstacles includes: Expand the planned path of the construction machinery into a planned channel, and the planned channel is a channel with the planned path as the center line and a preset cross-sectional shape; Determine whether the planned passage intersects with the dynamic obstacle.

7. The method according to any one of claims 1 to 5, wherein, the construction machinery includes a tower crane; the method further includes: when the lifted object of the tower crane is descending along the lifting axis and the distance between the lifted object and the end point of the planned path is a preset distance, switch the control mode of the tower crane to make the tower crane slowly run to the end point.

8. A path planning device for construction machinery, wherein, it includes: an extraction unit for reconstructing the dynamic obstacles in the working area of the construction machinery and extracting the bounding boxes of the dynamic obstacles during the operation of the construction machinery; a judgment unit for judging whether the planned path of the construction machinery intersects with the dynamic obstacles based on the bounding boxes of the dynamic obstacles; wherein, the planned path is obtained by pre-planning the path of the construction machinery based on static obstacles; a planning unit for re-planning the path of the construction machinery when the planned path of the construction machinery intersects with the dynamic obstacles, so that the running track of the construction machinery avoids the dynamic obstacles.

9. A computing device, wherein, it includes: a communication interface; at least one processor connected to the communication interface; and at least one memory connected to the processor and storing program instructions, and when the program instructions are executed by the at least one processor, the at least one processor executes the method according to any one of claims 1-7.

10. A computer-readable storage medium having program instructions stored thereon, wherein, when the program instructions are executed by a computer, the computer executes the method according to any one of claims 1-7.

Images