Tower crane lifting hook auxiliary control system based on multiple sensors

Abstract

The invention relates to a multi-sensor-based auxiliary control system for a lifting hook of a tower crane. Laser radars and cameras are mounted at a cargo boom head of the tower crane, a variable-amplitude trolley, a rotary platform and a rear balance arm of the tower crane; a lifting mechanism, a rotating mechanism and a luffing mechanism of the tower crane are respectively provided with a position encoder for detecting the movement displacement of the tower crane mechanism in real time; through computer vision and a three-dimensional reconstruction technology, image and distance data captured by a sensor are subjected to feature matching and coordinate calculation, three-dimensional data of a tower crane working environment is generated, space coordinates of a cargo hoisting starting point and a cargo hoisting ending point are marked and confirmed by using a positioning plate, and after a camera and a laser radar identify cargoes at the starting point coordinates, the cargo hoisting starting point is identified by a camera and a laser radar. The controller controls the trolley lifting hook to move to the position above the goods, at the moment, the remote controller is operated to adjust the position of the lifting hook, after it is confirmed that the goods are fixed, the data processing module automatically plans a path according to the combination of real-time three-dimensional information and computer vision information, the lifting hook is controlled to move to end point coordinates, and lifting work is completed.

Description

Technical Field

[0001] The invention relates to a tower crane hook auxiliary control system based on multiple sensors, belonging to the technical field of tower cranes. Background Art

[0002] Tower cranes are the most commonly used lifting equipment on construction sites, mainly used for vertical transportation of materials and component installation in multi-story and high-rise building construction. Application areas include high-rise building construction, bridge and tunnel construction, port and wharf engineering, wind power and nuclear power plant construction, etc. With the development of the times, the tower crane industry will continue to promote the innovative application of intelligent, automated, digital and other technologies in the future, while improving the performance, safety and efficiency of equipment.

[0003] During the service life of a conventional tower crane on a construction site, the tower crane base is usually fixed in a certain position and the working environment is fixed. The lifting of cargo by tower cranes requires the tower crane operator and ground loading and unloading personnel to communicate and cooperate through walkie-talkies. When the hook is lowered to the ground, the operator cannot accurately judge the position of the hook in the driver's cab due to factors such as the operator's vision. The ground staff needs to repeatedly pass information to the tower crane operator, which increases the complexity of lifting and cannot guarantee safety. Summary of the invention

[0004] In order to enhance the safety and reliability of tower crane hoisting operations, reduce operational errors and improve work efficiency, the present invention proposes a tower crane hook auxiliary control system based on multiple sensors. The tower crane is equipped with multiple sensors such as binocular cameras, laser radars, and position encoders. The three-dimensional reconstruction of the tower crane working scene is completed by using multiple sensors, and the spatial coordinates of the hook, cargo, and staff are obtained at the same time. The data processing module can automatically plan the path according to the identified spatial information to complete the cargo hoisting.

[0005] The present invention is realized by the following technical scheme: a tower crane hook auxiliary control system based on multiple sensors,

[0006] Laser radars and cameras are installed on the boom head, luffing trolley, slewing platform and rear balance arm of the tower crane. At the same time, the lifting, slewing and luffing mechanisms of the tower crane are equipped with position encoders to detect the displacement of the tower crane mechanism in real time and achieve accurate positioning of the tower crane hook.

[0007] Through computer vision and 3D reconstruction technology, the images and distance data captured by the sensor are feature matched and coordinates calculated to generate 3D data of the tower crane's working environment. The spatial coordinates of the starting and ending points of the cargo lifting are marked and confirmed using the positioning plate. After the camera and lidar identify the cargo at the starting coordinates, the controller controls the trolley hook to move above the cargo. At this time, the remote control is operated to adjust the hook position. After confirming that the cargo is fixed, the data processing module automatically plans the path based on the real-time 3D information measured by the sensor near the tower crane and the computer vision information, controls the hook to move to the end coordinates, and completes the lifting work.

[0008] The camera is a binocular camera. Multiple groups of binocular cameras are respectively installed on the boom head, boom root and trolley of the tower crane. The two cameras of the binocular camera jointly shoot the same scene. They are placed in different positions and capture images at different angles. By comparing the difference between the two images, the depth information of each object in the scene is calculated to achieve three-dimensional reconstruction, and the three-dimensional model of the scene is quickly obtained using the parallax method.

[0009] The laser radar detects the position and speed characteristics of the target by emitting a laser beam, sends a signal to the target, and receives the reflected signal, and obtains the distance, position, and shape information of the target after processing.

[0010] The position encoder converts angular displacement or linear displacement into electrical signals, and then compiles the electrical signals into a form that can be used for storage for subsequent data processing and analysis. By configuring position encoders on the lifting, rotation, and luffing mechanisms of the tower crane, accurate feedback information on the movement and speed of the tower crane can be obtained.

[0011] The data processing module determines the position of the cargo to be hoisted and the position of the target point of the hoisting transportation through a three-dimensional model generated by computer vision and three-dimensional reconstruction technology. After obtaining the position information, the data processing module will formulate a hook control strategy through the three-dimensional information of the three-dimensional model and environmental modeling. After the hook control strategy is determined, it starts to send a PLC controller control signal to automatically operate the tower crane.

[0012] The environmental modeling adopts the establishment of a two-dimensional environmental map model based on a grid method. The number of grids and the size of grids in the grid environmental map model can be changed as needed.

[0013] According to the information of the three-dimensional model generated by the three-dimensional reconstruction technology, the ground working area is divided into two-dimensional grid units of the same size and fixed, so that the device is established in a two-dimensional rectangular coordinate system, and each grid unit is numbered. The coordinates of the grid are defined as the center point coordinates of each grid unit, so that the grid coordinates and the serial number can be converted to each other. In the grid map model, the obstacle grid is represented by a black grid and the free grid is represented by a white grid. The mathematical expression of the corresponding relationship between the grid coordinates (X, Y) and the serial number N can be established as follows:

[0014]

[0015] In order to further accurately locate the position information of the lifted cargo, a positioning plate is used during the operation to assist in positioning the target point of the hook.

[0016] The positioning plate is a regularly arranged characteristic pattern with sharp contrasting colors. Patterns are painted on both sides of the positioning plate. The patterns on the front and back sides AB are painted in different styles to define the starting and end points of the lifting. After completing a lifting work, the reverse movement of the lifting is achieved by flipping the positioning plate.

[0017] The positioning plate adopts a nine-square chessboard pattern and uses black and white colors. The patterns on the front and back sides A and B of the positioning plate are different. The A side of the positioning plate is defined as the starting point of lifting, and the B side is defined as the target point of lifting. During the lifting operation, the staff responsible for lifting the goods will flip the positioning plate to the A side and place it near the goods. The staff responsible for unloading will place the B side of the positioning plate at the transportation target point. After the staff has placed the positioning plate, the staff will operate the remote control to determine the start of the operation.

[0018] The beneficial effects of the present invention are: by using binocular cameras, laser radar, position encoders in conjunction with computer vision, three-dimensional reconstruction technology and data processing modules, this control technology has high precision. This control technology can monitor the rotation angle of the tower crane, the position of the tower crane luffing trolley, the height of the tower crane hook from the ground and other information in real time, and can monitor the surrounding environment of the tower crane at the same time, and can monitor all working conditions such as the tower crane lifting off the ground, landing, and normal operation, so as to ensure safe and efficient work. This control technology has no blind spot in the field of vision. Compared with the traditional tower crane operating the tower crane on the tower, it does not need to rely on the naked eye to judge the hook position. When the tower crane is operating the tower crane on the tower, the field of vision is easily affected by the surrounding environment, and it is necessary to communicate frequently with the ground staff, which increases the danger of the tower crane and the ground staff, and also reduces the work efficiency. However, this tower crane hoisting control technology based on multiple sensors is not affected by obstructions in the environment, improves the work efficiency, and does not require the tower crane to go up the tower, which also improves the safety of the ground staff. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The present invention is further described below with reference to the accompanying drawings and embodiments.

[0020] Figure 1 is a system architecture diagram of the present invention;

[0021] Figure 2 It is a schematic diagram of the installation position of the binocular camera of the present invention;

[0022] Figure 3 It is a schematic diagram of a grid environment map model of the present invention;

[0023] Figure 4 is a schematic diagram of surface A of the positioning plate of the present invention;

[0024] Figure 5 is a schematic diagram of surface B of the positioning plate of the present invention;

[0025] Figure 6 It is a schematic diagram of the operating field of the tower of the present invention;

[0026] Figure 7 Schematic diagram of the binocular camera field of view of the present invention. DETAILED DESCRIPTION

[0027] like Figure 1 and Figure 2 The tower crane hook auxiliary control system shown in the figure is based on a multi-sensor, including a laser radar and a camera installed at the boom head, luffing trolley, slewing platform and rear balance arm of the tower crane; at the same time, the lifting, slewing and luffing mechanisms of the tower crane are all equipped with position encoders to detect the displacement of the tower crane mechanism in real time, so as to realize the precise positioning of the tower crane hook;

[0028] Through computer vision and 3D reconstruction technology, the images and distance data captured by the sensor are feature matched and coordinates calculated to generate 3D data of the tower crane's working environment. The spatial coordinates of the starting and ending points of cargo lifting are marked and confirmed using the positioning plate. After the camera and lidar identify the cargo at the starting coordinates, the controller controls the trolley hook to move to 1 meter above the cargo. At this time, the staff operates the remote control to adjust the hook position to facilitate cargo lifting. After the staff confirms that the cargo is fixed, the data processing module automatically plans the path based on the real-time 3D information measured by the sensor near the tower crane and the computer vision information, controls the hook to move to the end coordinates, and completes the lifting work.

[0029] The camera is a binocular camera. Multiple groups of binocular cameras are respectively installed on the boom head, boom root and trolley of the tower crane. The two cameras of the binocular camera jointly shoot the same scene. They are placed in different positions and capture images at different angles. By comparing the difference between the two images, the depth information of each object in the scene is calculated to achieve three-dimensional reconstruction, and the three-dimensional model of the scene is quickly obtained using the parallax method.

[0030] The laser radar detects the position and speed characteristics of the target by emitting a laser beam, sends a signal to the target, and receives the reflected signal, and obtains the distance, position, and shape information of the target after processing.

[0031] The position encoder converts angular displacement or linear displacement into electrical signals, and then compiles the electrical signals into a form that can be used for storage for subsequent data processing and analysis. By configuring position encoders on the lifting, rotation, and luffing mechanisms of the tower crane, accurate feedback information on the movement and speed of the tower crane can be obtained.

[0032] The data processing module determines the position of the cargo to be hoisted and the position of the target point of the hoisting transportation through a three-dimensional model generated by computer vision and three-dimensional reconstruction technology. After obtaining the position information, the data processing module will formulate a hook control strategy through the three-dimensional information of the three-dimensional model and environmental modeling. After the hook control strategy is determined, it starts to send a PLC controller control signal to automatically operate the tower crane.

[0033] The environmental modeling adopts the establishment of a two-dimensional environmental map model based on a grid method. The number of grids and the size of grids in the grid environmental map model can be changed as needed. Figure 3 As shown, the number of grids and the size of grids can be changed as needed.

[0034] According to the information of the three-dimensional model generated by the three-dimensional reconstruction technology, the ground working area is divided into two-dimensional grid units of the same size and fixed, so that the device is established in a two-dimensional rectangular coordinate system, and each grid unit is numbered. The coordinates of the grid are defined as the center point coordinates of each grid unit, so that the grid coordinates and the serial number can be converted to each other. In the grid map model, the obstacle grid is represented by a black grid and the free grid is represented by a white grid. The mathematical expression of the corresponding relationship between the grid coordinates (X, Y) and the serial number N can be established as follows:

[0035]

[0036] In order to further accurately locate the position information of the lifted cargo, a positioning plate is used during the operation to assist in positioning the target point of the hook.

[0037] The positioning plate is a regularly arranged characteristic pattern with a sharp contrast in color. The front and back sides of the positioning plate are painted with patterns. The patterns on the front and back sides AB are painted in different styles to define the starting point and end point of the lifting. After completing a lifting work, the positioning plate is flipped to achieve the reverse movement of the lifting. The positioning plate can be designed in size according to the height of the tower crane, so that the camera can quickly and accurately locate the lifting point.

[0038] The positioning plate adopts a nine-square chessboard pattern, using black and white colors, with a large color difference, simple and eye-catching. The patterns on the front and back sides A and B of the positioning plate are as follows Figure 4 , 5 As shown, the patterns on the front and back sides A and B of the positioning plate are different, where side A of the positioning plate is defined as the starting point for lifting, and side B is defined as the target point for lifting. During the lifting operation, the staff responsible for lifting the goods will flip the positioning plate to side A and place it near the goods, and the staff responsible for unloading will place side B of the positioning plate at the target point for transportation. After the staff has placed the positioning plate, they will operate the remote control to determine the start of the operation.

[0039] The present invention can monitor the rotation angle of the tower crane, the position of the tower crane luffing trolley, the height of the tower crane hook from the ground and other information in real time, and can also monitor the surrounding environment of the tower crane, and can detect all working conditions such as the tower crane lifting off the ground, landing on the ground, and normal operation, to ensure safe and efficient work.

[0040] The present invention has no blind spot in the field of vision. Compared with the traditional tower crane on the tower, the tower crane does not need to rely on the naked eye to judge the hook position. When the tower crane is on the tower, the vision is easily affected by the surrounding environment, and frequent communication with the ground staff is required, which increases the danger of the tower crane and the ground staff and reduces the work efficiency. Figure 6 This multi-sensor tower crane hoisting control technology is not affected by obstructions in the environment, which improves work efficiency. It also improves the safety of ground workers without the need for a crane to climb the tower. Figure 7 .

[0041] The present invention can upload data information of the tower crane and the surrounding environment to the network, and the staff can also monitor various information in real time through the network, and can observe the real-time camera image and the environmental model and PLC controller signal generated by three-dimensional reconstruction in the present invention through the on-site terminal display screen, so as to have the ability to respond quickly in emergency situations.

Claims

1. A tower crane hook auxiliary control system based on multiple sensors, characterized in that: include: Laser radars and cameras are installed on the boom head, luffing trolley, slewing platform and rear balance arm of the tower crane. At the same time, the lifting, slewing and luffing mechanisms of the tower crane are equipped with position encoders to detect the displacement of the tower crane mechanism in real time and achieve accurate positioning of the tower crane hook. Through computer vision and 3D reconstruction technology, the images and distance data captured by the sensor are feature matched and coordinates calculated to generate 3D data of the tower crane's working environment. The spatial coordinates of the starting and ending points of the cargo lifting are marked and confirmed using the positioning plate. After the camera and lidar identify the cargo at the starting coordinates, the controller controls the trolley hook to move above the cargo. At this time, the remote control is operated to adjust the hook position. After confirming that the cargo is fixed, the data processing module automatically plans the path based on the real-time 3D information measured by the sensor near the tower crane and the computer vision information, controls the hook to move to the end coordinates, and completes the lifting work.

2. The multi-sensor tower crane hook auxiliary control system according to claim 1, characterized in that: The camera is a binocular camera. Multiple groups of binocular cameras are respectively installed on the boom head, boom root and trolley of the tower crane. The two cameras of the binocular camera jointly shoot the same scene. They are placed in different positions and capture images at different angles. By comparing the difference between the two images, the depth information of each object in the scene is calculated to achieve three-dimensional reconstruction, and the three-dimensional model of the scene is quickly obtained using the parallax method.

3. The multi-sensor based tower crane hook auxiliary control system according to claim 1, characterized in that: The laser radar detects the position and speed characteristics of the target by emitting a laser beam, sends a signal to the target, and receives the reflected signal, and obtains the distance, position, and shape information of the target after processing.

4. The multi-sensor based tower crane hook auxiliary control system according to claim 1, characterized in that: The position encoder converts angular displacement or linear displacement into electrical signals, and then compiles the electrical signals into a form that can be used for storage for subsequent data processing and analysis. By configuring position encoders on the lifting, rotation, and luffing mechanisms of the tower crane, accurate feedback information on the movement and speed of the tower crane can be obtained.

5. The multi-sensor based tower crane hook auxiliary control system according to claim 1, characterized in that: The data processing module determines the position of the cargo to be hoisted and the position of the target point of the hoisting transportation through a three-dimensional model generated by computer vision and three-dimensional reconstruction technology. After obtaining the position information, the data processing module will formulate a hook control strategy through the three-dimensional information of the three-dimensional model and environmental modeling. After the hook control strategy is determined, it starts to send a PLC controller control signal to automatically operate the tower crane.

6. The multi-sensor based tower crane hook auxiliary control system according to claim 5, characterized in that: The environmental modeling adopts the establishment of a two-dimensional environmental map model based on a grid method. The number of grids and the size of grids in the grid environmental map model can be changed as needed. According to the information of the three-dimensional model generated by the three-dimensional reconstruction technology, the ground working area is divided into two-dimensional grid units of the same size and fixed, so that the device is established in a two-dimensional rectangular coordinate system, and each grid unit is numbered. The coordinates of the grid are defined as the center point coordinates of each grid unit, so that the grid coordinates and the serial number can be converted to each other. In the grid map model, the obstacle grid is represented by a black grid and the free grid is represented by a white grid. The mathematical expression of the corresponding relationship between the grid coordinates (X, Y) and the serial number N can be established as follows: In order to further accurately locate the position information of the lifted cargo, a positioning plate is used during the operation to assist in positioning the target point of the hook.

7. The multi-sensor based tower crane hook auxiliary control system according to claim 6, characterized in that: The positioning plate is a regularly arranged characteristic pattern with sharp contrasting colors. Patterns are painted on both sides of the positioning plate. The patterns on the front and back sides AB are painted in different styles to define the starting and end points of the lifting. After completing a lifting work, the reverse movement of the lifting is achieved by flipping the positioning plate.

8. The multi-sensor based tower crane hook auxiliary control system according to claim 6, characterized in that: The positioning plate adopts a nine-square chessboard pattern and uses black and white colors. The patterns on the front and back sides A and B of the positioning plate are different. The A side of the positioning plate is defined as the starting point of lifting, and the B side is defined as the target point of lifting. During the lifting operation, the staff responsible for lifting the goods will flip the positioning plate to the A side and place it near the goods. The staff responsible for unloading will place the B side of the positioning plate at the transportation target point. After the staff has placed the positioning plate, the staff will operate the remote control to determine the start of the operation.

Images