Method and device for generating ship unloading operation path based on portal crane automation

Abstract

The invention relates to the technical field, in particular to a ship unloading operation path generation method and device based on portal crane automation. A control scanning module is arranged to conduct 3D scanning and modeling on a ship body, modeling data are subjected to feature extraction and sent to a main control system, the main control system conducts algorithm process processing on 3D point cloud data sent by the scanning module, hatch position information is extracted, and 3D point cloud data of materials are obtained; and the main control system generates an operation point according to the 3D point cloud data, synthesizes the rotation direction, the starting point, the ending point and the unloading point of the portal crane, generates an operation path in combination with obstacle information, controls the portal crane to operate, and completes cyclic actions of closing the grab bucket to grab materials, transferring the grab bucket, opening the grab bucket to discharge materials and returning to a cabin. The ship body is scanned and modeled through the three-dimensional laser scanner, feature extraction is carried out on modeling data, a track from a grabbing point to a discharging point is planned, and the automatic and semi-automatic loading and unloading functions of the portal crane are achieved.

Description

Technical Field

[0001] The present invention relates to the technical field, and in particular to a method and a device for generating a ship unloading operation path based on gantry crane automation. Background Art

[0002] Among many lifting equipment, gantry cranes (hereinafter referred to as gantry cranes) play a very important role. Because of its various loading and unloading types, high loading and unloading efficiency, and convenient operation, it is widely used in large quantities in the operation area. As a large-scale engineering machinery equipment, the gantry crane has a large operating range. The traditional operation method is to rely solely on the operator in the driver's cab for control, which is labor-intensive, the operating efficiency cannot be guaranteed, and there are safety risks. With the continuous increase in the cargo throughput of the terminal, the single-person single-machine operation mode can no longer meet the actual needs. Summary of the invention

[0003] The purpose of the present invention is to address the above-mentioned deficiencies in the prior art and to provide a method and device for automatic ship unloading of a gantry crane based on laser scanning modeling. The hull is scanned and modeled by a three-dimensional laser scanner, and feature extraction is performed on the modeling data to plan a trajectory from the grabbing point to the discharging point, thereby realizing the automatic and semi-automatic loading and unloading functions of the gantry crane.

[0004] The technical solution of the present invention is as follows:

[0005] A method for generating a ship unloading operation path based on gantry crane automation, characterized in that:

[0006] The hull is 3D scanned and modeled by setting up a control scanning module, and features of the modeling data are extracted and sent to the main control system. The main control system processes the 3D point cloud data sent by the scanning module through an algorithm flow, extracts hatch position information, and obtains 3D point cloud data of the material;

[0007] The main control system generates operating points based on 3D point cloud data, integrates the rotation direction, starting point, end point, and unloading point of the gantry crane, and combines obstacle information to generate an operating path and control the operation of the gantry crane to complete the cycle of closed bucket grabbing, grab bucket transfer, open bucket unloading, and returning to the cabin.

[0008] Preferably, the hull is 3D scanned to obtain 3D point cloud data; a unified coordinate system is set in the 3D point cloud data, and the coordinate system is defined as the X axis parallel to the shoreline, the Y axis facing the sea side, the Z axis vertically upward, the origin is set at the gantry crane rotation center, and the ground height is 0. All information obtained based on this coordinate system is the coordinate basis for subsequent gantry crane execution.

[0009] Preferably, the main control system processes the 3D point cloud data through an algorithm flow to extract the coordinates of the hull hatch, wherein the hatch coordinate extraction content includes the hatch height coordinates and the X and Y coordinates of the four corner points of the hatch:

[0010] Preprocess the 3D point cloud data, homogenize the 3D point cloud data, remove noise, and obtain the original point cloud;

[0011] 2) The original point cloud is passed through the plane detection algorithm to extract the Z-direction plane, filter out the deck plane, and obtain the deck height coordinates;

[0012] 3) Combine the original point cloud with the deck height, perform Z-direction projection, and pass-filter the X and Y directions to obtain the Z-direction projection point cloud;

[0013] 4) Obtain the X-axis coordinate range of the hatch, combine the Z-axis projection point cloud with the X-axis coordinate value of the grab, directly filter the X-axis, obtain the X-minimum and X-maximum values ​​of the hatch, and then determine whether the X-value range is reasonable. If it is reasonable, the X-axis range of the hatch is obtained. If it is unreasonable, continue to extract the straight line parallel to the Y-axis. If the data extraction fails, the process ends. If the extraction is successful, the candidate cabin pairs in the X-axis are screened out, and the X-axis range of the hatch is obtained after processing the point cloud data;

[0014] 5) Obtain the Y coordinate range of the hatch, combine the Z-projection point cloud with the X coordinate range of the hatch, filter the X direction directly, obtain the Y minimum and Y maximum values ​​of the hatch, and then determine whether the Y value range is reasonable. If it is reasonable, the Y coordinate range of the hatch is obtained. If it is unreasonable, the extraction process ends;

[0015] 6) Obtain the Z coordinate range of the hatch, combine the original point cloud with the deck height and the X coordinate range and Y range of the hatch, and perform straight-through filtering to obtain the hatch height Z. Determine whether the hatch height Z is reasonable. If it is judged to be not reasonable, the calculation fails and the process ends. If it is judged to be reasonable, the hatch height Z is output.

[0016] The uniform processing of point cloud data mainly depends on the spatial interpolation algorithm. First, a grid is established on the target area, and then the nearest neighbor interpolation algorithm, arithmetic mean interpolation algorithm and distance inverse weighted interpolation algorithm are used to achieve uniform distribution of point cloud data.

[0017] Specific steps for point cloud homogenization:

[0018] Step 1: Create a grid on the point cloud target area;

[0019] Step 2: Remove noise points (also called isolated points / outliers / abnormal points) from the acquired point cloud;

[0020] Step 3: Use interpolation algorithm to fill in the residual point cloud and complete the point cloud;

[0021] Step 4: Use the algorithm to refine the point cloud to obtain a homogenized point cloud.

[0022] Noise points, also known as isolated points / outliers / abnormal points, refer to irrelevant or undesirable interference signals or errors in point cloud data. Noise usually comes from factors such as the brightness of ambient light, the accuracy and system errors of measuring equipment, the texture of object materials and surfaces, and human jitter. The brightness of ambient light, the accuracy and system errors of measuring equipment, the texture of object materials and surfaces, and human jitter. In point cloud data, the process of removing useless noise points through filtering and processing methods to improve data quality and accuracy. Remove outliers that are obviously sparsely distributed. According to the given mean and variance, points outside the variance can be eliminated, that is, points outside the variance are correct points.

[0023] The number of point clouds obtained by the scanner is extremely large, and the measurement data will reach tens of thousands or even hundreds of thousands of points. The huge amount of data contains unpredictable noise points. The source of noise points is not only the error caused by the laser scanning hardware itself, but also related to the actual collection situation, such as the presence of smoke, particles, edges and corners in the scanning area, which is obviously not conducive to the later work such as point cloud matching. Therefore, a series of data denoising is required to filter out the noise points.

[0024] Preferably, after obtaining the coordinates of the hull hatch position information, a filtering algorithm is used to create a three-dimensional voxel grid from the point cloud data obtained by scanning, and then the center of all points in the grid is used to approximate the display of other points in the voxel. In this way, all points in the voxel are ultimately represented by a center point, and the material area is segmented to obtain a 3D point cloud of the material.

[0025] Preferably, it also includes an anti-collision module, which is connected to the main control system. The anti-collision module enters obstacle information through laser scanning, obtains the position information and movement trend of the gantry crane through the main control system, and calculates the protection area of ​​the gantry crane boom by obtaining the position information and movement trend of the gantry crane. If the obstacle is within the protection area of ​​the boom, there is a possibility of collision, and the laser scanner automatically generates alarm information, and sends the alarm signal to the PLC through the Ethernet interface, so that the slewing mechanism slows down and stops. If the obstacle is outside the protection area of ​​the boom, there is no possibility of collision.

[0026] Preferably, the operation path includes the track of the gantry crane from the material grabbing point to the material discharging point. The operation path is combined with the position of the rotation center of the gantry crane and the equipment parameters of the gantry crane using the optimization idea, and then the path is planned according to the rotation direction, starting point, and end point of the gantry crane and the obstacle information. The operation track of each mechanism is planned so that the lifting mechanism, rotating mechanism, and luffing mechanism of the gantry crane can be linked to operate at appropriate positions, and at the same time, the boom and grab bucket of the gantry crane do not collide with obstacles during operation, and the operation efficiency is maximized.

[0027] As a preference, the gantry crane operator can also manually set the material grabbing area, unloading point, and obstacle point to the main control system.

[0028] A ship unloading device based on gantry crane automation, characterized in that it includes a scanning module and an anti-collision module, the scanning module includes a 3D laser scanning gimbal device I respectively arranged in the middle of the end of the gantry crane's trunk beam and on the side of the driver's cab platform, for identifying the required identification targets, the required identification targets include: ship hatch position, ship side height, materials, gantry crane loading and unloading position; the anti-collision module includes a laser scanning gimbal II respectively arranged on both sides of the boom of the gantry crane, the vertical field of view angle of the laser scanning gimbal II is 180°, the laser scanning gimbal II can be rotated to make the horizontal field of view angle reach any angle, the scanning surface of the laser scanning gimbal II is perpendicular to the ground, and the rotation axis of the laser scanning gimbal is perpendicular to the ground.

[0029] The two 3D laser scanning PTZ devices I can effectively expand the identifiable area and obtain more accurate data by scanning the data registration and fitting of the overlapping areas, so as to provide the gantry crane PLC with precise positioning control of the electrical and actuator movements, and successfully complete the fully automatic operation of the gantry crane with bucket. When the gantry crane is fully automatic, the 3D laser scanning PTZ device I adjusts the scanning direction and scanning area in real time according to the rotation speed of the gantry crane.

[0030] When the slewing mechanism is not in operation, the scanning surface of the laser scanning platform II is parallel to the boom direction and scans the area directly in front; when the slewing mechanism rotates to the right at the highest speed, the right laser scanning platform II rotates to the preset maximum angle. Once an obstacle is detected, the laser scanning platform II automatically generates an alarm message and sends the alarm signal to the portal crane PLC through the Ethernet interface, causing the slewing mechanism to slow down and stop. When the slewing mechanism rotates in the opposite direction, the left laser scanning platform II performs safety protection in the same way.

[0031] Beneficial effects of the present invention

[0032] The present invention uses scanning detection means and data algorithm support to scan and model the hull and extract features from the modeling data, thereby realizing automated bulk cargo operation of a gantry crane. During the operation, it ensures personnel safety, avoids production hazards, reduces dust pollution, reduces the labor intensity of drivers, and can display real-time location, behavior trajectory, etc., thereby promoting the intelligent construction of bulk cargo terminals and improving operating efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0034] Figure 1 It is a schematic diagram of obtaining 3D point cloud data by 3D scanning of the hull of the present invention.

[0035] Figure 2 Schematic diagram of the preprocessing process of 3D point cloud data of the present invention.

[0036] Figure 3 The present invention obtains the deck height algorithm processing flow chart.

[0037] Figure 4 This is a flowchart of the 3D point cloud Z-direction projection processing of the present invention.

[0038] Figure 5 It is a flow chart of obtaining hatch X coordinates of the present invention.

[0039] Figure 6 The present invention obtains the hatch Y coordinate flow chart.

[0040] Figure 7 The present invention is a flow chart for obtaining the hatch Z coordinate.

[0041] Figure 8 It is a schematic diagram of hatch point cloud data of the present invention.

[0042] Fig. 9 It is a schematic diagram of material point cloud data of the present invention.

[0043] Fig.10 It is a scanning schematic diagram of the anti-collision module of the present invention.

[0044] Fig.11 This is the path planning diagram of the present invention. DETAILED DESCRIPTION

[0045] In order to enable those skilled in the art to better understand the technical solutions in the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0046] Example 1

[0047] A method for generating a ship unloading operation path based on gantry crane automation, which performs 3D scanning and modeling of the hull by setting a control scanning module, extracts features from the modeling data and sends them to a main control system, which processes the 3D point cloud data sent by the scanning module through an algorithm flow, extracts hatch position information and obtains 3D point cloud data of the material; the main control system generates an operation point according to the 3D point cloud data, integrates the rotation direction, starting point, end point and unloading point of the gantry crane, and combines obstacle information to generate an operation path and control the operation of the gantry crane, thereby completing the cycle of grabbing materials with a closed bucket, transferring the grab bucket, discharging materials with an open bucket and returning to the cabin.

[0048] By performing 3D scanning on the hull, 3D point cloud data is obtained, such as Figure 1 As shown in the figure, a unified coordinate system is set in the 3D point cloud data, and the coordinate system is defined as the X axis parallel to the coastline, the Y axis facing the sea side, and the Z axis vertically upward. The origin is set at the gantry crane rotation center, and the ground height is 0. All information obtained based on this coordinate system is the coordinate basis for subsequent gantry crane execution.

[0049] The main control system processes the 3D point cloud data through an algorithm flow to extract the coordinates of the hull hatch. The hatch coordinate extraction includes the hatch height coordinates and the X and Y coordinates of the four corner points of the hatch:

[0050] 1) Preprocess the 3D point cloud data, homogenize the 3D point cloud data, remove noise, and obtain the original point cloud; the processing flow is as follows Figure 2 As shown;

[0051] 2) The original point cloud is passed through the plane detection algorithm to extract the Z-direction plane, filter out the deck plane, and obtain the deck height coordinates; the processing flow is as follows Figure 3 As shown;

[0052] 3) Combine the original point cloud with the deck height, perform Z-direction projection, and filter the X and Y directions to obtain the Z-direction projection point cloud; the processing flow is as follows: Figure 4 As shown;

[0053] 4) Obtain the X-axis coordinate range of the hatch, combine the Z-axis projection point cloud with the X-axis coordinate value of the grab, directly filter the X-axis, obtain the X-minimum and X-maximum values ​​of the hatch, and then determine whether the X-value range is reasonable. If it is reasonable, the X-axis range of the hatch is obtained. If it is unreasonable, continue to extract the straight line parallel to the Y-axis. If the data extraction fails, the process ends. If the extraction is successful, the candidate X-axis cabin pairs are screened out, and the X-axis range of the hatch is obtained after processing the point cloud data. The processing flow is as follows: Figure 5 As shown;

[0054] 5) Get the Y coordinate range of the hatch, combine the Z-projection point cloud with the X coordinate range of the hatch, filter the X direction, get the Y minimum and Y maximum values ​​of the hatch, and then judge whether the Y value range is reasonable. If it is reasonable, the Y coordinate range of the hatch is obtained. If it is unreasonable, the extraction fails and the process ends. The processing flow is as follows: Figure 6 As shown;

[0055] 6) Get the hatch Z coordinate range, combine the original point cloud with the deck height and the hatch X coordinate range and hatch Y range, and perform straight-through filtering to get the hatch height Z. Determine whether the hatch height Z is reasonable. If it is not, the calculation fails and the process ends. If it is reasonable, the hatch height Z is output. The processing flow is as follows: Figure 7 As shown, the hatch point cloud is as follows Figure 8 shown.

[0056] After obtaining the coordinates of the hull hatch position information, the material area is segmented using the filtering algorithm to obtain the 3D point cloud of the material. The 3D point cloud of the material is shown in the figure below: Fig. 9 shown.

[0057] The filtering algorithm uses the following logic:

[0058] main(int argc, char**argv)

[0059] {

[0060] PointCloud <pointxyz>::Ptr cloud(newpcl::PointCloud <pointxyz>);

[0061]

[0062] The method for generating the ship unloading operation path also includes an anti-collision module, which is connected to the main control system. The anti-collision module records obstacle information through laser scanning, obtains the position information and movement trend of the gantry crane through the main control system, and calculates the protection area of ​​the gantry crane boom by obtaining the position information and movement trend of the gantry crane. If the obstacle is within the protection area of ​​the boom, there is a possibility of collision, and the laser scanner automatically generates an alarm message and sends the alarm signal to the PLC through the Ethernet interface to slow down the slewing mechanism and stop it. If the obstacle is outside the protection area of ​​the boom, there is no possibility of collision. The schematic diagram of the anti-collision module scanning is shown in the figure below. Fig.10 shown.

[0063] The operation path includes the trajectory of the gantry crane from the material grabbing point to the material discharging point. Using the optimization idea, the operation path is combined with the position of the gantry crane's rotation center and the equipment parameters of the gantry crane. Then, the path is planned according to the rotation direction, starting point, and end point of the gantry crane and the obstacle information. The path planning diagram is as follows: Fig.11 shown.

[0064] As a preference, the gantry crane operator can also manually set the material grabbing area, unloading point, and obstacle point to the main control system.

[0065] A ship unloading device based on gantry crane automation includes a scanning module and an anti-collision module. The scanning module includes a 3D laser scanning gimbal device I respectively arranged in the middle of the end of the gantry crane's trunk beam and on the side of the driver's cab platform, and is used to identify required identification targets. The required identification targets include: ship hatch position, ship side height, materials, and gantry crane loading and unloading position; the anti-collision module includes a laser scanning gimbal II respectively arranged on both sides of the boom of the gantry crane, the vertical field of view of the laser scanning gimbal II is 180°, the laser scanning gimbal II can be rotated to make the horizontal field of view reach any angle, the scanning surface of the laser scanning gimbal II is perpendicular to the ground, and the rotation axis of the laser scanning gimbal is perpendicular to the ground.

[0066] The two 3D laser scanning PTZ devices I can effectively expand the identifiable area and obtain more accurate data by scanning the data registration and fitting of the overlapping areas, so as to provide the gantry crane PLC with precise positioning control of the electrical and actuator movements, and successfully complete the fully automatic operation of the gantry crane with bucket. When the gantry crane is fully automatic, the 3D laser scanning PTZ device I adjusts the scanning direction and scanning area in real time according to the rotation speed of the gantry crane.

[0067] When the slewing mechanism is not in operation, the scanning surface of the laser scanning platform II is parallel to the boom direction and scans the area directly in front; when the slewing mechanism rotates to the right at the highest speed, the right laser scanning platform II rotates to the preset maximum angle. Once an obstacle is detected, the laser scanning platform II automatically generates an alarm message and sends the alarm signal to the portal crane PLC through the Ethernet interface, causing the slewing mechanism to slow down and stop. When the slewing mechanism rotates in the opposite direction, the left laser scanning platform II performs safety protection in the same way.< / pointxyz> < / pointxyz>

Claims

1. A method for generating a ship unloading operation path based on gantry crane automation, characterized in that: The hull is 3D scanned and modeled by setting up a control scanning module, and the modeling data is feature extracted and sent to the main control system. The main control system processes the 3D point cloud data sent by the scanning module through an algorithm flow, extracts the hatch position information and obtains the 3D point cloud data of the material; The main control system generates operating points based on 3D point cloud data, integrates the rotation direction, starting point, end point, and unloading point of the gantry crane, and combines obstacle information to generate an operating path and control the operation of the gantry crane to complete the cycle of closed bucket grabbing, grab bucket transfer, open bucket unloading, and returning to the cabin.

2. The method for generating a ship unloading operation path based on gantry crane automation according to claim 1, characterized in that: 3D point cloud data is obtained by performing 3D scanning on the hull; a unified coordinate system is set in the 3D point cloud data, and the coordinate system is defined as the X-axis parallel to the shoreline, the Y-axis facing the sea side, the Z-axis vertically upward, the origin is set at the gantry crane rotation center, and the ground height is 0.

3. The method for generating a ship unloading operation path based on gantry crane automation according to claim 1, characterized in that: The main control system processes the 3D point cloud data through an algorithm flow to extract the coordinates of the hull hatch. The hatch coordinate extraction content includes the hatch height coordinates and the X and Y coordinates of the four corner points of the hatch: 1) Preprocess the 3D point cloud data, homogenize the 3D point cloud data, remove noise, and obtain the original point cloud; 2) The original point cloud is passed through the plane detection algorithm to extract the Z-direction plane, filter out the deck plane, and obtain the deck height coordinates; 3) Combine the original point cloud with the deck height, perform Z-direction projection, and pass-filter the X and Y directions to obtain the Z-direction projection point cloud; 4) Obtain the X-axis coordinate range of the hatch, combine the Z-axis projection point cloud with the X-axis coordinate value of the grab, directly filter the X-axis, obtain the X-minimum and X-maximum values ​​of the hatch, and then determine whether the X-value range is reasonable. If it is reasonable, the X-axis range of the hatch is obtained. If it is unreasonable, continue to extract the straight line parallel to the Y-axis. If the data extraction fails, the process ends. If the extraction is successful, the candidate cabin pairs in the X-axis are screened out, and the X-axis range of the hatch is obtained after processing the point cloud data; 5) Obtain the Y coordinate range of the hatch, combine the Z-projection point cloud with the X coordinate range of the hatch, filter the X direction directly, obtain the Y minimum and Y maximum values ​​of the hatch, and then determine whether the Y value range is reasonable. If it is reasonable, the Y coordinate range of the hatch is obtained. If it is unreasonable, the extraction process ends; 6) Obtain the Z coordinate range of the hatch, combine the original point cloud with the deck height and the X coordinate range and Y range of the hatch, and perform straight-through filtering to obtain the hatch height Z. Determine whether the hatch height Z is reasonable. If it is judged to be not reasonable, the calculation fails and the process ends. If it is judged to be reasonable, the hatch height Z is output.

4. The method for generating a ship unloading operation path based on gantry crane automation according to claim 1 is characterized in that: After obtaining the coordinates of the hull hatch position information, a filtering algorithm is used to create a three-dimensional voxel grid from the point cloud data obtained by scanning, and then the center of all points in the grid is used to approximate the other points in the voxel. In this way, all points in the voxel are finally represented by a center point, thereby achieving a filtering effect, segmenting the material area, and obtaining the 3D point cloud of the material.

5. The method for generating a ship unloading operation path based on gantry crane automation according to claim 1 is characterized in that: It also includes an anti-collision module, which is connected to the main control system. The anti-collision module records obstacle information through laser scanning, obtains the position information and movement trend of the portal crane through the main control system, and calculates the protection area of ​​the portal crane boom by obtaining the position information and movement trend of the portal crane; If the obstacle is within the protection area of ​​the boom, there is a possibility of collision. The laser scanner automatically generates an alarm message and sends the alarm signal to the PLC through the Ethernet interface, causing the slewing mechanism to slow down and stop. If the obstacle is outside the protection area of ​​the boom, there is no possibility of collision.

6. The method for generating a ship unloading operation path based on gantry crane automation according to claim 1 is characterized in that: The operation path includes the trajectory of the gantry crane from the material grabbing point to the material discharging point. Using the optimization idea, the operation path is combined with the position of the rotation center of the gantry crane and the equipment parameters of the gantry crane, and then the path is planned according to the rotation direction, starting point, and end point of the gantry crane combined with the obstacle information.

7. The method for generating a ship unloading operation path based on gantry crane automation according to claim 1, characterized in that: The gantry crane operator can also manually set the material grabbing area, unloading point and obstacle point on the main control system.

8. A ship unloading device based on gantry crane automation, characterized in that: It includes a scanning module and an anti-collision module. The scanning module includes a 3D laser scanning gimbal device I respectively arranged in the middle of the end of the gantry crane's trunk beam and on the side of the driver's cab platform, and is used to identify the required identification targets. The required identification targets include: ship hatch position, ship side height, materials, and gantry crane loading and unloading position; the anti-collision module includes a laser scanning gimbal II respectively arranged on both sides of the boom of the gantry crane, the vertical field of view angle of the laser scanning gimbal II is 180°, the laser scanning gimbal II can be rotated to make the horizontal field of view angle reach any angle, the scanning surface of the laser scanning gimbal II is perpendicular to the ground, and the rotation axis of the laser scanning gimbal is perpendicular to the ground.

Images