A method for recognizing the outline of quay crane container based on improved laser features

Abstract

This invention discloses a quay crane container outline recognition method based on improved laser features. This improved laser echo intensity feature uses two 2D lasers to scan the container horizontally and vertically at 180 degrees, respectively. The method, combined with the movement of quay crane trolleys, collects a complete operational map. The container outline and trolley position are constructed using a clustering method improved based on laser echo intensity and a laser radar odometer. The data acquisition module receives the container point cloud scanned by the laser sensor, pre-processes the acquired container laser point data, and inputs it into the container recognition module. The container outline and trolley position information are calculated and then constructed into a quay crane operation map based on the container outline and trolley position coordinates, enabling automated container loading and unloading and collision avoidance.

Description

Technical Field

[0001] The invention relates to a quay crane container contour recognition method based on improved laser features. Background Art

[0002] Ship and container contour recognition is commonly used in the process of loading and unloading containers by quay cranes at docks. It is used to construct and obtain information such as the outline of the ship cabin and container, the container position, the container height, and the position of the spreader, so as to realize the automated control of the quay crane loading and unloading operations. The traditional quay crane loading and unloading method for containers is still at the stage of visual judgment by the driver of the vehicle. Due to the position of the driver and the environmental restrictions near the operating crane equipment, the position of the container can often only be judged from a local perspective, and the operation efficiency is low, and the risk of collision is high. In recent years, there have also been some quay crane automation operation device systems based on LiDAR. However, due to the noise interference of the raw laser data and the inappropriate subsequent data processing, the system recognition accuracy is not high, and there are cases of misjudgment and false alarm of anti-collision. Summary of the Invention

[0003] In order to overcome the above-mentioned shortcomings of the prior art, the present invention proposes a quay crane container contour recognition method based on improved laser features.

[0004] The technical solution adopted by the present invention to solve the technical problem is: a method for recognizing the outline of a quay crane container based on improved laser features, comprising the following steps:

[0005] Step 1: Install a transverse laser radar and a longitudinal laser radar on the lower right side and the right side of the bottom of the trolley cab of the quay crane respectively, and install an identification plate at the front end of the quay crane trolley boom; set the quay crane coordinate origin, and the length and height of the quay crane boom are known. The center position of the identification plate is obtained by clustering the laser point data scanned by the laser radar, and then the relative coordinates of the laser radar and the trolley position are calculated from the known relative positions;

[0006] Step 2: The lateral laser radar and longitudinal laser radar scan the container 180 degrees horizontally and 180 degrees vertically, respectively, to obtain two sets of real-time 2D laser point data for the cabin and container. The two sets of real-time 2D laser point data are synchronized using the operation information of the trolley during the quay crane operation and cached for subsequent data processing.

[0007] Step 3: Segment and filter the two sets of real-time 2D laser point data to obtain the laser points in the effective areas of the cabin and container, and extract the special locations of the container, hatch cover and ship side based on feature recognition;

[0008] Step 4: Use a clustering method based on laser echo intensity characteristics to classify laser points. Use straight-line fitting to obtain the horizontal and vertical lines of the container outline. Combine and complete the contours calculated from the two sets of laser data to construct the cabin and container outlines, and finally fill in the grid map information.

[0009] Step 5: The output contour and vehicle position information are processed into a historical structure, and the vehicle point cloud data obtained by the lidar is used to optimize the vehicle position and continuously update the cabin and container map information;

[0010] Step 6: Based on the constructed cabin and container map information, provide container location information and collision avoidance strategies for the automated operation of the quay crane vehicle.

[0011] Compared with the prior art, the present invention has the following positive effects:

[0012] The improved quay crane container contour recognition method based on the laser echo intensity feature of the present invention utilizes two 2D lasers to scan the container horizontally 180 degrees and vertically 180 degrees respectively, and combines the movement of the quay crane and large and small vehicles to collect a complete operation map, and constructs the container contour and the vehicle position through the clustering method based on the laser echo intensity improvement and the laser radar odometer. The present invention proposes a quay crane container contour recognition method based on the improved laser echo intensity feature. The data acquisition module receives the container point cloud scanned by the laser sensor, and pre-processes the acquired container laser point data and inputs it into the container recognition module. After calculation, the container contour and the vehicle position information are obtained. The quay crane operation map information is constructed based on the container contour result and the vehicle position coordinate information, realizing the automated container loading and unloading and anti-collision functions. The specific advantages include:

[0013] (1) In this invention, it is proposed to use threshold segmentation and point cloud filtering algorithm based on topological relationship to preprocess the collected laser point data, which has the following advantages: ① removing invalid range data points and accurately identifying the range; ② removing laser noise points, improving recognition accuracy and reducing misjudgment;

[0014] (2) The present invention proposes a method combining improved DBSCAN clustering based on laser echo intensity characteristics and least squares fitting to accurately classify laser points. The laser echo intensity is used to help the system more accurately identify target features such as containers, hatch covers, and ship sides, and then the corresponding target contour information is obtained through linear fitting.

[0015] (3) The present invention is based on grid map construction and combines historical information for optimization and completion, thereby improving the accuracy and completeness of the container outline construction and optimizing the automated control and anti-collision strategy of the quay crane trolley;

[0016] (4) The present invention uses a lidar odometer as the vehicle positioning estimate, which can well weight the real position of the vehicle and obtain better position information, thereby further improving the accuracy of the grid map and the control accuracy of the vehicle. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The present invention will now be described by way of example with reference to the accompanying drawings, in which:

[0018] Figure 1 This is a flow chart of the quay crane container scanning system;

[0019] Figure 2 This is a schematic diagram of the quay crane operation system;

[0020] Figure 3 This is a schematic diagram of the operation in the adjacent direction;

[0021] Figure 4 It is a schematic diagram of the construction of the horizontal grid outline;

[0022] Figure 5 It is a schematic diagram of the construction of the longitudinal grid outline;

[0023] Figure 6 It is a schematic diagram of the neighborhood information of the raster map;

[0024] The reference numerals in the figure include: trolley 1, transverse laser 2, longitudinal laser 3, identification plate 4, quay crane trolley 5, container ship 6, and sea level 7. DETAILED DESCRIPTION

[0025] The present invention is a method for recognizing the outline of a quay crane container based on laser scanning. It proposes a data preprocessing technology using threshold segmentation and point cloud filtering based on topological relationships. On this basis, it proposes a clustering and grid map construction method based on improved laser echo intensity characteristics to calculate the container outline, and estimates the vehicle position through the laser radar odometer. Figure 1 The specific steps are as follows:

[0026] Step S1: Figure 2 As shown, a transverse laser 2 and a longitudinal laser 3 are installed on the lower right side and bottom right side of the cab of trolley 1, respectively. A regularly shaped blue identification plate 4 is placed at the far end of the boom of the quay crane 5, ensuring that the identification plate is within the scanning range of the longitudinal laser 3. The quay crane coordinate origin is set, the length and height of the quay crane boom are known, the scanned laser points are collected and clustered using the K-Means algorithm to obtain the center position of the identification plate, and then the relative coordinates of the laser and the trolley position are calculated based on the known coordinates.

[0027] Step S2: Acquire the operation information of the trolley 1 when the quay crane trolley 5 is in operation, and collect the real-time 2D laser point data of the cabin and containers of the container ship 6 scanned by the horizontal laser 2 and the longitudinal laser 3. Figure 3 and Figure 4 As shown, it includes a laser sensor that scans adjacent bay positions horizontally and a laser sensor that scans the current bay position vertically. The data of the two laser sensors are time-synchronized and cached for subsequent data processing;

[0028] Step S3: Segment and filter preprocess the two sets of ship container laser data collected in step S2 to obtain laser points in the effective areas of the cabin and container, and extract target features of special locations such as the container, hatch cover, and ship side based on feature recognition;

[0029] 1) Threshold segmentation is performed on the laser point data. The reference coordinate origin of the quay crane is set. The sea level, the far end of the quay crane, the height, and other positions, as well as the range of the trolley spreader, are delineated. The points at the boundary of the container are intercepted from the laser points. This method can quickly remove most interference points of non-container objects, facilitating subsequent recognition and processing.

[0030] 2) Rasterizing the segmented laser point data. Since 2D laser data often detects interference points such as raindrops and dust, and the complex shipboard environment means the laser may also scan objects such as catwalks and fences, causing fluctuations in the laser scanning results. For shipboard objects, the system only needs to focus on sufficiently large objects such as containers, hatch covers, and ship sides. Therefore, this paper proposes a point cloud filtering algorithm based on topological relationships. This method can effectively eliminate noise interference and better cope with harsh environments such as rain and snow.

[0031] Project the laser point onto the 2D grid map. Figure 1 The grid is a square with a side length of 10*10cm. The number of laser points falling in each grid is counted. According to the projection of the laser point on the X axis, the number of points in the N neighborhood of each point is counted as the weight (N is generally an empirical value set according to the number of laser points). Figure 6 For example, in the neighborhood range of N=3, the left side shows the laser data projection and the number of points, and the right side shows the weight calculation results in the neighborhood: the weight of x1 is 6, and the weight of x2 is 3. At this time, the point x1 whose weight meets the set conditions (for example, greater than 3) is updated to the historical grid map, and x2 is temporarily determined to be an interference point;

[0032] In summary, the steps for point cloud filtering and raster map updating based on neighborhood topological relationships are as follows:

[0033] i. Projecting the segmented laser data onto a local grid map, i.e., a grid map generated by projecting a frame of laser data;

[0034] ii. Count the number of laser points and map them to a topological relationship graph;

[0035] iii. Calculate the laser point weight within the N neighborhood of each laser point;

[0036] iv. Determine whether the laser point is a noise point based on the weight, and update the non-noise point to the historical grid map;

[0037] Step S4: The laser point data in step S3 are classified using a clustering method based on the laser echo intensity characteristics, and the horizontal and vertical lines of the ship box contour are obtained by linear fitting. Figure 4 and Figure 5 As shown, the contours calculated from the two sets of laser data are combined and completed to construct the cabin and container contours, and finally fill in the raster map information;

[0038] 1) Cluster the laser points within each grid using the improved DBSCAN algorithm based on laser echo intensity. DBSCAN (Density-Based Spatial Clustering of Applications with Noise) is a density-based spatial clustering algorithm. By configuring the loading neighborhood radius and the minimum number of neighborhood points, for each point P in the data set: calculate the set of points N(P) in the neighborhood of point P. If |N(P)|>=minimum number of neighborhood points, then mark point P as a core point. For each core point P, create a new cluster C and add P to cluster C. Add all points in P's neighborhood to cluster C and mark them as visited. For each point that has just been added to cluster C, if it is a core point, recursively add the unvisited points in its neighborhood to cluster C. Calculate the echo intensity of each laser point in cluster C. The laser radar echo power formula is:

[0039]

[0040] Among them, P r is the echo power, P s is the transmission power, D r is the laser radar receiving aperture, is the object surface reflectivity, α is the angle of incidence, L is the distance from the target to the lidar, θ is the system transmission rate, and μ is the atmospheric transmission rate. This shows that echo intensity is primarily related to the object surface reflectivity and distance. Based on the laser echo intensity, the points in cluster C are further categorized into plane points, corner points, and noise points (plane points are points created by the laser hitting a plane, corner points are points created by the laser hitting a corner, and noise points are points created by the laser itself or by interfering objects). Each point in cluster C is assigned a category attribute, noise points are removed, and the clustering results are ultimately output.

[0041] 2) Perform the least squares method to fit a straight line to the laser points clustered based on echo intensity. Take each grid point as a set of data points ((x_1, y_1), (x_2, y_2), ..., (x_n, y_n)) and find a straight line (y = ax + b), where a is the slope and b is the intercept, such that the sum of the squares of the deviations (errors) from all data points to the straight line is minimized.

[0042] 3) The line segments fitted from each grid are spliced ​​together to ultimately construct a complete and closed container outline. Due to occlusion or the influence of the laser angle, the outline information of the back of the container may sometimes not be scanned. To complete this missing outline information, the outline is adaptively filled. If the length of the horizontal line segment after fitting and splicing does not meet the container width, the line segment is automatically extended to the container width. If the length of the vertical line segment is less than the container height, the line segment is automatically extended to the container height. This method makes the outline more accurate;

[0043] Step S5: The output profile and laser position information are processed into a history, the lidar odometer is calculated, the vehicle position is optimized, and the ship type and container map information are continuously updated.

[0044] 1) Update the container outline information calculated from each frame of laser data into the global history construction, and update the grid map and vehicle position in real time;

[0045] 2) The LiDAR odometer obtains the car point cloud and calculates the motion of the LiDAR between two consecutive scans. The estimated motion is used to correct the distortion of the point cloud. The undistorted point cloud corrected by the odometer and the estimated pose are used to obtain the car motion estimate. The car position is weighted and optimized based on the motion estimate and the PLC information of the car. The start time and end time of the kth frame are respectively and The measured position of the wheel odometer in the corresponding time interval is P k ={P i},i=1、…、m,P k represents the wheel odometry pose set in the world coordinate system of the kth frame, P (k,i) is in the interval Inside The position at the moment. (k,s) is the kth frame The corresponding wheel odometry pose. By linear interpolation, we can get The estimated radar pose P at time (k,i) , as shown below.

[0046]

[0047] Step S6: Based on the constructed ship-container map information, provide container location information and anti-collision strategies to the automated operation of the quay crane trolley.

Claims

1. A method for recognizing the outline of a quay crane container based on improved laser features, characterized by: The steps include: Step 1: Install a transverse laser radar and a longitudinal laser radar on the lower right side and the right side of the bottom of the trolley cab of the quay crane respectively, and install an identification plate at the front end of the quay crane trolley boom; set the quay crane coordinate origin, and the length and height of the quay crane boom are known. The center position of the identification plate is obtained by clustering the laser point data scanned by the laser radar, and then the relative coordinates of the laser radar and the trolley position are calculated from the known relative positions; Step 2: The lateral laser radar and longitudinal laser radar scan the container 180 degrees horizontally and 180 degrees vertically, respectively, to obtain two sets of real-time 2D laser point data for the cabin and container. The two sets of real-time 2D laser point data are synchronized using the operation information of the trolley during the quay crane operation and cached for subsequent data processing. Step 3: Segment and filter the two sets of real-time 2D laser point data to obtain the laser points in the effective areas of the cabin and container, and extract the positions of the container, hatch cover and ship's side based on feature recognition; Step 4: Use a clustering method based on laser echo intensity characteristics to classify laser points. Use straight-line fitting to obtain the horizontal and vertical lines of the container outline. Combine and complete the contours calculated from the two sets of laser data to construct the cabin and container outlines, and finally fill in the grid map information. Step 5: The output contour and vehicle position information are processed into a historical structure, and the vehicle point cloud data obtained by the lidar is used to optimize the vehicle position and continuously update the cabin and container map information; Step 6: Based on the constructed cabin and container map information, provide container location information and collision avoidance strategies for the automated operation of the quay crane vehicle.

2. The method for recognizing the outline of a quay crane container based on improved laser features according to claim 1, characterized in that: The method for segmenting the two sets of real-time 2D laser point data described in step three is as follows: set the reference coordinate origin of the quay crane, define the sea level, use the known length and height of the quay crane boom, and the range of the trolley spreader to cut out the points of the ship container boundary from the laser points, and obtain the laser points of the effective area of ​​the cabin and container.

3. The method for recognizing the outline of a quay crane container based on improved laser features according to claim 2, characterized in that: The filtering method described in step 3 is: The first step is to project the laser points of the effective area of ​​the cabin and container onto the grid map; The second step is to count the number of laser points falling within each grid and map them to a topological relationship graph; Step 3: Calculate the laser point weight within the N neighborhood of each laser point; The fourth step is to determine whether the laser point weight is greater than the set threshold: if not, the laser point is determined to be a noise point; if so, the laser point is determined to be a non-noise point, and then the non-noise point is updated to the historical grid map.

4. The method for recognizing the outline of a quay crane container based on improved laser features according to claim 3 is characterized by: Step 4 describes a method for classifying laser points using a clustering method based on laser echo intensity features: for each laser point P in the grid, calculate the set N(P) of points in its N neighborhood; determine whether |N(P)| is greater than or equal to the set minimum number of neighborhood points. If so, mark point P as a core point; for each core point P, create a new cluster C, add P to cluster C, and add all points in P's N neighborhood to cluster C, and then mark them as visited; for each point that has just been added to cluster C, if it is a core point, recursively add the unvisited points in its neighborhood to cluster C, calculate the echo power of each laser point in cluster C, and set the category attribute for each point in cluster C according to the echo power. After removing the noise points, output the final clustering result.

5. The method for recognizing the outline of a quay crane container based on improved laser features according to claim 4, characterized in that: Calculate the echo power of each laser point according to the following formula : ， in is the transmit power, is the laser radar receiving aperture, is the reflectivity of the object surface, α is the incident angle, is the distance from the target to the lidar, is the system transmission rate, is the atmospheric transmission rate.

6. The method for recognizing the outline of a quay crane container based on improved laser features according to claim 4, characterized in that: The category attributes of laser points include plane points, corner points and noise points.

7. The method for recognizing the outline of a quay crane container based on improved laser features according to claim 4, characterized in that: The method of straight line fitting in step 4 is: take each grid point as a set of data points ((x1,y1),(x2,y2),...,(x n ,y n )), find a straight line , where a is the slope and b is the intercept, so that the sum of the squared deviations of all data points from the straight line is minimized.

8. The method for recognizing the outline of a quay crane container based on improved laser features according to claim 7, characterized in that: In step 4, when completing the outline of the container, if the length of the horizontal line segment after fitting and splicing does not meet the width of the container, the line segment is automatically extended to the width of the container. If the length of the vertical line segment is less than the height of the container, the line segment is automatically extended to the height of the container.

9. The method for recognizing the outline of a quay crane container based on improved laser features according to claim 1, characterized in that: In step 5, when optimizing the position of the car using the point cloud data obtained by the lidar, the following formula is used to calculate the estimated pose of the car: : ， in, and are the start time and end time of the kth frame respectively, and the measured pose of the wheel odometer in the corresponding time interval is , , represents the wheel odometry pose set in the world coordinate system of the kth frame, is in the interval Inside The posture of the moment, and Respectively expressed in and The wheel odometry pose at the moment.

10. The method for recognizing the outline of a quay crane container based on improved laser features according to claim 1, characterized in that: The identification plate is a blue identification plate with a regular shape.

Images