Quay crane ship box contour recognition method based on laser feature improvement

By installing transverse and longitudinal lidar on the shore bridge, combining threshold segmentation and topological relationship point cloud filtering technology, laser echo intensity is used for clustering and fitting, and the ship box profile and car position are constructed, which solves the traditional loading and unloading of the shore bridge. The identification of low accuracy and anti-collision misjudgment problems have been achieved, and more efficient automatic loading and unloading and safety control have been achieved.

CN119148158AActive Publication Date: 2024-12-17QINGDAO KAIFA XINGYUN INTELLIGENT CONTROL TECH CO LTD
View PDF 6 Cites 0 Cited by

Patent Information

Application Number
CN202411120666.0
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-15
Publication Date
2024-12-17
Estimated Expiration
2044-08-15

AI Technical Summary

Technical Problem

During the loading and unloading of traditional shore bridge boxes, due to the driver's perspective and the interference of lidar data noise, the problem of low recognition accuracy and anti-collision misjudgment and false alarms.

Method used

The 180-degree scanning is carried out using horizontal and longitudinal lidars, combined with the movement information of the shore bridge trolley, and the ship box profile and the The car location is optimized, grid map information is provided, and automated loading and unloading and anti-collision strategies are provided.

Benefits of technology

It improves the accuracy of ship box identification, reduces misjudgment, enhances the automated control and anti-collision functions of shore and bridge trolleys, and improves operating efficiency and safety.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119148158A_ABST
    Figure CN119148158A_ABST
Patent Text Reader

Abstract

The invention discloses a quay crane ship box contour recognition method based on laser feature improvement, and the method comprises the steps: carrying out the transverse 180-degree scanning and longitudinal 180-degree scanning of a ship box through two 2D lasers, and collecting a complete operation map through combining with the movement of a quay crane large vehicle and a quay crane small vehicle; a ship box contour and a trolley position are constructed through a clustering method based on laser echo intensity improvement and a laser radar odometer. Ship box point cloud scanned by a laser sensor is received through a data acquisition module, obtained ship box laser point data are preprocessed and then input into a ship box identification module, ship box contour and trolley position information are obtained through calculation, quay crane operation map information is constructed according to a ship box contour result and trolley position coordinate information, and the quay crane operation map information is displayed. Automatic container loading and unloading and anti-collision functions are achieved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention relates to a quay crane container contour recognition method based on laser feature improvement. Background Art

[0002] Ship container contour recognition is usually used in the process of loading and unloading containers by quay cranes at docks. It is used to construct and obtain information such as the cabin and container contours, container positions, container heights, and sling positions, so as to realize automated control of quay crane loading and unloading operations. The traditional quay crane loading and unloading method for containers still remains at the stage of judgment by the driver's naked eye. Due to the position of the driver and the environmental restrictions near the operating crane equipment, the position of the container can often only be judged from a local perspective, and the operation efficiency is low, and the risk of collision is high. In recent years, there have also been quay crane automation operation device systems based on LiDAR. However, due to the noise interference of the laser raw data and improper subsequent data processing, the system recognition accuracy is not high, and there are situations such as misjudgment and false alarm of collision avoidance. Summary of the invention

[0003] In order to overcome the above-mentioned shortcomings of the prior art, the present invention proposes a quay crane container contour recognition method based on improved laser features.

[0004] The technical solution adopted by the present invention to solve the technical problem is: a method for recognizing the outline of a quay crane container based on improved laser features, comprising the following steps:

[0005] Step 1: Install a transverse laser radar and a longitudinal laser radar on the lower right side and the right side of the bottom of the trolley cab of the quay crane trolley, respectively, and install an identification plate at the front end of the quay crane trolley boom; set the coordinate origin of the quay crane, and the length and height of the quay crane boom are known. The center position of the identification plate is obtained by clustering the laser point data scanned by the laser radar, and then the relative coordinates of the laser radar and the trolley position are calculated from the known relative positions;

[0006] Step 2: The lateral laser radar and the longitudinal laser radar scan the ship container 180 degrees horizontally and 180 degrees vertically respectively to obtain two sets of real-time 2D laser point data of the cabin and the container. The operation information of the trolley during the operation of the quay crane is used to synchronize the two sets of real-time 2D laser point data and cache them for subsequent data processing;

[0007] Step 3: Segment and filter the two sets of real-time 2D laser point data respectively to obtain the laser points of the effective areas of the cabin and container, and extract the special positions of the container, hatch cover and ship side according to feature recognition;

[0008] Step 4: Use a clustering method based on the laser echo intensity feature to classify the laser points, obtain the horizontal and vertical lines of the container outline through straight line fitting, combine and complete the outlines calculated from the two sets of laser data, construct the cabin and container outlines, and finally fill in the grid map information;

[0009] Step 5: The output contour and the position information of the trolley are processed into a historical structure, and the point cloud data of the trolley obtained by the laser radar is used to optimize the position of the trolley and continuously update the cabin and container map information;

[0010] Step 6: Based on the constructed cabin and container map information, provide container location information and anti-collision strategies for the automated operation of the quay crane vehicle.

[0011] Compared with the prior art, the present invention has the following positive effects:

[0012] The improved quay crane container contour recognition method based on the laser echo intensity feature of the present invention utilizes two 2D lasers to scan the container horizontally 180 degrees and vertically 180 degrees respectively, and collects a complete operation map in combination with the movement of the quay crane and large and small vehicles, and constructs the container contour and the vehicle position through a clustering method based on the laser echo intensity improvement and a laser radar odometer. The present invention proposes an improved quay crane container contour recognition method based on the laser echo intensity feature, in which the data acquisition module receives the container point cloud scanned by the laser sensor, and pre-processes the acquired container laser point data and inputs it into the container recognition module, and obtains the container contour and the vehicle position information through calculation, and constructs the quay crane operation map information according to the container contour result and the vehicle position coordinate information, thereby realizing automated container loading and unloading and anti-collision functions. Specific advantages include:

[0013] (1) In the present invention, it is proposed to use threshold segmentation and point cloud filtering algorithm based on topological relationship to pre-process the collected laser point data, which has the following advantages: ① removing invalid range data points and accurately identifying the range; ② removing laser noise points, improving recognition accuracy and reducing misjudgment;

[0014] (2) The present invention proposes a method combining improved DBSCAN clustering and least squares fitting based on laser echo intensity characteristics to accurately classify laser points, and uses laser echo intensity to help the system more accurately identify target features such as containers, hatch covers and ship sides, and then obtain the corresponding target contour information through linear fitting;

[0015] (3) The present invention is based on grid map construction and combined with historical information for optimization and completion, which improves the accuracy and completeness of the container outline construction and optimizes the automated control and anti-collision strategy of the quay crane vehicle;

[0016] (4) The present invention uses a laser radar odometer as the vehicle positioning estimate, which can well weight the real position of the vehicle and obtain better position information, thereby further improving the accuracy of the grid map and the control accuracy of the vehicle. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The present invention will be described by way of example with reference to the accompanying drawings, in which:

[0018] Figure 1 This is a flow chart of the quay crane container scanning system;

[0019] Figure 2 This is a schematic diagram of the quay crane operation system;

[0020] Figure 3 This is a schematic diagram of the operation in the adjacent direction;

[0021] Figure 4 It is a schematic diagram of the construction of the horizontal grid outline;

[0022] Figure 5 It is a schematic diagram of the construction of the longitudinal grid outline;

[0023] Figure 6 It is a schematic diagram of the neighborhood information of the raster map;

[0024] The reference numerals in the figure include: trolley 1, transverse laser 2, longitudinal laser 3, identification plate 4, quay crane trolley 5, container ship 6, sea level 7. DETAILED DESCRIPTION

[0025] The present invention is a method for recognizing the outline of a quay crane container based on laser scanning. It proposes a data preprocessing technology using threshold segmentation and point cloud filtering based on topological relationships. On this basis, it proposes a clustering and grid map construction method based on improved laser echo intensity characteristics to calculate the container outline, and estimates the position of the vehicle through a laser radar odometer. Figure 1 As shown, the specific steps are as follows:

[0026] Step S1: Figure 2 As shown, a transverse laser 2 and a longitudinal laser 3 are installed at the lower right side and the right side of the bottom of the cab of the trolley 1, respectively, and a blue identification plate 4 with a regular shape is set at the far end of the arm of the quay crane trolley 5, and it is ensured that the identification plate is within the scanning range of the longitudinal laser 3. The coordinate origin of the quay crane is set, the length and height of the quay crane arm are known, the scanned laser points are collected and the center position of the identification plate is obtained by clustering using the K-Means algorithm, so as to infer the relative coordinates of the laser and the position of the trolley according to the known coordinates;

[0027] Step S2: Acquire the operation information of the trolley 1 when the quay crane trolley 5 is operating, and collect the real-time 2D laser point data of the cabin and containers of the container ship 6 scanned by the transverse laser 2 and the longitudinal laser 3. Figure 3 and Figure 4 As shown, it includes a laser sensor for horizontally scanning adjacent bay positions and a laser sensor for vertically scanning the current bay position, and the data of the two laser sensors are time-synchronized and cached for subsequent data processing;

[0028] Step S3: Segment and filter preprocess the two groups of ship container laser data collected in step S2, obtain laser points in the effective area of ​​the cabin and container, and extract target features of special positions such as containers, hatch covers and ship sides according to feature recognition;

[0029] 1) Perform threshold segmentation on the laser point data, set the reference coordinate origin of the quay crane, define the sea level 7, the far end of the quay crane, the height, and the range of the trolley spreader, and intercept the points of the ship box boundary from the laser points. This method can quickly remove most of the interference points of non-ship box objects, which is convenient for subsequent identification and processing;

[0030] 2) Rasterize the segmented laser point data. Since 2D laser data often scans interference points such as raindrops and dust, and the ship's environment is complex, the laser will also scan catwalks, fences and other objects. These interferences also cause the laser scanning results to fluctuate. For objects on board, the system only needs to focus on sufficiently large objects such as containers, hatch covers and ship sides. Therefore, the present invention proposes a point cloud filtering algorithm based on topological relationships. This method can effectively eliminate noise interference and better cope with harsh environments such as rainy and snowy days.

[0031] Project the laser points onto a 2D grid map. Figure 1 The grid is a square with a side length of 10*10cm. The number of laser points falling in each grid is counted. According to the projection of the laser point on the X-axis, the number of points in the N neighborhood of each point is counted as the weight (N is generally an empirical value set according to the number of laser points). Figure 6 Take the neighborhood range of N=3 as an example. The left side shows the laser data projection and the number of points, and the right side shows the weight calculation result in the neighborhood: the weight of x1 is 6, and the weight of x2 is 3. At this time, the point x1 whose weight meets the set condition (for example, greater than 3) is updated to the historical grid map, and x2 is temporarily determined as an interference point;

[0032] In summary, the steps of point cloud filtering and grid map updating based on neighborhood topological relationships are as follows:

[0033] i. Projecting the segmented laser data onto a local grid map, that is, a grid map generated by projecting a frame of laser data;

[0034] ii. Count the number of laser points and map them to a topological relationship diagram;

[0035] iii. Calculate the laser point weight within the N neighborhood of each laser point;

[0036] iv. Determine whether the laser point is a noise point based on the weight, and update the non-noise point to the historical grid map;

[0037] Step S4: The laser point data in step S3 are classified using a clustering method based on the laser echo intensity characteristics, and the horizontal and vertical lines of the ship box contour are obtained by straight line fitting. Figure 4 and Figure 5 As shown, the contours calculated from the two sets of laser data are combined and completed to construct the cabin and container contours, and finally fill in the grid map information;

[0038] 1) Cluster the laser points in each grid, using the improved DBSCAN algorithm based on laser echo intensity. DBSCAN (Density-Based Spatial Clustering of Applications with Noise) is a density-based spatial clustering algorithm. By configuring the loading neighborhood radius and the minimum number of neighborhood points, for each point P in the data set: calculate the set of points N(P) in the neighborhood of point P. If |N(P)|>=minimum number of neighborhood points, mark point P as a core point. For each core point P, create a new cluster C, add P to cluster C, add all points in P's neighborhood to cluster C, and mark them as visited. For each point that has just been added to cluster C, if it is a core point, recursively add the unvisited points in its neighborhood to cluster C. Calculate the echo intensity of each laser point in cluster C. The echo power formula of the laser radar is:

[0039]

[0040] Where P r is the echo power, P s is the transmission power, D r is the laser radar receiving aperture, is the reflectivity of the object surface, α is the incident angle, L is the distance from the target to the laser radar, θ is the system transmission rate, and μ is the atmospheric transmission rate. It can be seen that the echo intensity is mainly related to the reflectivity and distance of the object surface. According to the laser echo intensity, the points in cluster C are classified into more detailed categories, including plane points, corner points, and noise points (plane points are points generated by laser hitting the plane, corner points are points generated by laser hitting the corner, and noise points are points generated by the laser itself or hitting the interference). Set the category attribute for each point in cluster C, remove the noise points and finally output the clustering results;

[0041] 2) Fit a straight line to the laser points clustered based on echo intensity using the least squares method, take each grid point as a set of data points ((x_1, y_1), (x_2, y_2), ..., (x_n, y_n)), and find a straight line (y = ax + b), where a is the slope and b is the intercept, so that the sum of the squares of the deviations (errors) from all data points to the straight line is minimized;

[0042] 3) Splice the line segments fitted by each grid to finally construct a complete and closed container outline. Due to occlusion or the influence of the laser angle, the contour information of the back of the container may not be scanned sometimes. In order to complete the missing contour information, the contour is adaptively filled. If the length of the horizontal line segment after fitting and splicing does not meet the width of the container, the line segment is automatically extended to the width of the container. If the length of the vertical line segment is less than the height of the container, the line segment is automatically extended to the height of the container. This method makes the contour more accurate;

[0043] Step S5: The output profile and laser position information are processed for history construction, the laser radar odometer is calculated, the vehicle position is optimized, and the ship type and container map information are continuously updated.

[0044] 1) Update the container outline information calculated by each frame of laser data to the global history construction, and update the grid map and the position of the car in real time;

[0045] 2) The laser radar odometer obtains the car point cloud and calculates the motion of the laser radar between two consecutive scans. The estimated motion is used to correct the distortion of the point cloud. The undistorted point cloud corrected by the odometer and the estimated position are used to obtain the motion estimate of the car. The car position is weighted and optimized based on the motion estimate combined with the car's PLC information. The start time and end time of the kth frame are respectively and The measured position of the wheel odometer in the corresponding time interval is P k = {P i},i=1、…、m,P k represents the wheel odometer pose set in the world coordinate system of the kth frame, P (k,i) is in the interval Inside The position at the moment. (k,s) is the kth frame The corresponding wheel odometer pose. By linear interpolation, we can get The estimated radar pose P at time (k,i) , as shown below.

[0046]

[0047] Step S6: Based on the constructed ship container map information, provide container location information and anti-collision strategy for the automated operation of the quay crane vehicle.

Claims

1. A method for recognizing the outline of a quay crane container based on improved laser features, characterized in that: The steps include: Step 1: Install a transverse laser radar and a longitudinal laser radar on the lower right side and the right side of the bottom of the trolley cab of the quay crane trolley, respectively, and install an identification plate at the front end of the quay crane trolley boom; set the coordinate origin of the quay crane, and the length and height of the quay crane boom are known. The center position of the identification plate is obtained by clustering the laser point data scanned by the laser radar, and then the relative coordinates of the laser radar and the trolley position are calculated from the known relative positions; Step 2: The lateral laser radar and the longitudinal laser radar scan the ship container 180 degrees horizontally and 180 degrees vertically respectively to obtain two sets of real-time 2D laser point data of the cabin and the container. The operation information of the trolley during the operation of the quay crane is used to synchronize the two sets of real-time 2D laser point data and cache them for subsequent data processing; Step 3: Segment and filter the two sets of real-time 2D laser point data respectively to obtain the laser points of the effective areas of the cabin and container, and extract the special positions of the container, hatch cover and ship side according to feature recognition; Step 4: Use a clustering method based on the laser echo intensity feature to classify the laser points, obtain the horizontal and vertical lines of the container outline through straight line fitting, combine and complete the outlines calculated from the two sets of laser data, construct the cabin and container outlines, and finally fill in the grid map information; Step 5: The output contour and the position information of the trolley are processed into a historical structure, and the point cloud data of the trolley obtained by the laser radar is used to optimize the position of the trolley and continuously update the cabin and container map information; Step 6: Based on the constructed cabin and container map information, provide container location information and anti-collision strategies for the automated operation of the quay crane vehicle.

2. According to claim 1, a method for recognizing the outline of a quay crane container based on improved laser features is characterized by: The method for segmenting the two sets of real-time 2D laser point data described in step three is: setting the reference coordinate origin of the quay crane, defining the sea level, using the known length and height of the quay crane boom, and the range of the trolley sling, cutting out the points of the ship container boundary from the laser points, and obtaining the laser points of the effective area of ​​the cabin and container.

3. The method for recognizing the outline of a quay crane container based on improved laser features according to claim 2 is characterized in that: The filtering method described in step three is: The first step is to project the laser points of the effective area of ​​the cabin and container onto the grid map; Step 2: Count the number of laser points falling within each grid and map them to a topological relationship diagram; Step 3: Calculate the laser point weights within the N neighborhood of each laser point; Step 4: Determine whether the laser point weight is greater than the set threshold: if not, the laser point is determined to be a noise point; if so, the laser point is determined to be a non-noise point, and then the non-noise point is updated to the historical grid map.

4. The method for recognizing the outline of a quay crane container based on improved laser features according to claim 3 is characterized in that: Step 4 describes a method for classifying laser points using a clustering method based on laser echo intensity features: for each laser point P in the grid, calculate the set N(P) of points in its N neighborhood; determine whether |N(P)| is greater than or equal to the set minimum number of neighborhood points, and if so, mark point P as a core point; for each core point P, create a new cluster C, add P to cluster C, and add all points in P's N neighborhood to cluster C, and then mark them as visited; for each point just added to cluster C, if it is a core point, recursively add the unvisited points in its neighborhood to cluster C, calculate the echo power of each laser point in cluster C, and set the category attribute for each point in cluster C according to the echo power, remove the noise points and output the final clustering result.

5. The method for recognizing the outline of a quay crane container based on improved laser features according to claim 4 is characterized in that: The echo power P of each laser point is calculated as follows r : Where P s is the transmission power, D r is the laser radar receiving aperture, is the reflectivity of the object surface, α is the incident angle, L is the distance from the target to the lidar, θ is the system transmission rate, and μ is the atmospheric transmission rate.

6. The method for recognizing the outline of a quay crane container based on improved laser features according to claim 4 is characterized in that: The category attributes of laser points include plane points, corner points and noise points.

7. The method for recognizing the outline of a quay crane container based on improved laser features according to claim 4 is characterized in that: The method of straight line fitting described in step 4 is: take each grid point as a set of data points ((x_1, y_1), (x_2, y_2), ..., (x_n, y_n)), find a straight line y = ax + b, where a is the slope and b is the intercept, so that the sum of the squares of the deviations of all data points to the straight line is minimized.

8. The method for recognizing the outline of a quay crane container based on improved laser features according to claim 7 is characterized in that: In step 4, when completing the outline of the ship box, if the length of the horizontal line segment after fitting and splicing does not meet the width of the container, the line segment is automatically extended to the width of the container; if the length of the vertical line segment is less than the height of the container, the line segment is automatically extended to the height of the container.

9. The method for recognizing the outline of a quay crane container based on improved laser features according to claim 1, characterized in that: In step 5, when optimizing the position of the car using the car point cloud data obtained by the laser radar, the following formula is used to calculate the estimated position P of the car: (k,i) : in, and are the start time and end time of the kth frame respectively, and the measured position of the wheel odometer in the corresponding time interval is P k = {P i }, i = 1, ..., m, P k represents the wheel odometer pose set in the world coordinate system of the kth frame, P (k,i) is in the interval Inside The position at the moment, P (k,e) and P (k,s) Respectively expressed in and The wheel odometer pose at the time.

10. The method for recognizing the outline of a quay crane container based on improved laser features according to claim 1, characterized in that: The identification plate is a blue identification plate with a regular shape.

Citation Information

Patent Citations

  • System and method for alignment and safety control of trucks under bridge cranes of container wharves

    CN104477779A

  • System for profile identification of ship under container terminal quay crane and anti-collision of lifting sling and method for profile identification of ship under container terminal quay crane

    CN105819341A

  • Trolley path planning method for remote control of automatic container shore bridge

    CN110329910A

  • Vessel control system using lidar

    KR102495879B1

  • Intelligent horizontal transportation system and method for automatic side-loading / unloading container tarminal

    US20230072997A1