A method for modeling bulk cargo gantry crane cabin and its materials based on 3D laser radar

Abstract

The present invention proposes a method for modeling a bulk cargo gantry crane cabin and materials in the cabin based on 3D laser radar, wherein a first radar is installed on the gantry crane revolving body, and a second radar with a self-stabilizing device is installed on the head of the gantry crane boom elephant trunk beam; multiple coordinate systems are established, including the gantry crane base coordinate system, the gantry crane revolving body coordinate system, the first radar coordinate system and the second radar coordinate system; the coordinate transformation matrix of the two radars in the gantry crane revolving body coordinate system is determined, thereby obtaining the positions of the two radars on the revolving body; the two radars are scanned according to the operation instructions, and the first initial point cloud and the second initial point cloud are obtained and registered, thereby obtaining the transformation matrix T 12 , and then correct the second initial point cloud, merge the first initial point cloud and the corrected second initial point cloud to obtain the final point cloud. The present invention uses a 3D laser radar with better scanning effect, improves the point cloud registration method, and realizes efficient and accurate scanning modeling that is difficult to perform using a simple scanning device.

Description

Technical Field

[0001] The present invention relates to the field of automated operation of bulk cargo gantry cranes at ports, and in particular to a modeling method of a bulk cargo gantry crane cabin and materials therein based on a 3D laser radar. Background Art

[0002] The gantry crane (gantry crane) is an important shore equipment for port operations. Its working process is to unload bulk cargo (coal, grain, etc.) from the cargo ship to the designated area on the dock through the grab, or to perform reverse loading operations. Due to the perspective of the gantry crane driver, it is difficult to grasp the status of the materials in the cabin during loading operations. Generally, it is necessary to install a camera above the grab to increase the driver's field of view; or arrange a conductor on the cargo ship to guide the driver to perform operations; these two methods obviously require manual operation.

[0003] With the increasing demand for automated loading and unloading of gantry cranes, it is necessary to use scanning equipment to accurately model cargo ships and materials, so as to obtain information such as the location of cabin hatches and material status, and transmit it to the control system to control the gantry cranes for automated operations.

[0004] The structure of the gantry crane mainly includes the base at the bottom and the rotating body at the top. The base part can move linearly along the track, the rotating body part can rotate 360° around the rotation center, and its arm part is a four-bar linkage mechanism that can be radially extended and retracted.

[0005] In terms of bulk carrier scanning modeling, the current mainstream technical solution is to install a single-line 2D laser radar with a pan-tilt head at the appropriate position of the gantry crane cab and the head of the trunk beam. While scanning the cargo ship's operating area, the radar rotates a certain angle along the pan-tilt axis. The radar's own rotation scan plus the movement around the pan-tilt head form a 3D point cloud of the area of ​​interest. The two point clouds are then merged to form a complete 3D point cloud from different perspectives.

[0006] This scanning method has the following main disadvantages:

[0007] 1. The scanning device has a complex structure. It consists of a laser radar and a servo drive system. The large number of mechanical and electrical components makes subsequent maintenance inconvenient and reduces the stability of the system.

[0008] 2. Low scanning efficiency. The resolution of the point cloud on the PTZ rotation axis is determined by the speed of the drive system. The faster the speed, the lower the resolution. In order to obtain higher accuracy, the speed of the drive system needs to be lower, resulting in low scanning efficiency, thus affecting work efficiency.

[0009] 3. The modeling results are not accurate enough. On the one hand, the single-line 2D laser radar may vibrate or shake during the scanning process, and the longer the scanning process, the more obvious the impact, which causes the point cloud obtained by the elephant trunk head radar to be distorted, affecting the modeling results; on the other hand, when merging point clouds, usually only the most widely used point cloud registration algorithm, namely the iterative closest point algorithm, is used. However, during the scanning and modeling process of the cargo ship operating area, the positions of each radar are not all fixed, and the point cloud effect obtained by the scanning is unstable and the overlap rate is low. Therefore, the iterative closest point algorithm is not suitable for the scanning and modeling process of the cargo ship operating area. This results in the point cloud obtained by the radar scanning not being accurate enough when merging and registering, which makes the modeling results not accurate enough.

[0010] Therefore, there is currently no generally applicable method that can solve the problem that it is difficult to perform scanning modeling efficiently and accurately using a simple scanning device. Summary of the invention

[0011] In view of this, the present invention proposes a bulk cargo gantry crane cabin and cabin material modeling method based on 3D laser radar, which is used to solve the problem that it is difficult to perform efficient and accurate scanning and modeling using a simple scanning device.

[0012] The technical solution of the present invention is achieved in this way:

[0013] In a first aspect, the present invention discloses a method for modeling a bulk cargo gantry crane cabin and materials therein based on a 3D laser radar, the method comprising:

[0014] S1, the first radar is installed on the gantry crane's rotating body, and the second radar with a self-stabilizing device is installed on the head of the gantry crane's boom. The second radar is affected by gravity, and the scanning direction of the second radar always remains vertically downward during the gantry crane operation; the first radar and the second radar are both 3D laser radars;

[0015] S2, establishing multiple coordinate systems, including the gantry crane base coordinate system, the gantry crane rotary body coordinate system, the first radar coordinate system and the second radar coordinate system; determining the coordinate transformation matrix of the first radar and the second radar in the gantry crane rotary body coordinate system, obtaining the precise position of the first radar on the rotary body and the estimated position of the second radar on the rotary body, and completing the preliminary calibration of the first radar and the second radar;

[0016] S3, according to the operation instruction, the first radar and the second radar simultaneously scan the operation area to obtain a first initial point cloud and a second initial point cloud;

[0017] S4, registering the first initial point cloud with the second initial point cloud to obtain a transformation matrix T 12, and then correct the second initial point cloud, merge the first initial point cloud and the corrected second initial point cloud to obtain the final point cloud.

[0018] The present invention uses the above method to quickly scan and model the cargo ship cabin and bulk materials in the cabin within the gantry crane operating area using two 3D laser radars installed on the gantry crane's rotating body and the trunk beam, thereby forming a 3D point cloud with a small dead zone and high precision. The 3D laser radar does not require the addition of an external auxiliary motion device, has a simple structure, and scans quickly. It can generate a point cloud that meets the resolution and density requirements in a short time, has high scanning efficiency, and obtains a point cloud with low distortion. The point clouds obtained by the two radar scans are accurately aligned to obtain a more accurate modeling result.

[0019] On the basis of the above technical solution, preferably, in step S1, installing the first radar on the gantry crane revolving body specifically includes:

[0020] The first radar is installed near the pitch boom where the scanning operation area is not blocked.

[0021] On the basis of the above technical solution, preferably, step S2 specifically includes:

[0022] S2-1, establish multiple coordinate systems, including a gantry crane base coordinate system, a gantry crane rotary body coordinate system, a first radar coordinate system, and a second radar coordinate system; the gantry crane base coordinate system takes the gantry crane rotary center as the origin, the Z axis perpendicular to the horizontal plane upwards, the Y axis perpendicular to the dock shoreline pointing to the water side as the Y axis, and the X axis direction is determined according to the right-hand rule; the origin, Z axis and base coordinate system of the gantry crane rotary body coordinate system coincide, the Y axis always points to the boom direction, and the X axis direction is determined according to the right-hand rule;

[0023] S2-2, the door crane rotary body coordinate system and the door crane base differ only by a rotation angle θ, the value of which is obtained by a sensor including an absolute encoder installed on the rotary body. The transformation matrix of the value of the rotation angle θ is denoted as T WR ;

[0024] S2-3, the first radar coordinate system and the second radar coordinate system have been clearly defined inside the product. By preliminarily calibrating the first radar and the second radar, the coordinate transformation matrix of the first radar and the second radar in the gantry crane rotating body coordinate system is obtained to obtain the precise position of the first radar on the rotating body and the estimated position of the second radar on the rotating body.

[0025] Based on the above technical solution, preferably, step S2-3 specifically includes:

[0026] S2-3-1, use the inherent objects on the dock to calibrate the external parameters of the first radar, and determine the coordinate transformation matrix T of the first radar relative to the gantry crane rotating body coordinate system RL1 =[R t];

[0027] S2-3-2, roughly determine the coordinate transformation matrix T of the second radar and the gantry crane rotating body coordinate system based on the sensor including the absolute value encoder on the gantry crane RL2 .

[0028] Based on the above technical solution, preferably, step S2-3-1 specifically includes:

[0029] The first radar performs a 3D scan on the dock scene to obtain a calibration point cloud, and extracts features from the calibration point cloud for calibration;

[0030] Extract the mutually perpendicular ground and electrical room plane under the gantry crane from the calibration point cloud, determine the three rotational degrees of freedom, and obtain the coordinate transformation matrix T RL1 The rotation part is denoted as R; according to the mechanical size of the gantry crane, multiple points with known coordinates on the gantry crane structure are selected in the calibration point cloud. According to their relative relationship in the calibration point cloud and the actual scene, the average value P1 of the selected multiple points is calculated, and the corresponding point P2 of P1 in the actual scene is obtained through rotation and translation transformation. The translational degree of freedom t is calculated using the mean point:

[0031] P2=R*P1+t

[0032] Get the translation part t of the coordinate transformation matrix; determine the coordinate transformation matrix T of the first radar relative to the gantry crane rotating body coordinate system RL1 =[R t].

[0033] The present invention determines the precise position of the first radar on the rotating body and the estimated position of the second radar on the rotating body through the above method, which is convenient for subsequent alignment and correction of the second radar, and the precise position of the second radar on the rotating body is obtained, so that the point clouds obtained by scanning the first radar and the second radar can be accurately matched.

[0034] On the basis of the above technical solution, preferably, step S3 specifically includes:

[0035] The first radar and the second radar scan the working area simultaneously, obtain a point cloud respectively, and transform the two point clouds into the gantry crane base coordinate system according to the coordinate transformation matrix chain of the first radar and the second radar; suppose any coordinate in the two point clouds is P L1 , P L2 , and their coordinates in the gantry crane base coordinate system are P W1 , P W2 , then:

[0036] PW1 =T WR *T RL1 *P L1

[0037] P W2 =T WR *T RL2 *P L2

[0038] The two point clouds P in the gantry crane base coordinate system are called W1 and P W2 They are the first initial point cloud and the second initial point cloud respectively.

[0039] In the present invention, through the above method, the first radar and the second radar respectively scan the working area to obtain point clouds, preparing for subsequent modeling.

[0040] On the basis of the above technical solution, preferably, step S4 specifically includes:

[0041] S4-1, preprocessing the first initial point cloud and the second initial point cloud, wherein the preprocessing includes removing irrelevant point clouds by straight-through filtering and performing radius filtering to remove noise points, thereby obtaining two corresponding filtered point clouds; and continuing to execute step S4-2;

[0042] S4-2, downsampling the two filtered point clouds through voxel filtering to obtain a first sparse point cloud and a second sparse point cloud with appropriate point cloud scales; and continuing to execute step S4-3;

[0043] S4-3, coarsely aligning the first sparse point cloud and the second sparse point cloud; the coarse alignment includes, according to the alignment algorithm module provided by the PCL library, taking the first sparse point cloud as the target point cloud and the second sparse point cloud as the source point cloud, calculating the FPFH features of the two point clouds, setting parameters, and performing alignment calculation on the two point clouds to obtain the coordinate transformation matrix T1 from the second sparse point cloud to the first sparse point cloud; and performing coordinate transformation on the second sparse point cloud according to T1 to obtain an updated second initial point cloud; and continuing to execute step S4-4;

[0044] S4-4, performing through-filtering and radius filtering again on the first sparse point cloud and the updated second initial point cloud, modifying parameters during through-filtering, reducing the X-direction range, and canceling the relaxation of the threshold; and continuing to execute step S4-5;

[0045] S4-5, downsample the two point clouds and set the filter edge length to l fin ; Continue to execute step S4-6;

[0046] S4-6. Perform ICP fine registration on the two point clouds. The fine registration includes using the first sparse point cloud as the target point cloud, the updated second initial point cloud as the source point cloud, setting parameters, performing ICP registration calculation to obtain the coordinate transformation matrix T2, and performing coordinate transformation on the updated second initial point cloud according to T2 to obtain the second initial point cloud in the final portal base coordinate system. Then continue to execute step S4-7.

[0047] S4-7. Obtain the transformation matrix T according to the coordinate transformation matrices T1 and T2 12 = T1 * T2, and then correct the second initial point cloud in the portal base coordinate system: P W1 = T 12 * P W2 , to obtain the corrected second initial point cloud. Merge the first initial point cloud and the corrected second initial point cloud to obtain the final point cloud.

[0048] Through the above method, the present invention processes the point clouds of the operation area obtained by the first radar and the second radar, namely the first initial point cloud and the second initial point cloud, and performs precise registration, so as to obtain a more accurate modeling result. Since the position of the first radar is fixed and the position of the second radar is dynamically changing, the second initial point cloud is registered and corrected to determine the accurate position of the second radar on the rotating body, so as to obtain the second initial point cloud in the portal base coordinate system, which is convenient for merging with the first initial point cloud.

[0049] On the basis of the above technical solutions, preferably, step S4-1 specifically includes:

[0050] Set the initial filter side length to l max , and the expected point cloud scale to count min , and perform direct filtering processing;

[0051] For the first initial point cloud, in the portal coordinate system, set the Y value of the dock coastline to y l , and filter out the first initial point cloud within the range of -50.0 < x < 50.0 and y l < y < y l + 50.0;

[0052] For the second initial point cloud, in the portal coordinate system, set the Y value of the dock coastline to y l , relax the filtering threshold, and filter out the second initial point cloud within the range of -50.0 - δ1 < x < 50.0 + δ1 and y l - δ1 < y < y l + 50.0 + δ1 with δ1 = 3.0;

[0053] The first initial point cloud and the second initial point cloud that have completed the through filtering are subjected to radius filtering to remove noise points, thereby obtaining two corresponding filtered point clouds.

[0054] The present invention pre-processes the first initial point cloud and the second initial point cloud by the above method to remove irrelevant point clouds. When the second initial point cloud is subjected to the first straight-through filtering process, due to the coordinate transformation matrix T of the second radar RL2 It is not an exact value, so the filtering threshold is relaxed to a certain extent.

[0055] Based on the above technical solution, preferably, step S4-2 specifically includes:

[0056] For the first initial point cloud, with side length l cur = l max Perform voxel filtering on it, and the resulting point cloud size is count cur , perform the following judgment:

[0057] i. If count cur ≥ count min , indicating that the point cloud is already dense, the voxel filtering process ends, and the final point cloud size is count cur ;

[0058] ii. Otherwise, it means that the current point cloud is too sparse, so the side length is halved, i.e. l cur = l cur / 2.0, perform voxel filtering again;

[0059] iii. If the result count cur <count min , continue to execute step ii, and repeat until the voxel filtering process is completed; otherwise, it means that there is a suitable filtering edge length l∈(l cur ~2l cur ), perform voxel filtering based on this side length, and obtain a scale of count min The first sparse point cloud, record the filter side length as l fin ;

[0060] For the second sparse point cloud, set the filter edge length to l fin , perform voxel filtering to obtain the second sparse point cloud.

[0061] The present invention uses the above method to obtain two point clouds with basically the same uniformity and moderate density, which is convenient for subsequent rough alignment.

[0062] In a second aspect of the present invention, a computer-readable storage medium is disclosed, on which is stored a program for a method for modeling a bulk cargo gantry crane cabin and materials therein based on a 3D laser radar. When the program for modeling a bulk cargo gantry crane cabin and materials therein based on a 3D laser radar is executed, the method for modeling a bulk cargo gantry crane cabin and materials therein based on a 3D laser radar as described in the first aspect of the present invention is implemented.

[0063] The method for modeling a bulk cargo gantry crane cabin and materials therein based on 3D laser radar of the present invention has the following beneficial effects compared with the prior art:

[0064] (1) By using two 3D laser radars installed on the gantry crane's rotating body and trunk, the cargo ship's cabin and bulk cargo materials in the gantry crane's operating area are quickly scanned and modeled to form a 3D point cloud with small dead zone and high accuracy. The 3D laser radar does not require the addition of external auxiliary motion devices, has a simple structure, can generate point clouds that meet the resolution and density requirements in a short time, and has high scanning efficiency.

[0065] (2) For the fixed first radar and the second radar whose position changes dynamically, their coordinate transformation matrices relative to the gantry crane's rotating body coordinate system are determined respectively, so that the coordinate transformation matrices of the first radar and the second radar in the gantry crane's rotating body coordinate system can be determined, and then the relative positions of the two radars in the point cloud obtained by scanning the operating area in the gantry crane's base coordinate system can be determined;

[0066] (3) The point clouds obtained by scanning the working area by the two radars, namely the first initial point cloud and the second initial point cloud, are accurately aligned, wherein the second initial point cloud with dynamically changing position is aligned and corrected according to the first initial point cloud, and the precise position of the second radar on the rotating body is determined to obtain the second initial point cloud in the gantry crane base coordinate system. Finally, the second initial point cloud in the gantry crane base coordinate system is merged with the first initial point cloud to obtain a more accurate modeling result. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0068] Figure 1 This is a workflow diagram of a method for modeling a bulk cargo gantry crane cabin and materials therein based on 3D laser radar according to the present invention;

[0069] Figure 2This is a schematic diagram of the gantry crane structure and radar installation position in a bulk cargo gantry crane cabin and cabin material modeling method based on 3D laser radar of the present invention;

[0070] Figure 3 It is a schematic diagram of the gantry crane coordinate system in a method for modeling a bulk cargo gantry crane cabin and materials in the cabin based on 3D laser radar of the present invention;

[0071] Figure 4 This is a point cloud processing flow chart of a method for modeling a bulk cargo gantry crane cabin and materials in the cabin based on 3D laser radar in the present invention;

[0072] Figure 5 The present invention is a flowchart of the registration process in a method for modeling a bulk cargo gantry crane cabin and materials therein based on 3D laser radar. DETAILED DESCRIPTION

[0073] The following will be combined with the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0074] Example

[0075] The present invention is a bulk cargo gantry crane cabin and cabin material modeling method based on 3D laser radar. Figure 1 , the processing steps are as follows:

[0076] The first step is to install the first radar on the gantry crane's rotating body, and install the second radar with a self-stabilizing device on the head of the gantry crane's boom. The second radar is affected by gravity. During the operation of the gantry crane, the second radar's scanning direction is always kept vertically downward; both the first radar and the second radar are 3D laser radars. Go to the second step.

[0077] Based on the above solution, preferably, the first radar is installed near the pitch boom where the scanning operation area is not blocked.

[0078] The second step is to establish multiple coordinate systems, including the gantry crane base coordinate system (global coordinate system), the gantry crane rotary body coordinate system, the first radar coordinate system and the second radar coordinate system; determine the coordinate transformation matrix of the first radar and the second radar in the gantry crane rotary body coordinate system, obtain the precise position of the first radar on the rotary body and the estimated position of the second radar on the rotary body, and complete the preliminary calibration of the first radar and the second radar. Go to the third step.

[0079] Based on the above solution, preferably, the second step specifically includes:

[0080] S2-1, establish multiple coordinate systems, including a gantry crane base coordinate system, a gantry crane rotary body coordinate system, a first radar coordinate system, and a second radar coordinate system; the gantry crane base coordinate system takes the gantry crane rotary center as the origin, the Z axis perpendicular to the horizontal plane upwards, the Y axis perpendicular to the dock shoreline pointing to the water side as the Y axis, and the X axis direction is determined according to the right-hand rule; the origin, Z axis and base coordinate system of the gantry crane rotary body coordinate system coincide, the Y axis always points to the boom direction, and the X axis direction is determined according to the right-hand rule;

[0081] S2-2, the door crane rotary body coordinate system and the door crane base differ only by a rotation angle θ, the value of which is obtained by a sensor including an absolute encoder installed on the rotary body. The transformation matrix of the value of the rotation angle θ is denoted as T WR ;

[0082] S2-3, the first radar coordinate system and the second radar coordinate system have been clearly defined inside the product, but the precise positions of the first radar and the second radar on the rotating body are unknown, so it is necessary to determine the coordinate transformation matrix of the two radars in the rotating body coordinate system by a suitable method; by preliminarily calibrating the first radar and the second radar, the coordinate transformation matrix of the first radar and the second radar in the gantry crane rotating body coordinate system is obtained to obtain the precise position of the first radar on the rotating body and the estimated position of the second radar on the rotating body.

[0083] Based on the above solution, preferably, step S2-3 specifically includes:

[0084] S2-3-1, due to the conditions at the wharf, the traditional calibration method using calibration objects such as calibration plates is not feasible. Therefore, the external parameters of the first radar are calibrated using the inherent objects on the wharf to determine the coordinate transformation matrix T of the first radar relative to the gantry crane's rotating body coordinate system. RL1 =[R t];

[0085] S2-3-2, the second radar is installed at the head of the trunk beam, and its position changes with the radial extension and contraction of the four-link arm. Therefore, the coordinate transformation matrix of the radar relative to the gantry crane's rotating body coordinate system is dynamically changing, and its precise value cannot be obtained through a single calibration, and needs to be determined through a registration algorithm; the coordinate transformation matrix T of the second radar and the gantry crane's rotating body coordinate system is roughly determined based on the sensor including the absolute value encoder on the gantry crane. RL2 .

[0086] Based on the above solution, preferably, step S2-3-1 specifically includes:

[0087] The first radar performs a 3D scan on the dock scene to obtain a calibration point cloud, and extracts features from the calibration point cloud for calibration;

[0088] The mutually perpendicular ground and electrical room planes under the gantry crane are extracted from the calibration point cloud, and the three rotational degrees of freedom are determined, thereby obtaining the coordinate transformation matrix T RL1 The rotation part is denoted as R; according to the mechanical size of the gantry crane, multiple points with known coordinates on the gantry crane structure are selected in the calibration point cloud. According to their relative relationship in the calibration point cloud and the actual scene, the average value P1 of the selected multiple points is calculated, and the corresponding point P2 of P1 in the actual scene is obtained through rotation and translation transformation. The translational degree of freedom t is calculated using the mean point:

[0089] P2=R*P1+t

[0090] Thus, the translation part t of the coordinate transformation matrix is ​​obtained; the coordinate transformation matrix T of the first radar relative to the gantry crane rotating body coordinate system is determined RL1 =[R t].

[0091] In the third step, according to the operation instruction, the first radar and the second radar simultaneously scan the operation area to obtain a first initial point cloud and a second initial point cloud.

[0092] Based on the above scheme, preferably, the third step specifically includes:

[0093] The first radar and the second radar scan the working area simultaneously, obtain a point cloud respectively, and transform the two point clouds into the global coordinate system of the gantry crane according to the coordinate transformation matrix chain of the first radar and the second radar; suppose any coordinate in the two point clouds is P L1 , P L2 , whose coordinates in the global coordinate system of the gantry crane are P W1 , P W2 , then:

[0094] P W1 =T WR *T RL1 *P L1

[0095] P W2 =T WR *T RL2 *P L2

[0096] The two point clouds P in the global coordinate system are called W1 and P W2 They are the first initial point cloud and the second initial point cloud respectively.

[0097] Due to T RL2 It is not an exact value. The two transformed point clouds cannot be directly merged. It is necessary to align the first initial point cloud with the second initial point cloud to obtain the transformation matrix T between the first initial point cloud and the second initial point cloud. 12 .

[0098] Step 4: Register the first initial point cloud and the second initial point cloud to obtain a transformation matrix T 12 , then correct the second initial point cloud to obtain the accurate position of the second radar on the rotating body, and merge the first initial point cloud and the corrected second initial point cloud to obtain the final point cloud.

[0099] Based on the above solution, preferably, Step 4 specifically includes:

[0100] S4-1: Preprocess the first initial point cloud and the second initial point cloud. The preprocessing includes removing irrelevant point clouds from the first initial point cloud and the second initial point cloud through a pass-through filter method, and performing radius filtering to remove noise points to obtain two corresponding filtered point clouds; then continue to execute Step S4-2;

[0101] S4-2: To speed up the processing speed of subsequent algorithms and improve the geometric distribution consistency of the point cloud, perform downsampling on the two filtered point clouds through voxel filtering to obtain a first sparse point cloud and a second sparse point cloud with appropriate point cloud scales; then continue to execute Step S4-3;

[0102] S4-3: Coarsely register the first sparse point cloud and the second sparse point cloud processed in Step S4-2. The coarse registration includes, according to the registration algorithm module provided by the PCL library, using the first sparse point cloud as the target point cloud and the second sparse point cloud as the source point cloud, calculating the FPFH features of the two point clouds, setting parameters, and performing registration calculation on the two point clouds to obtain the coordinate transformation matrix T1 from the second sparse point cloud to the first sparse point cloud; and perform coordinate transformation on the second sparse point cloud according to T1 to obtain the updated second initial point cloud; then continue to execute Step S4-4;

[0103] S4-4: Perform pass-through filtering and radius filtering on the first sparse point cloud and the updated second initial point cloud again. When performing pass-through filtering, modify the parameters to reduce the X-direction range and cancel the relaxation of the threshold, and filter out the point clouds within the range of -25.0 < x < 25.0 and y l < y < y l < y + 50.0 for both the first sparse point cloud and the updated second initial point cloud; then continue to execute Step S4-5;

[0104] S4-5: Perform downsampling on the two point clouds, and still set the filter side length to l fin ; then continue to execute Step S4-6;

[0105] S4-6. Perform ICP fine registration on the two point clouds. The fine registration includes using the first sparse point cloud as the target point cloud, the updated second initial point cloud as the source point cloud, setting parameters, performing ICP registration calculation to obtain the coordinate transformation matrix T2, and performing coordinate transformation on the updated second initial point cloud according to T2 to obtain the second initial point cloud in the final global coordinate system; then continue to execute step S4-7;

[0106] S4-7. Obtain the transformation matrix T according to the coordinate transformation matrices T1 and T2 12 = T1 * T2, and then correct the second initial point cloud in the global coordinate system: P W1 = T 12 * P W2 , to obtain the corrected second initial point cloud; merge the first initial point cloud and the corrected second initial point cloud to obtain the final point cloud.

[0107] Based on the above solution, preferably, step S4-1 specifically includes:

[0108] Set the initial filtering side length to l max , the expected point cloud scale to count min , and perform直通filtering processing;

[0109] For the first initial point cloud, in the gantry coordinate system, set the Y value of the dock coastline to y l , the Y value is determined, and filter out the first initial point cloud within the range of -50.0 < x < 50.0 and y l < y < y l + 50.0;

[0110] For the second initial point cloud, in the gantry coordinate system, set the Y value of the dock coastline to y l , the Y value is determined, and the coordinate transformation matrix T of the second radar RL2 is not an exact value. Therefore, relax the filtering threshold and filter out the second initial point cloud within the range of -50.0 - δ1 < x < 50.0 + δ1 and y l - δ1 < y < y l + 50.0 + δ1 with δ1 = 3.0;

[0111] Perform radius filtering on the first initial point cloud and the second initial point cloud that have completed直通filtering to remove noise points and obtain the corresponding two filtered point clouds.

[0112] Based on the above solution, preferably, step S4-2 specifically includes:

[0113] For the first initial point cloud, with the side length l cur = l maxPerform voxel filtering on it, and the resulting point cloud size is count cur , perform the following judgment:

[0114] i. If count cur ≥ count min , indicating that the point cloud is already dense, the voxel filtering process ends, and the final point cloud size is count cur ;

[0115] ii. Otherwise, it means that the current point cloud is too sparse, so the side length is halved, i.e. l cur = l cur / 2.0, perform voxel filtering again;

[0116] iii. If the result count cur <count min , continue to execute step ii, and repeat until the voxel filtering process is completed; otherwise, it means that there is a suitable filtering edge length l∈(l cur ~2l cur ), perform voxel filtering based on this side length, and obtain a scale of count min Specifically, by using the idea of ​​dichotomy, the above interval is continuously narrowed through cyclic iteration, and the filter side length is finally determined. The filter side length at this time is recorded as l fin , get the first sparse point cloud;

[0117] For the second sparse point cloud, set the filter edge length to l fin , perform voxel filtering to obtain the second sparse point cloud.

[0118] After this step, the two sparse point clouds obtained have basically the same uniformity and moderate density.

[0119] The present invention adopts 3D laser radar, which can realize rapid modeling of scenes with high efficiency, low distortion of point cloud and high authenticity. The encoder and other sensors on the equipment are used to obtain the initial transformation matrix of the radar at the bridge of the elephant's trunk, and then the precise position of the second radar is determined through a coarse registration and ICP fine registration process, which improves the success probability of the registration algorithm on the one hand and the accuracy of the registration result on the other hand. The point cloud finally formed has a small dead zone and high quality, which provides a strong guarantee for the subsequent algorithm input.

[0120] The present invention also discloses a computer-readable storage medium, on which is stored a program for a method for modeling a bulk cargo gantry crane cabin and materials therein based on a 3D laser radar. When the program for modeling a bulk cargo gantry crane cabin and materials therein based on a 3D laser radar is executed, a method for modeling a bulk cargo gantry crane cabin and materials therein based on a 3D laser radar as described in an embodiment of the present invention is implemented.

[0121] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for modeling bulk cargo gantry crane cabin and materials in the cabin based on 3D laser radar, characterized in that: The following steps are involved: S1, the first radar is installed on the gantry crane's rotating body, and the second radar with a self-stabilizing device is installed on the head of the gantry crane's boom. The second radar is affected by gravity, and the scanning direction of the second radar always remains vertically downward during the gantry crane operation; the first radar and the second radar are both 3D laser radars; S2, establishing multiple coordinate systems, including the gantry crane base coordinate system, the gantry crane rotary body coordinate system, the first radar coordinate system and the second radar coordinate system; determining the coordinate transformation matrix of the first radar and the second radar in the gantry crane rotary body coordinate system, obtaining the precise position of the first radar on the rotary body and the estimated position of the second radar on the rotary body, and completing the preliminary calibration of the first radar and the second radar; S3, according to the operation instruction, the first radar and the second radar simultaneously scan the operation area to obtain a first initial point cloud and a second initial point cloud; S4, registering the first initial point cloud with the second initial point cloud to obtain a transformation matrix T 12 , then correct the second initial point cloud, merge the first initial point cloud and the corrected second initial point cloud to obtain the final point cloud; The step S2 specifically includes: S2-1, establish multiple coordinate systems, including a gantry crane base coordinate system, a gantry crane rotary body coordinate system, a first radar coordinate system, and a second radar coordinate system; the gantry crane base coordinate system takes the gantry crane rotary center as the origin, the Z axis perpendicular to the horizontal plane upwards, the Y axis perpendicular to the dock shoreline pointing to the water side as the Y axis, and the X axis direction is determined according to the right-hand rule; the origin, Z axis and base coordinate system of the gantry crane rotary body coordinate system coincide, the Y axis always points to the boom direction, and the X axis direction is determined according to the right-hand rule; S2-2, the door crane rotary body coordinate system and the door crane base differ only by a rotation angle θ, the value of which is obtained by a sensor including an absolute encoder installed on the rotary body. The transformation matrix of the value of the rotation angle θ is denoted as T WR ; S2-3, the first radar coordinate system and the second radar coordinate system have been clearly defined within the product. By preliminarily calibrating the first radar and the second radar, the coordinate transformation matrix of the first radar and the second radar in the gantry crane rotating body coordinate system is obtained to obtain the precise positions of the first radar and the second radar on the rotating body and the estimated positions of the second radar on the rotating body; The step S2-3 specifically includes: S2-3-1, use the inherent objects on the dock to calibrate the external parameters of the first radar, and determine the coordinate transformation matrix T of the first radar relative to the gantry crane rotating body coordinate system RL1 =[R t]; S2-3-2, roughly determine the coordinate transformation matrix T of the second radar and the gantry crane rotating body coordinate system based on the sensor including the absolute value encoder on the gantry crane RL2 ; The step S2-3-1 specifically includes: The first radar performs a 3D scan on the dock scene to obtain a calibration point cloud, and extracts features from the calibration point cloud for calibration; Extract the mutually perpendicular ground and electrical room plane under the gantry crane from the calibration point cloud, determine the three rotational degrees of freedom, and obtain the coordinate transformation matrix T RL1 The rotation part is denoted as R; according to the mechanical size of the gantry crane, multiple points with known coordinates on the gantry crane structure are selected in the calibration point cloud. According to their relative relationship in the calibration point cloud and the actual scene, the average value P1 of the selected multiple points is calculated, and the corresponding point P2 of P1 in the actual scene is obtained through rotation and translation transformation. The translational degree of freedom t is calculated using the mean point: P2=R*P1+t Get the translation part t of the coordinate transformation matrix; determine the coordinate transformation matrix T of the first radar relative to the gantry crane rotating body coordinate system RL1 =[R t]; The step S3 specifically includes: The first radar and the second radar scan the working area simultaneously, obtain a point cloud respectively, and transform the two point clouds into the gantry crane base coordinate system according to the coordinate transformation matrix chain of the first radar and the second radar; suppose any coordinate in the two point clouds is P L1 , P L2 , and their coordinates in the gantry crane base coordinate system are P W1 , P W2 , then: P W1 =T WR *T RL1 *P L1 P W2 =T WR *T RL2 *P L2 The two point clouds P in the gantry crane base coordinate system are called W1 and P W2 They are the first initial point cloud and the second initial point cloud respectively; The step S4 specifically includes: S4-1, preprocessing the first initial point cloud and the second initial point cloud, wherein the preprocessing includes removing irrelevant point clouds by straight-through filtering and performing radius filtering to remove noise points, thereby obtaining two corresponding filtered point clouds; and continuing to execute step S4-2; S4-2, downsampling the two filtered point clouds through voxel filtering to obtain a first sparse point cloud and a second sparse point cloud with appropriate point cloud scales; and continuing to execute step S4-3; S4-3, coarsely aligning the first sparse point cloud and the second sparse point cloud; the coarse alignment includes, according to the alignment algorithm module provided by the PCL library, taking the first sparse point cloud as the target point cloud and the second sparse point cloud as the source point cloud, calculating the FPFH features of the two point clouds, setting parameters, and performing alignment calculation on the two point clouds to obtain the coordinate transformation matrix T1 from the second sparse point cloud to the first sparse point cloud; and performing coordinate transformation on the second sparse point cloud according to T1 to obtain an updated second initial point cloud; and continuing to execute step S4-4; S4-4, performing through-filtering and radius filtering again on the first sparse point cloud and the updated second initial point cloud, modifying parameters during through-filtering, reducing the X-direction range, and canceling the relaxation of the threshold; and continuing to execute step S4-5; S4-5, downsample the two point clouds and set the filter edge length to l fin ; Continue to execute step S4-6; S4-6, performing ICP fine registration on the two point clouds; the coarse registration includes taking the first sparse point cloud as the target point cloud, the updated second initial point cloud as the source point cloud, setting parameters, performing ICP registration calculation, and obtaining a coordinate transformation matrix T2; and performing coordinate transformation on the updated second initial point cloud according to T2 to obtain a second initial point cloud in the final gantry crane base coordinate system; and continuing to execute step S4-7; S4-7, obtain the transformation matrix T according to the coordinate transformation matrices T1 and T2 12 =T1*T2, and then calibrate the second initial point cloud in the gantry crane base coordinate system: P W1 =T 12 *P W2 , obtain the corrected second initial point cloud; merge the first initial point cloud and the corrected second initial point cloud to obtain the final point cloud.

2. A method for modeling a bulk cargo gantry crane cabin and its materials based on 3D laser radar as claimed in claim 1, characterized in that: In the step S1, installing the first radar on the gantry crane revolving body specifically includes: The first radar is installed near the pitch boom where the scanning operation area is not blocked.

3. A method for modeling a bulk cargo gantry crane cabin and materials therein based on 3D laser radar as claimed in claim 1, characterized in that: The step S4-1 specifically includes: Set the initial filter edge length to l max , the expected point cloud size is count min , perform through filtering; For the first initial point cloud, in the portal crane coordinate system, set the Y value of the quay shoreline to y l , filter out the first initial point cloud within the range of -50.0 < x < 50.0 and y l < y < y l + 50.0; For the second initial point cloud, in the portal crane coordinate system, set the Y value of the quay shoreline to y l , relax the filtering threshold, and select the second initial point cloud within the range of -50.0 - δ1 < x < 50.0 + δ1 and y l -δ1 < y < y l + 50.0 + δ1 with δ1 = 3.0; The first initial point cloud and the second initial point cloud that have completed the through filtering are subjected to radius filtering to remove noise points, thereby obtaining two corresponding filtered point clouds.

4. A method for modeling a bulk cargo gantry crane cabin and materials therein based on 3D laser radar as claimed in claim 1, characterized in that: The step S4-2 specifically includes: For the first initial point cloud, with side length l cur = l max Perform voxel filtering on it, and the resulting point cloud size is count cur , perform the following judgment: i. If count cur ≥ count min , indicating that the point cloud is already dense, the voxel filtering process ends, and the final point cloud size is count cur ; ii. Otherwise, it means that the current point cloud is too sparse, so the side length is halved, that is, l cur = l cur / 2.0, perform voxel filtering again; iii. If the result count cur <count min , continue to execute step ii, and repeat until the voxel filtering process is completed; otherwise, it means that there is a suitable filtering edge length l∈(l cur ~2l cur ), perform voxel filtering based on this side length, and obtain a scale of count min The first sparse point cloud, record the filter side length as l fin ; For the second sparse point cloud, set the filter edge length to l fin , perform voxel filtering to obtain the second sparse point cloud.

5. A computer-readable storage medium, characterized in that: The storage medium stores a method program for modeling a bulk cargo gantry crane cabin and materials therein based on 3D laser radar, and when the method program for modeling a bulk cargo gantry crane cabin and materials therein based on 3D laser radar is executed, a method for modeling a bulk cargo gantry crane cabin and materials therein based on 3D laser radar as claimed in any one of claims 1 to 4 is implemented.

Images