Screw ship unloader cabin material collaborative perception and autonomous operation method based on multi-source laser scanning array

Abstract

The invention relates to a screw ship unloader cabin material collaborative perception and autonomous operation method based on a multi-source laser scanning array. The problems that in the prior art, a single laser scanning system is large in scanning blind area, high in positioning error and poor in hatch recognition stability are solved. The method comprises the following steps: S1, heterogeneous laser scanning array networking; s2, pose difference-IMU deep coupling of motion point cloud distortion suppression is carried out; s3, manually marking target spot binding; s4, point cloud data processing; and S5, multi-stage anti-collision control is carried out. The method has the advantages that the hatch recognition stability and the coal pile model construction precision are remarkably improved; the point cloud is not distorted in a dynamic environment, so that the manual intervention and the accident rate are effectively reduced; and the technical requirements of unattended ship unloading operation are met, and high industrial adaptability is achieved.

Description

Technical Field

[0001] The present invention relates to the technical field of loading and unloading automation, and in particular to a method for collaborative sensing and autonomous operation of spiral ship unloading engine room materials based on a multi-source laser scanning array. Background Art

[0002] Currently, bulk cargo unloading operations at ports rely primarily on manual labor, resulting in low efficiency and significant safety risks. Although laser scanning technology is gradually being applied to automated unloading, existing single-use laser scanning systems often suffer from large blind spots, high positioning errors, and poor hatch recognition stability in spiral continuous ship unloaders due to complex cabin structures, irregular material stacking, and severe environmental interference, making them difficult to meet the demands of unmanned, high-efficiency operations. Summary of the Invention

[0003] The purpose of the present invention is to address the problems existing in the prior art and propose a method for collaborative perception and autonomous operation of screw ship unloader cabin materials based on a multi-source laser scanning array. The multi-source laser scanning array collaborative perception method is combined with an artificial marking target mechanism and a dynamic distortion compensation algorithm to achieve high-precision modeling and autonomous operation control of the entire cabin area of ​​the screw ship unloader under complex working conditions, thereby improving unloading efficiency and operation safety.

[0004] To achieve the above objectives, the present invention adopts the following technical solutions: a method for collaborative sensing and autonomous operation of screw unloading engine room materials based on a multi-source laser scanning array, comprising the following steps: S1, heterogeneous laser scanning array networking; S2, pose difference-IMU deep coupling for motion point cloud distortion suppression; S3, artificially labeled target binding; S4, point cloud data processing; S5, multi-level collision avoidance control.

[0005] In step S1, the heterogeneous laser scanning array networking includes 3D laser scanner layout, 2D swing table laser scanner layout and blind spot safety monitoring coverage.

[0006] In the above-mentioned method for collaborative perception and autonomous operation of screw unloader engine room materials based on a multi-source laser scanning array, the 3D laser scanner layout includes: S011, Four 3D scanners are symmetrically arranged at the four corners of the rotary platform to form a trapezoidal arrangement. S012, each device has a field of view of 120°×25°, and the overall network achieves full coverage of every corner of the hatch. S013. Use the dock coordinate system to calibrate the external parameters of each scanner to achieve real-time fusion of multi-source point clouds in a unified space.

[0007] In the aforementioned method for collaborative perception and autonomous operation of screw unloader engine room materials based on a multi-source laser scanning array, the 2D swing table laser scanner layout includes: S021, 2D swing table laser scanner is installed at the end of the horizontal arm and driven by servo controller to swing with an angle range of ±90°. The scanning surface of S022, 2D swing table laser scanner is perpendicular to the longitudinal axis of the hull, and is used to penetrate the blind spots of coal piles for blind spot scanning. S023, the swing frequency is set to 2Hz to meet the contour collection requirements of the bottom of the cabin and the corner area; Blind spot safety monitoring coverage includes: S031. By coordinating the platform angle and unloading path planning, intermittent coverage of blind spots is achieved; S032. Form dynamic triangular monitoring areas in risk areas such as discharge heads, bulkheads, and beams.

[0008] In the above-mentioned method for collaborative sensing and autonomous operation of screw ship unloader cabin materials based on a multi-source laser scanning array, step S2 includes the following steps: S21. Synchronous acquisition of multi-source data: 3D laser scanner and 2D swing laser scanner synchronously acquire IMU attitude information and GNSS differential positioning data, with data synchronization accuracy controlled within 10ms. S22, posture difference calculation: Taking the laser scanner installation point as the reference, calculate its posture changes Δθ, Δx, Δy, Δz in continuous time frames, and form a time difference posture transformation matrix for post-processing compensation of position drift at the scanning point. S23, deep coupling compensation algorithm: embeds IMU attitude changes into point cloud projection algorithm in real time, corrects the spatial mapping result of each frame of point cloud, and then uses Kalman filter to filter the GNSS-IMU data fusion to further reduce interference jitter. S24. Effect evaluation: The deep coupling compensation algorithm reduces the maximum distortion error of the swing arm scanning point cloud from more than 200mm to less than 100mm.

[0009] In the above-mentioned method for collaborative sensing and autonomous operation of screw ship unloader cabin materials based on a multi-source laser scanning array, step S3 includes the following steps: S31. Target placement: Place high-reflectivity active targets at the ends of the barge hatch reinforcements and in the structural stability area. The reflectivity should be ≥90%. S32, Initial Modeling and Binding: Scan the target position before entering the hatch for the first time, and use the point cloud intensity threshold method to extract the high reflective area. S33, calculate the three-dimensional coordinates of its center of mass as the target reference point, and perform rigid coordinate registration with the hatch boundary to establish the spatial transformation matrix T, S334, dynamic identification and coordinate mapping: Each time the barge is subsequently entered for operation, the target position is scanned and identified first, and the deviation from the initial T matrix is ​​calculated to achieve automatic real-time correction of the unloader's operating space.

[0010] In the above-mentioned method for collaborative sensing and autonomous operation of screw ship unloader cabin materials based on a multi-source laser scanning array, step S4 includes the following steps: S41, dynamic filtering; S42, multi-source point cloud registration; S43, feature extraction; S44. Model reconstruction.

[0011] In the above-mentioned method for collaborative sensing and autonomous operation of screw ship unloader cabin materials based on a multi-source laser scanning array, step S41 includes the following steps: S411, remove outliers based on statistics from the collected original point cloud data, S412, calculate the Euclidean distance between each point and its k nearest neighbor points. If the deviation from the mean distance exceeds the set threshold, it is considered to be noise. S413, typical parameters: k=50, standard deviation multiple threshold is set to 1.0-2.0, S414: The percentage of valid point clouds retained after processing is increased to over 95%; Step S42 includes the following steps: S421, using the fixed position 3D laser scanner point cloud as the global reference frame, S422, use the improved iterative closest point algorithm to perform coordinate transformation on the local point cloud generated by the 2D swing table laser scanner, S423, the matching strategy uses edge features and curvature to collaboratively control error convergence, S424, the final registration error is controlled within 0.5 degree angular accuracy and 30 mm linear accuracy; In the above-mentioned method for collaborative sensing and autonomous operation of spiral ship unloader cabin materials based on a multi-source laser scanning array, step S43 includes the following steps: S431, hatch edge recognition uses a straight line detection algorithm based on Hough transform, S432, the regional growth clustering method is used for the coal pile area, and the point cloud intensity and geometric curvature are introduced as dual features for joint extraction. S433, intensity reflects the reflective target or metal boundary, and geometric information is used to distinguish the coal pile and the cabin structure; Step S44 includes the following steps: S441. The reconstruction results are presented in the form of a three-dimensional grid to build a digital twin model of the barge cabin. S442, use Octree voxel filtering method to downsample and optimize the model to improve modeling efficiency, S443, the update frequency is 6 times per second, which meets the real-time requirements of unloading path planning and collision avoidance control.

[0012] In the above-mentioned method for collaborative sensing and autonomous operation of screw ship unloader cabin materials based on a multi-source laser scanning array, step S5 includes the following steps: S51, real-time monitoring of the relative distance between the discharge head and key parts of the bulkhead and beam; S52: Over-limit triggers emergency deceleration or emergency stop, with a response time of less than 0.3 seconds.

[0013] Compared with the existing technology, the advantages of the present invention are: significantly improving the hatch recognition stability and the coal pile model construction accuracy; the point cloud is no longer distorted in a dynamic environment, and the system positioning stability reaches 99.2%; the operating efficiency is improved by about 22%, effectively reducing manual intervention and accident rates; it meets the technical requirements of unmanned unloading operations and has strong industrial adaptability. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 is a flow chart of the method of the present invention; Figure 2 This is a schematic diagram of the hardware deployment in the present invention; Figure 3 This is a schematic diagram of the deployment of the 3D laser scanner in the present invention; Figure 4 This is a schematic diagram of the deployment of artificially labeled targets in the present invention; Figure 5 This is a schematic diagram of the deployment of the 2D laser scanner in the present invention; Figure 6 Scanning a point cloud image for the barge of the present invention; Figure 7 This is the point cloud image scanned by the 2D laser scanner in the present invention. DETAILED DESCRIPTION

[0015] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0016] like Figure 1-2 As shown, a method for collaborative perception and autonomous operation of screw unloading engine room materials based on a multi-source laser scanning array includes the following steps: S1, heterogeneous laser scanning array networking; S2, pose difference-IMU deep coupling for motion point cloud distortion suppression; S3, artificially labeled target binding; S4, point cloud data processing; S5, multi-level collision avoidance control.

[0017] like Figure 4 As shown, the artificially marked targets use high reflectivity markers to improve the quality and efficiency of scanning and recognition.

[0018] In step S1, the heterogeneous laser scanning array networking includes 3D laser scanner layout, 2D swing table laser scanner layout and blind spot safety monitoring coverage.

[0019] like Figure 3 As shown, the 3D laser scanner layout includes: S011, Four 3D scanners are symmetrically arranged at the four corners of the rotary platform to form a trapezoidal arrangement. S012, each device has a field of view of 120°×25°, and the overall network achieves full coverage of every corner of the hatch. S013. Use the dock coordinate system to calibrate the external parameters of each scanner to achieve real-time fusion of multi-source point clouds in a unified space; like Figure 5 As shown, the 2D swing table laser scanner layout includes: S021, 2D swing table laser scanner is installed at the end of the horizontal arm and driven by servo controller to swing with an angle range of ±90°. The scanning surface of S022, 2D swing table laser scanner is perpendicular to the longitudinal axis of the hull, and is used to penetrate the blind spots of coal piles for blind spot scanning. S023, the swing frequency is set to 2Hz to meet the contour collection requirements of the bottom of the cabin and the corner area; Blind spot safety monitoring coverage includes: S031. By coordinating the platform angle and unloading path planning, intermittent coverage of blind spots is achieved; S032: Dynamic triangular monitoring zones are formed in risk areas such as the discharge head, bulkhead, and beams. This can achieve a sensing coverage rate of no less than 98%, effectively coping with high-obstruction operating conditions.

[0020] To achieve blind-spot-free perception of the entire barge cabin by the screw ship unloader, the present invention uses multiple types of 3D laser scanners and 2D swing-table laser scanners with different fields of view and deployment angles, which are distributed in multiple functional parts of the equipment according to the preset operating logic of the ship unloader, thereby forming a complementary, multi-angle, high-density three-dimensional perception network for the complex cabin space.

[0021] The system also integrates multiple 3D laser scanners with a wide horizontal viewing angle (120°×25°) for full-cabin contour modeling and positioning benchmark establishment, as well as a 2D laser scanner with a servo swing table for penetrating scanning, real-time blind spot filling and collision avoidance detection, forming a laser scanning networking solution of "heterogeneous acquisition, heterogeneous blind spot filling, and heterogeneous fusion".

[0022] Step S2 includes the following steps: S21. Synchronous acquisition of multi-source data: The 3D laser scanner and 2D swing laser scanner synchronously acquire IMU attitude information (roll angle, pitch angle, yaw angle) and GNSS differential positioning data, with data synchronization accuracy controlled within 10ms.

[0023] S22, posture difference calculation: Taking the laser scanner installation point as the reference, calculate its posture changes Δθ, Δx, Δy, Δz in continuous time frames, and form a time difference posture transformation matrix for post-processing compensation of position drift at the scanning point.

[0024] The laser scanners here include 3D laser scanners and 2D swing laser scanners.

[0025] S23, deep coupling compensation algorithm: embeds IMU posture changes into the point cloud projection algorithm in real time, corrects the spatial mapping results of each frame of point cloud, and then uses Kalman filtering to filter the GNSS-IMU data fusion to further reduce interference jitter.

[0026] S24. Effect evaluation: The deep coupling compensation algorithm reduces the maximum distortion error of the swing arm scanning point cloud from more than 200mm to less than 100mm; Meet the accuracy requirements of the anti-collision control system and improve the accuracy of unloading path planning.

[0027] To address the problem of 2D laser scanning point cloud distortion caused by horizontal arm swing, a compensation algorithm based on posture difference and IMU deep coupling was proposed. GNSS RTK and IMU were used to compensate for the horizontal arm swing and low-frequency vibration to achieve dynamic point cloud coordinate stabilization.

[0028] Step S3 includes the following steps: S31. Target placement: High-reflectivity active targets with a reflectivity of ≥90% are placed at the ends of the barge hatch reinforcement ribs and in the structurally stable areas. The targets are square in shape, approximately 500mm in length and width, and are surfaced with mirror-grade film material to facilitate laser identification.

[0029] like Figure 6-7 As shown in the figure, a heterogeneous laser scanning array network is used to collect point cloud data of the barge hatch area, and the high-intensity point cloud area corresponding to the target is extracted by the intensity threshold segmentation method; S32, Initial Modeling and Binding: Scan the target location before entering the hatch for the first time and use the point cloud intensity threshold method to extract the highly reflective areas; S33, calculating the three-dimensional coordinates of its center of mass as the target reference point, and performing rigid coordinate registration with the hatch boundary to establish a spatial transformation matrix T; In step S33, the centroid coordinates of the target point cloud are calculated as the reference point for hatch space calibration. During the first calibration process, the spatial rigid transformation relationship between the target and the edge of the barge hatch is obtained to establish the spatial mapping matrix T. S334, dynamic identification and coordinate mapping: Each time the barge is subsequently entered for operation, the target position is scanned and identified first, and the deviation from the initial T matrix is ​​calculated to achieve automatic real-time correction of the unloader's operating space.

[0030] That is, in the subsequent unloading operation, by re-identifying the target position and dynamically deducing the spatial position and posture of the barge hatch in the dock coordinate system based on the transformation matrix T, the stable positioning of the barge cabin in a changing environment is achieved.

[0031] This mechanism improves the hatch positioning stability from 85% to 99%, significantly suppressing the disturbance effect of hull shaking on the operating coordinates.

[0032] High-reflectivity targets are placed at the barge hatch reinforcement ribs, and the rigid spatial relationship between the dock and the hatch is established by identifying their three-dimensional center of mass coordinates. The manual marker binding mechanism is used to solve the coordinate drift problem caused by hull drift, scanning interruption or structure obstruction.

[0033] Step S4 includes the following steps: S41, dynamic filtering; S42, multi-source point cloud registration; S43, feature extraction; S44. Model reconstruction.

[0034] Step S41 includes the following steps: S411, remove outliers based on statistics from the collected original point cloud data, S412. Calculate the Euclidean distance between each point and its k nearest neighbor points. If the deviation from the mean distance exceeds a set threshold, it is considered to be noise (such as floating points caused by dust or fog). S413, typical parameters: k=50, standard deviation multiple threshold is set to 1.0-2.0, S414: The percentage of valid point clouds retained after processing is increased to over 95%; Step S42 includes the following steps: S421, using the fixed position 3D laser scanner point cloud as the global reference frame, S422, use the improved iterative closest point (ICP) algorithm to perform coordinate transformation on the local point cloud generated by the 2D swing table laser scanner, S423, the matching strategy uses edge features and curvature to collaboratively control error convergence, S424, the final registration error is controlled within 0.5 degree angular accuracy and 30 mm linear accuracy; Step S43 includes the following steps: S431, hatch edge recognition uses a straight line detection algorithm based on Hough transform, S432, the regional growth clustering method is used for the coal pile area, and the point cloud intensity and geometric curvature are introduced as dual features for joint extraction. S433, intensity reflects the reflective target or metal boundary, and geometric information is used to distinguish the coal pile and the cabin structure; Step S44 includes the following steps: S441. The reconstruction results are presented in the form of a three-dimensional grid to build a digital twin model of the barge cabin. S442, use Octree voxel filtering method to downsample and optimize the model to improve modeling efficiency, S443, the update frequency is 6 times per second, which meets the real-time requirements of unloading path planning and collision avoidance control. Currently, through field testing, the hatch recognition rate has been increased to 98.7%, and the modeling error is controlled at ±2.8% (<100mm).

[0035] Step S5 includes the following steps: S51, real-time monitoring of the relative distance between the discharge head and key parts of the bulkhead and beam; S52: Over-limit triggers emergency deceleration or emergency stop, with a response time of less than 0.3 seconds.

[0036] In summary, the principle of this embodiment is as follows: by deploying multiple 3D laser scanners on the rotary platform to form full hatch scanning coverage, and deploying a 2D laser scanner at the end of the horizontal arm and achieving occlusion compensation through swinging; using the combination of GNSSRTK and IMU to achieve dynamic posture compensation of point cloud coordinates, and secondly, by manually setting reflective targets, constructing a rigid mapping between the barge hatch and the dock coordinate system, filtering, multi-source registration, feature extraction and digital modeling of the collected point cloud, and driving the path generation and collision avoidance control of the unloading operation. By integrating multiple 3D and 2D laser scanning, differential positioning and deep coupling of IMU, artificial target binding mechanism and multi-level collision avoidance control logic, a digital twin of the entire cabin domain is constructed, which significantly improves the intelligence level of coal pile identification, hatch modeling and autonomous unloading, and has a high industrial application prospect.

[0037] The specific embodiments described herein are merely illustrative of the spirit of the present invention. Persons skilled in the art may make various modifications, additions, or substitutions to the described specific embodiments without departing from the spirit of the present invention or exceeding the scope of the appended claims.

Claims

1. A method for collaborative sensing and autonomous operation of screw unloading engine room materials based on a multi-source laser scanning array, characterized in that: The following steps are involved: S1, heterogeneous laser scanning array networking; S2, pose difference-IMU deep coupling for motion point cloud distortion suppression; S3, artificially labeled target binding; S4, point cloud data processing; S5, multi-level collision avoidance control.

2. The method for collaborative sensing and autonomous operation of screw unloader cabin materials based on a multi-source laser scanning array according to claim 1 is characterized in that: In step S1, the heterogeneous laser scanning array networking includes a 3D laser scanner layout, a 2D swing table laser scanner layout, and blind spot safety monitoring coverage.

3. The method for collaborative sensing and autonomous operation of screw unloader cabin materials based on a multi-source laser scanning array according to claim 2 is characterized in that: The 3D laser scanner layout includes: S011, Four 3D scanners are symmetrically arranged at the four corners of the rotary platform to form a trapezoidal arrangement. S012, each device has a field of view of 120°×25°, and the overall network achieves full coverage of every corner of the hatch. S013. Use the dock coordinate system to calibrate the external parameters of each scanner to achieve real-time fusion of multi-source point clouds in a unified space.

4. A spiral laser scanner based on a multi-source laser scanning array according to claim 3 The collaborative perception and autonomous operation method for ship unloading engine room materials is characterized by: The 2D swing table laser scanner layout includes: S021, 2D swing table laser scanner is installed at the end of the horizontal arm and driven by servo controller to swing with an angle range of ±90°. The scanning surface of S022, 2D swing table laser scanner is perpendicular to the longitudinal axis of the hull, and is used to penetrate the blind spots of coal piles for blind spot scanning. S023, the swing frequency is set to 2Hz to meet the contour collection requirements of the bottom of the cabin and the corner area; The blind spot safety monitoring coverage includes: S031. By coordinating the platform angle and unloading path planning, intermittent coverage of blind spots is achieved; S032. Form dynamic triangular monitoring areas in risk areas such as discharge heads, bulkheads, and beams.

5. The method for collaborative sensing and autonomous operation of screw ship unloader cabin materials based on a multi-source laser scanning array according to claim 4 is characterized in that: Step S2 includes the following steps: S21. Synchronous acquisition of multi-source data: 3D laser scanner and 2D swing laser scanner synchronously acquire IMU attitude information and GNSS differential positioning data, with data synchronization accuracy controlled within 10ms. S22, posture difference calculation: Taking the laser scanner installation point as the reference, calculate its posture changes Δθ, Δx, Δy, Δz in continuous time frames, and form a time difference posture transformation matrix for post-processing compensation of position drift at the scanning point. S23, deep coupling compensation algorithm: embeds IMU attitude changes into point cloud projection algorithm in real time, corrects the spatial mapping result of each frame of point cloud, and then uses Kalman filter to filter the GNSS-IMU data fusion to further reduce interference jitter. S24. Effect evaluation: The deep coupling compensation algorithm reduces the maximum distortion error of the swing arm scanning point cloud from more than 200mm to less than 100mm.

6. The method for collaborative sensing and autonomous operation of screw unloader cabin materials based on a multi-source laser scanning array according to claim 5 is characterized in that: Step S3 includes the following steps: S31. Target placement: Place high-reflectivity active targets at the ends of the barge hatch reinforcements and in the structural stability area. The reflectivity should be ≥90%. S32, Initial Modeling and Binding: Scan the target position before entering the hatch for the first time, and use the point cloud intensity threshold method to extract the high reflective area. S33, calculate the three-dimensional coordinates of its center of mass as the target reference point, and perform rigid coordinate registration with the hatch boundary to establish the spatial transformation matrix T, S334, dynamic identification and coordinate mapping: Each time the barge is subsequently entered for operation, the target position is scanned and identified first, and the deviation from the initial T matrix is ​​calculated to achieve automatic real-time correction of the unloader's operating space.

7. The method for collaborative sensing and autonomous operation of screw ship unloader cabin materials based on a multi-source laser scanning array according to claim 6 is characterized in that: Step S4 includes the following steps: S41, dynamic filtering; S42, multi-source point cloud registration; S43, feature extraction; S44. Model reconstruction.

8. The method for collaborative sensing and autonomous operation of screw unloader cabin materials based on a multi-source laser scanning array according to claim 7 is characterized in that: Step S41 includes the following steps: S411, remove outliers based on statistics from the collected original point cloud data, S412, calculate the Euclidean distance between each point and its k nearest neighbor points. If the deviation from the mean distance exceeds the set threshold, it is considered to be noise. S413, typical parameters: k=50, standard deviation multiple threshold is set to 1.0-2.0, S414: The percentage of valid point clouds retained after processing is increased to over 95%; Step S42 includes the following steps: S421, using the fixed position 3D laser scanner point cloud as the global reference frame, S422, use the improved iterative closest point algorithm to perform coordinate transformation on the local point cloud generated by the 2D swing table laser scanner, S423, the matching strategy uses edge features and curvature to collaboratively control error convergence, S424. The final registration error is controlled within 0.5 degree angular accuracy and 30 mm linear accuracy.

9. The method for collaborative sensing and autonomous operation of screw ship unloader cabin materials based on a multi-source laser scanning array according to claim 8 is characterized in that: Step S43 includes the following steps: S431, hatch edge recognition uses a straight line detection algorithm based on Hough transform, S432, the regional growth clustering method is used for the coal pile area, and the point cloud intensity and geometric curvature are introduced as dual features for joint extraction. S433, intensity reflects the reflective target or metal boundary, and geometric information is used to distinguish the coal pile and the cabin structure; Step S44 includes the following steps: S441. The reconstruction results are presented in the form of a three-dimensional grid to build a digital twin model of the barge cabin. S442, use Octree voxel filtering method to downsample and optimize the model to improve modeling efficiency, S443, the update frequency is 6 times per second, which meets the real-time requirements of unloading path planning and collision avoidance control.

10. The method for collaborative sensing and autonomous operation of screw unloader cabin materials based on a multi-source laser scanning array according to claim 1, characterized in that: Step S5 includes the following steps: S51, real-time monitoring of the relative distance between the discharge head and key parts of the bulkhead and beam; S52: Over-limit triggers emergency deceleration or emergency stop, with a response time of less than 0.3 seconds.

Images