Three-dimensional refined modeling method for cargo hold of ship unloader

Abstract

The invention belongs to the technical field of mechanical engineering and computer vision crossing, particularly relates to a three-dimensional refined modeling method for a cargo hold of a ship unloader, and aims to solve the problem of high-precision real-time modeling of a dynamic scene of the cargo hold under strong reflection and vibration interference. According to the method, space-time alignment point clouds are synchronously collected through a multi-source laser scanning unit, and dynamic denoising is realized by combining inertia compensation and polarized light reflection suppression; a curvature and normal vector self-adaptive multi-scale voxel division strategy is adopted, the efficiency is improved in a flat area, and millimeter-level details are reserved in a reactor peak; an incremental Poisson reconstruction algorithm of a time weight factor is introduced, and a continuous frame gradient field is fused to generate a dynamic evolution surface model. According to the scheme, the void ratio of the model is lower than 3%, the error of a key area is smaller than 20 mm, the updating delay is controlled within 200 milliseconds, meanwhile, the semantic segmentation precision of 95% or above is achieved, the path planning safety and the operation efficiency of the ship unloader are remarkably improved, and the intelligent real-time sensing requirement of a port is met.

Description

Technical Field

[0001] This invention belongs to the interdisciplinary field of mechanical engineering and computer vision, and specifically relates to a method for detailed three-dimensional modeling of the cargo hold of a ship unloader. Background Technology

[0002] With the continuous improvement of port automation and intelligence, ship unloaders, as core equipment for bulk cargo handling, increasingly rely on accurate perception and dynamic modeling capabilities of the ship's cargo hold space to ensure operational efficiency and safety. Traditional modeling methods are mostly based on two-dimensional planar projection or coarse-grained point cloud segmentation, which makes it difficult to fully reproduce the complex geometric structure and material distribution characteristics inside the cargo hold. This results in a lack of spatial adaptability in equipment path planning, easily leading to collision risks or grabbing empty loads. Especially under conditions of multiple types of materials, irregular stacking, and dynamic changes in tilt angle, the system places higher demands on the real-time reconstruction accuracy and detail fidelity of the three-dimensional morphology of the cargo hold.

[0003] However, existing 3D reconstruction algorithms generally suffer from feature matching failures and surface detail loss when dealing with strong reflections from metal bulkheads, dust occlusion, and local shadow interference. This results in high porosity and blurred edges in the model, making it unable to support high-precision pose guidance. Furthermore, cargo hold point cloud data exhibits non-uniform density and multi-scale topological characteristics, while mainstream meshing methods lack adaptive resolution adjustment mechanisms, making it difficult to balance global contour integrity with the representation of local concavity and convexity details. In addition, the modeling process often employs static frame fusion strategies, failing to effectively correlate the spatiotemporal evolution between consecutive scans. This leads to the smooth masking of dynamic material stacking deformation and micro-vibrations in the cargo hold, weakening the model's ability to map real-world operational scenarios.

[0004] Therefore, a method for detailed 3D modeling of cargo holds for ship unloader operation scenarios is desired. Summary of the Invention

[0005] The purpose of this invention is to provide a method for detailed three-dimensional modeling of the cargo hold of a ship unloader, which can effectively solve the problems mentioned in the background art.

[0006] To achieve the above objectives, the technical solution adopted by the present invention is as follows: A method for refining the three-dimensional modeling of a ship unloader's cargo hold includes the following specific steps: Step (1) Acquiring multi-source heterogeneous point cloud data of the cargo hold: Through multiple sets of laser scanning units installed on the main beam and grab bucket mechanism of the ship unloader, three-dimensional point cloud data from different perspectives are collected synchronously during the cargo hold operation. The laser scanning unit adopts the principle of multi-frequency modulated continuous wave laser ranging, combined with the inertial measurement unit to compensate for the pose deviation caused by equipment vibration in real time, and obtains the original point cloud sequence with spatiotemporal synchronization; Step (2) Performing dynamic denoising and reflection suppression of the point cloud: The original point cloud sequence is subjected to statistical outlier removal based on local density distribution. The number of neighborhood points is set to 64, the standard deviation multiple is 2.2, and a polarized light-assisted reflection intensity correction model is introduced to suppress false point clouds caused by specular reflection of the metal bulkhead and retain the real geometric surface data; Step (3) Constructing an adaptive multi-scale voxel mesh: According to the consistency index of local curvature and normal vector of the point cloud, the voxel mesh is dynamically divided. To improve processing efficiency, a 50 mm voxel size is used for flat areas such as cargo hold walls, and a 10 mm voxel size is used for material pile tips, corners, and grab bucket disturbance areas to retain detailed features, forming a non-uniform but topologically coherent voxel representation; Step (4) Implementing spatiotemporal fusion incremental surface reconstruction: Based on the pose association information between continuous scanning frames, an improved Poisson surface reconstruction algorithm is adopted, and a time dimension weight factor is introduced to weight and fuse the gradient fields of corresponding spatial regions in adjacent frames to generate a continuous surface model with dynamic evolution characteristics, avoiding deformation smoothing caused by static fusion; Step (5) Generating and outputting a refined three-dimensional cargo hold model: Hole identification and semantic guidance repair are performed on the reconstructed surface model, and local surface fitting is used to fill holes with a diameter of less than 200 mm in combination with the prior knowledge of the cargo hold structure, and finally a three-dimensional refined cargo hold model containing complete geometric contours and material distribution details is output for the unloader path planning system to call.

[0007] Preferably, the laser scanning unit in step (1) includes at least 3 sets of horizontal scanners installed at the front end of the unloader beam and 2 sets of top-view scanners integrated on the top of the grab bucket. The horizontal scanners cover a horizontal angle of 270 degrees and a vertical angle of 30 degrees. The top-view scanners cover a vertical downward field of view of 90 degrees. The synchronous triggering frequency of all scanners is not less than 10 Hz, and the point cloud data output delay is less than 50 milliseconds.

[0008] Preferably, in step (2), the polarization-assisted reflection intensity correction model collects the echo intensity through dual polarization channels, calculates the Stokes parameter and constructs a Fresnel reflection coefficient estimator, and performs intensity attenuation compensation for high reflection areas with an incident angle greater than 60 degrees, so that the standard deviation of point cloud intensity is reduced to less than 35% of the original value, effectively distinguishing between real material surfaces and specular reflection artifacts.

[0009] Preferably, in step (3), the local curvature calculation adopts fifth-order moving least squares fitting, the neighborhood radius is adaptively set to 8 times the average spacing of the point cloud, and the normal vector consistency is determined by calculating the standard deviation of the angle between the normal vectors in the neighborhood. When the standard deviation is less than 5 degrees, it is determined to be a flat area; otherwise, it is determined to be a high-detail area. Based on this, the voxel size is dynamically allocated.

[0010] Preferably, in step (4), the time dimension weight factor is dynamically adjusted according to the inter-frame time interval and the dynamic change rate of the cargo hold. The weight factor for the static bulkhead area is set to 0.9, and the weight factor for the material pile surface area is set to 0.6, so as to ensure that the model can maintain structural stability and respond to the deformation of the material pile, and the continuity error of the reconstructed surface normal vector is less than 3 degrees.

[0011] Preferably, in step (5), the hole identification adopts the visibility analysis method based on ray projection, combined with the cabin boundary constraints defined by the cargo hold CAD structural model. Holes located in the cabin wall area are preferentially repaired by plane fitting, and holes located in the material pile area are repaired by quadratic surface fitting. After repair, the change rate of Gaussian curvature on the model surface does not exceed 1.2 times that of the original area.

[0012] Preferably, it also includes: performing semantic segmentation on the generated 3D cargo hold model, and dividing the model into four semantic labels based on the local geometric features and height distribution statistics of the point cloud: bulkhead, bottom, material pile and empty hold area, with a segmentation accuracy of greater than or equal to 95%, providing semantic-level environmental perception for the trajectory planning of the unloading machine grab bucket.

[0013] Preferably, it also includes: establishing a cargo hold model update mechanism, which triggers the model incremental update process when the Chamfer distance between two consecutive reconstructed models is greater than the preset threshold of 50 mm, and only re-executes steps (3) to (5) for the changed area, so that the model update delay is controlled within 200 milliseconds to meet the real-time operation requirements.

[0014] Preferably, the method is integrated into the central control system of the ship unloader, and is linked with the grab bucket posture sensor, material flow meter and anti-collision module. The three-dimensional cargo hold model is updated at a frequency of not less than 5 Hz, the absolute positioning error of the model is less than 30 mm, and the relative deformation detection sensitivity reaches the order of 10 mm.

[0015] Compared with the prior art, the present invention has the following beneficial effects: High-precision geometric reconstruction capability Breakthrough in modeling strong interference environments: Through polarized light-assisted reflection suppression and dynamic denoising mechanisms, it effectively addresses strong reflections from metal bulkheads, dust occlusion, and local shadow interference. The model porosity is less than 3%, and the edge clarity is improved by more than 40%, which is significantly better than traditional point cloud reconstruction methods. Multi-scale detail fidelity: The adaptive voxel mesh strategy takes into account both the integrity of the global contour and the expression of local concavity and convex details. The geometric restoration error of key areas such as material stack tip and cabin corner is less than 20 mm, while the error of traditional uniform mesh method is generally greater than 80 mm.

[0016] Real-time modeling capability for dynamic scenes Spatiotemporal fusion reconstruction mechanism: The incremental surface reconstruction algorithm, which introduces a time dimension weighting factor, effectively preserves the dynamic material stacking deformation and cabin micro-vibration characteristics, improves the model's ability to map real operating scenarios by 60%, and avoids deformation smoothing distortion caused by static fusion. Highly efficient model update: Based on the incremental update mechanism triggered by Chamfer distance, it only processes the changed areas, and the model update latency is controlled within 200 milliseconds, which meets the real-time modeling requirements of ship unloaders at high speeds and improves efficiency by more than 5 times compared to full reconstruction.

[0017] System integration and operational support capabilities Semantic-level environmental perception: Combining geometric features and height statistics, the semantic segmentation module accurately delineates key areas such as bulkheads and material piles, providing structured environmental information for path planning, reducing the grab bucket empty load rate by 25%, and improving the collision warning accuracy to 98%; Highly robust engineering implementation: Multi-source heterogeneous point cloud acquisition and inertial compensation mechanism effectively overcome vibration interference during ship unloading operations, with the absolute positioning error of the model being less than 30 mm, supporting high-precision pose guidance and significantly improving the safety and efficiency of ship unloading operations. Attached Figure Description

[0018] Figure 1 This is a flowchart of the overall technical solution of the present invention; Figure 2 This is a schematic diagram of the working principle of the laser scanning unit of the present invention; Figure 3 This is a flowchart of the point cloud dynamic denoising and reflection suppression process of the present invention; Figure 4 This is the adaptive multi-scale voxel mesh construction diagram of the present invention. Detailed Implementation

[0019] Please refer to Figures 1 to 4 To make the objectives, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below with reference to specific embodiments.

[0020] Currently, during ship unloading operations, the internal environment of the cargo hold is complex and variable, with multiple factors such as strong reflections from metal walls, dust obscuring the view, localized shadows, and equipment vibration interference. This makes it difficult for traditional 3D modeling methods to obtain a cargo hold model that combines high accuracy, real-time performance, and detailed fidelity. To address these technical problems, this invention proposes a 3D refined modeling method based on multi-source heterogeneous point cloud acquisition and spatiotemporal fusion reconstruction, and applies it to a 3D refined modeling method for a ship unloader's cargo hold.

[0021] In the above-mentioned method for detailed 3D modeling of a ship unloader's cargo hold, step (1) involves acquiring multi-source heterogeneous point cloud data of the cargo hold: Multiple laser scanning units installed on the ship unloader's main beam and grab bucket mechanism synchronously collect 3D point cloud data from different perspectives during cargo hold operations. The laser scanning units employ the principle of multi-frequency modulated continuous wave laser ranging, combined with an inertial measurement unit to compensate for pose deviations caused by equipment vibration in real time, thereby acquiring an original point cloud sequence with spatiotemporal synchronization. Specifically, the laser scanning units include at least three horizontal scanners installed at the front end of the ship unloader's main beam and two top-view scanners integrated on the top of the grab bucket. The horizontal scanners cover a horizontal angle of 270 degrees and a vertical angle of 30 degrees, while the top-view scanners cover a vertical downward field of view of 90 degrees. All scanners are synchronously triggered at a frequency of no less than 10 Hz, and the point cloud data output delay is less than 50 milliseconds. The horizontal scanner employs multi-frequency modulated continuous wave laser ranging technology, emitting modulated beams at frequencies of 100 MHz, 200 MHz, and 400 MHz. Distance information is calculated by detecting the phase difference of the echo signal through a receiver, achieving high-precision ranging of distant targets with a range up to 100 meters and an accuracy better than ±5 mm. The top-view scanner is integrated into the top of the grab bucket. Its optical system includes a lens group with a focal length of 150 mm, a field of view of 90 degrees, a pixel resolution of 1280×1024, and a frame rate of up to 30 Hz. It is used to capture high-resolution point cloud data of the material pile surface below the grab bucket. All scanners achieve time-series alignment of data acquisition through a timestamp synchronization mechanism. The timestamp is generated by the main control unit and broadcast to each scanner, ensuring that the time error of each sensor's data is less than 1 millisecond. The inertial measurement unit (IMU) is a six-degree-of-freedom IMU, comprising a three-axis accelerometer and a three-axis gyroscope, with a sampling frequency of 1000 Hz. It is used to monitor the vibration state of the unloader's beam and grab bucket mechanism in real time. Its output data, after Kalman filtering, is input as pose compensation parameters to the point cloud registration module. The point cloud data is stored in binary format, with each record containing X, Y, and Z coordinate values ​​(unit: mm), intensity values ​​(unit: 0-255), a timestamp (unit: microseconds), and a scanner ID field. The data stream is transmitted to the central processing unit via gigabit Ethernet, requiring a bandwidth of 1.2 gigabits per second to ensure packet loss-free and low-latency transmission. During the acquisition process, the original point cloud sequence achieves spatiotemporal synchronization of multi-source point cloud data through a dual mechanism of hardware-level synchronization triggering and software-level timestamp correction, achieving a synchronization accuracy of ±2 milliseconds, meeting the accuracy requirements of subsequent dynamic fusion and reconstruction.

[0022] In the above-mentioned method for detailed 3D modeling of the cargo hold of a ship unloader, step (2) involves dynamic point cloud denoising and reflection suppression: the original point cloud sequence is subjected to statistical outlier removal based on local density distribution. The number of neighboring points is set to 64, the standard deviation multiple is 2.2, and a polarized light-assisted reflection intensity correction model is introduced to suppress false point clouds caused by specular reflection of the metal bulkhead and retain the real geometric surface data. Specifically, the statistical outlier removal algorithm based on local density distribution adopts the Local Outlier Factor (LOF) method in statistics. First, 64 nearest neighbors are selected for each point cloud point, and the ratio of the local density of the point to the local density of other points in the neighborhood is calculated. When the ratio is greater than 2.2 times the standard deviation, the point is determined to be an outlier and removed. The number of neighboring points of 64 is based on the empirical setting of an average point cloud spacing of about 15 mm to ensure that the neighborhood contains a sufficient number of points to reflect the local density characteristics, while avoiding misjudgment of the boundary area due to an excessively large neighborhood. The standard deviation factor of 2.2 is the optimal threshold obtained through extensive experimental verification, which can effectively remove noise points while preserving true edge points to the maximum extent. The polarization-assisted reflection intensity correction model acquires echo intensity through dual polarization channels, calculates Stokes parameters, and constructs a Fresnel reflection coefficient estimator. It compensates for intensity attenuation in high-reflection areas with incident angles greater than 60 degrees, reducing the standard deviation of point cloud intensity to below 35% of the original value, effectively distinguishing between real material surfaces and specular reflection artifacts. Specifically, the dual polarization channels acquire echo intensity in the s-polarization and p-polarization directions respectively. By calculating Stokes parameters S0, S1, S2, and S3, the polarization state of the incident light is reconstructed, and the Fresnel reflection coefficient is estimated. The Fresnel reflection coefficient estimator is based on Brewster angle theory. When the incident angle is greater than 60 degrees, the reflectivity of the metal surface increases sharply, leading to abnormally enhanced point cloud intensity and the formation of false point clouds. The model establishes a nonlinear relationship between reflection intensity and incident angle to compensate for the attenuation of intensity values ​​in high-reflection areas. The compensation formula is as follows:

[0023] in, This is the original strength value. Angle of incidence The value shown is the corrected intensity value. This formula gradually reduces the intensity weight when the incident angle is greater than 60 degrees, lowering the standard deviation of the point cloud intensity from the original 120 to 42, a reduction of 65%, significantly improving the point cloud quality in the metal bulkhead region. The reflection suppression process and the dynamic denoising process are executed in parallel; intensity correction is performed first, followed by outlier removal, ensuring that the final point cloud data is free of both noise and reflection artifacts.

[0024] In the above-mentioned method for 3D fine modeling of the cargo hold of a ship unloader, step (3) involves constructing an adaptive multi-scale voxel mesh: based on the consistency index of local curvature and normal vector of the point cloud, the voxel resolution is dynamically divided. A 50 mm voxel size is used for flat areas such as the cargo hold walls to improve processing efficiency, while a 10 mm voxel size is used for material pile tips, corners, and grab bucket disturbance areas to preserve detailed features, forming a non-uniform but topologically coherent voxel representation. Specifically, the local curvature calculation uses a fifth-order moving least squares fitting, with the neighborhood radius adaptively set to 8 times the average spacing of the point cloud. Normal vector consistency is determined by calculating the standard deviation of the angle between the normal vectors in the neighborhood. When the standard deviation is less than 5 degrees, it is determined to be a flat area; otherwise, it is determined to be a high-detail area. The voxel size is dynamically allocated accordingly. The fifth-order moving least squares fitting constructs a local coordinate system around each point cloud point, and uses the coordinates and normal vector information of the points in the neighborhood to solve a fifth-order polynomial surface. The curvature is calculated from the second derivative matrix of this surface. The neighborhood radius is adaptively set to 8 times the average spacing of the point cloud to ensure sufficient neighborhood points are available for fitting even in sparse regions, improving the stability of curvature estimation. The normal vector consistency determination is achieved by calculating the angle between the normal vectors of all points within the neighborhood and taking its standard deviation. When the standard deviation is less than 5 degrees, the region is considered a flat surface, such as a bulkhead or hold bottom; when the standard deviation is greater than or equal to 5 degrees, the region is considered to have significant geometric changes, such as the surface of a material stack or a hold corner. The voxel mesh construction process first divides the entire cargo hold space into an initial voxel mesh with a voxel size of 50 mm. Then, based on the curvature and normal vector consistency results, high-detail regions are further subdivided, reducing the voxel size to 10 mm to form a non-uniform voxel structure. The voxel mesh is stored using an octree data structure, where each node represents a voxel, and its child nodes represent finer-grained voxel subdivisions, supporting efficient spatial querying and traversal. The voxelization representation achieves a balance between global efficiency and local detail while maintaining topological coherence. The geometric restoration error of the material pile tip region is less than 20 mm, while the error of traditional uniform mesh methods is generally greater than 80 mm.

[0025] In the above-mentioned method for three-dimensional fine modeling of the cargo hold of a ship unloader, step (4) implements incremental surface reconstruction with spatiotemporal fusion: based on the pose association information between consecutive scanning frames, an improved Poisson surface reconstruction algorithm is adopted, and a time dimension weight factor is introduced to weight and fuse the gradient fields of corresponding spatial regions in adjacent frames to generate a continuous surface model with dynamic evolution characteristics, avoiding deformation smoothing caused by static fusion. Specifically, the improved Poisson surface reconstruction algorithm introduces a time dimension weight factor on the basis of the traditional Poisson equation. This factor is dynamically adjusted according to the time interval between frames and the dynamic change rate of the cargo hold. The weight factor for the static bulkhead region is set to 0.9, and the weight factor for the material pile surface region is set to 0.6, ensuring that the model can maintain structural stability and respond to material pile deformation, and the continuity error of the reconstructed surface normal vector is less than 3 degrees. The pose association information is realized through the ICP algorithm. Using the point cloud data of the previous frame and the current frame, the rigid body transformation matrix between the two is calculated to achieve spatial alignment. The calculation formula of the time dimension weight factor is as follows:

[0026] in, This is the inter-frame time interval. The time decay coefficient, For decay rate, This is a factor related to the rate of dynamic change. For the static bulkhead region, =0.9, =0.1, =0.1, the weighting factor is stabilized at 0.9 to ensure structural invariance; for the surface region of the material pile, =0.6, =0.3, =0.4, the weighting factor decays rapidly over time, allowing new data to dominate surface updates. The gradient field weighted fusion process is performed on a voxel mesh. The gradient field within each voxel is calculated from the normal vectors and position information of its neighboring point cloud points. The gradient fields of adjacent frames are weighted and averaged using a time weighting factor to generate the fused gradient field. The fused gradient field is input to the Poisson equation solver, and the Laplace equation is solved using the finite element method to generate a continuous implicit surface function. Finally, a triangular mesh model is generated using an isosurface extraction algorithm (such as Marching Cubes). The continuity error of the reconstructed surface normal vector is measured by calculating the standard deviation of the angle between the normal vectors of adjacent triangular facets, and controlled within 3 degrees to ensure that the model surface is smooth and free of jagged distortion.

[0027] In the above-mentioned method for refining the three-dimensional cargo hold of a ship unloader, step (5) generates and outputs a refined three-dimensional cargo hold model: hole identification and semantic-guided repair are performed on the reconstructed surface model; holes with a diameter less than 200 mm are filled using local surface fitting, based on prior knowledge of the cargo hold structure; and finally, a refined three-dimensional cargo hold model containing complete geometric contours and material distribution details is output for use by the ship unloader path planning system. Specifically, the hole identification uses a visibility analysis method based on ray casting. Combined with the cargo hold CAD structural model's defined boundary constraints, holes located in the bulkhead area are preferentially repaired using planar fitting, while holes located in the material pile area are repaired using quadratic surface fitting. After repair, the Gaussian curvature change rate of the model surface does not exceed 1.2 times that of the original area. The ray casting method emits multiple rays outward from the hole boundary. If the rays do not intersect any model surface, the area is determined to be a hole. The cargo hold boundary constraints are derived from a pre-loaded cargo hold CAD model with an accuracy of ±1 mm, used to limit the search range for hole identification and avoid misjudgment. The planar fitting repair uses the least squares method to fit the boundary points of the hole, generating the best-fitting plane to fill the hole. The quadratic surface fitting repair uses the curvature information of the fitted boundary points to generate a surface that conforms to the local geometric features. The Gaussian curvature change rate of the repaired model surface is controlled within 1.2 times by calculating the curvature difference between the repaired area and the original area, ensuring a smooth transition between the repaired area and the surrounding surface. The three-dimensional refined cargo hold model is output in STL format, including the vertex coordinates of the triangular mesh, normal vectors, and material information, with a file size controlled within 50 megabytes to meet real-time transmission requirements. After the model is output, it is immediately transmitted to the central control system of the unloader, linked with the grab bucket posture sensor, material flow meter, and anti-collision module. The three-dimensional cargo hold model update frequency is no less than 5 Hz, the absolute positioning error of the model is less than 30 mm, and the relative deformation detection sensitivity reaches the order of 10 mm.

[0028] Furthermore, the algorithm includes semantic segmentation of the generated 3D cargo hold model. Based on the local geometric features and height distribution statistics of the point cloud, the model is divided into four semantic labels: bulkhead, hold floor, material pile, and empty hold area. The segmentation accuracy is greater than or equal to 95%, providing semantic-level environmental awareness for the unloader's grab trajectory planning. Specifically, the semantic segmentation algorithm first extracts the local geometric features of each point cloud point, including curvature, rate of change of normal vector, local density, etc., as well as height distribution statistics, such as height relative to the hold floor and height gradient. The feature vector has a dimension of 64 and is classified using a trained deep neural network model. This model adopts a convolutional neural network structure, containing three convolutional layers and two fully connected layers, with ReLU activation function and cross-entropy loss function. The model is trained on a dataset containing 100,000 labeled samples, and the segmentation accuracy on the test set reaches 96.3%. The semantic label output is an additional attribute of the point cloud data, with each point carrying its category label for subsequent path planning.

[0029] In addition, a cargo hold model update mechanism is established. When the Chamfer distance between two consecutive reconstructed models exceeds a preset threshold of 50 mm, the incremental model update process is triggered. Steps (3) to (5) are only re-executed for the changed areas, and the model update delay is controlled within 200 milliseconds to meet the requirements of real-time operation. Specifically, the Chamfer distance calculation formula is as follows:

[0030] in, and For two consecutive frames, for The number of midpoints. The threshold of 50 mm is set based on the typical deformation of material piles during ship unloader operations, ensuring timely model updates when significant changes occur. The incremental update process identifies changed areas by comparing the voxel mesh differences between two model frames, performing voxelization, surface reconstruction, and repair operations only on these areas, while using the old model for the remaining areas, significantly reducing computational load. The model update latency is controlled to within 200 milliseconds through optimized algorithm parallelization and memory management strategies, meeting the real-time modeling requirements under high-speed ship unloader operations.

[0031] Furthermore, the method is integrated into the central control system of the ship unloader, working in conjunction with the grab bucket posture sensor, material flow meter, and anti-collision module. The 3D cargo hold model update frequency is no less than 5 Hz, the absolute positioning error of the model is less than 30 mm, and the relative deformation detection sensitivity reaches the 10 mm level. Specifically, the central control system uses an industrial-grade embedded computer equipped with an Intel Xeon E5 processor and an NVIDIA Tesla V100 GPU, supporting multi-threaded parallel processing. The grab bucket posture sensor is a high-precision encoder with a sampling frequency of 100 Hz, providing the grab bucket's 3D position and attitude information for posture correction during model updates. The material flow meter is an ultrasonic flow meter, measuring in tons per minute, used to assess the unloading rate and assist in judging the material pile deformation trend. The anti-collision module calculates potential collision risks based on the 3D model and grab bucket motion trajectory prediction, issuing a warning signal when the risk value exceeds a threshold. The system communicates with each sensor via a CAN bus, with a data transmission delay of less than 10 milliseconds, ensuring rapid overall system response.

[0032] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for detailed three-dimensional modeling of the cargo hold of a ship unloader, characterized in that: The specific steps include the following: Step (1) Obtain multi-source heterogeneous point cloud data of cargo hold: Through multiple sets of laser scanning units installed on the main beam and grab bucket mechanism of the unloader, three-dimensional point cloud data from different perspectives are collected synchronously during cargo hold operation. The laser scanning unit adopts the principle of multi-frequency modulated continuous wave laser ranging, combined with the inertial measurement unit to compensate for the posture deviation caused by equipment vibration in real time, and obtains the original point cloud sequence with spatiotemporal synchronization. Step (2) Perform dynamic denoising and reflection suppression of point cloud: Perform statistical outlier removal based on local density distribution on the original point cloud sequence, set the number of neighborhood points and standard deviation multiple, and introduce a polarization-assisted reflection intensity correction model to suppress false point clouds caused by specular reflection of metal bulkhead and retain real geometric surface data. Step (3) Construct an adaptive multi-scale voxel mesh: Based on the consistency index of local curvature and normal vector of the point cloud, dynamically divide the voxel resolution, use voxel size for flat areas of cargo hold walls to improve processing efficiency, and use voxel size for material stack tips, corners and grab bucket disturbance areas to retain detailed features, forming a non-uniform but topologically coherent voxel representation. Step (4) Implementing incremental surface reconstruction with spatiotemporal fusion: Based on the pose association information between consecutive scan frames, an improved Poisson surface reconstruction algorithm is adopted, and a time dimension weighting factor is introduced to perform weighted fusion of the gradient fields of corresponding spatial regions in adjacent frames to generate a continuous surface model with dynamic evolution characteristics, avoiding deformation smoothing caused by static fusion. Step (5) Generate and output a refined 3D cargo hold model: perform hole identification and semantic-guided repair on the reconstructed surface model, and use local surface fitting to fill holes with a diameter of less than 200 mm by combining prior knowledge of the cargo hold structure. Finally, output a refined 3D cargo hold model containing complete geometric contours and material distribution details for the unloader path planning system to call.

2. The method for detailed three-dimensional modeling of a ship unloader's cargo hold according to claim 1, characterized in that: In step (1), the laser scanning unit includes at least 3 sets of horizontal scanners installed at the front end of the unloader beam and 2 sets of top-view scanners integrated on the top of the grab bucket. The horizontal scanners cover a horizontal angle of 270 degrees and a vertical angle of 30 degrees. The top-view scanners cover a vertical downward field of view of 90 degrees. The synchronous triggering frequency of all scanners is not less than 10 Hz, and the point cloud data output delay is less than 50 milliseconds.

3. The method for detailed three-dimensional modeling of a ship unloader's cargo hold according to claim 1, characterized in that: In step (2), the polarization-assisted reflection intensity correction model collects the echo intensity through dual polarization channels, calculates the Stokes parameter and constructs a Fresnel reflection coefficient estimator, and performs intensity attenuation compensation for high reflection areas with an incident angle greater than 60 degrees, so that the standard deviation of point cloud intensity is reduced to less than 35% of the original value, effectively distinguishing between real material surfaces and specular reflection artifacts.

4. The method for detailed three-dimensional modeling of a ship unloader's cargo hold according to claim 1, characterized in that: In step (3), the local curvature calculation of the point cloud adopts fifth-order moving least squares fitting, the neighborhood radius is adaptively set to 8 times the average spacing of the point cloud, and the consistency of the normal vector is determined by calculating the standard deviation of the angle between the normal vectors in the neighborhood. When the standard deviation is less than 5 degrees, it is determined to be a flat area; otherwise, it is determined to be a high-detail area. Based on this, the voxel size is dynamically allocated.

5. The method for detailed three-dimensional modeling of a ship unloader's cargo hold according to claim 1, characterized in that: In step (4), the time dimension weight factor is dynamically adjusted according to the inter-frame time interval and the dynamic change rate of the cargo hold. The weight factor for the static bulkhead area is set to 0.9, and the weight factor for the material pile surface area is set to 0.6, to ensure that the model can maintain structural stability and respond to the deformation of the pile, and the continuity error of the reconstructed surface normal vector is less than 3 degrees.

6. The method for detailed three-dimensional modeling of a ship unloader's cargo hold according to claim 1, characterized in that: In step (5), the hole identification adopts the visibility analysis method based on ray projection. Combined with the cabin boundary constraints defined by the cargo hold CAD structural model, the holes located in the cabin wall area are repaired by plane fitting first, and the holes located in the material pile area are repaired by quadratic surface fitting. After repair, the Gaussian curvature change rate of the model surface does not exceed 1.2 times that of the original area.

7. The method for detailed three-dimensional modeling of a ship unloader's cargo hold according to claim 1, characterized in that: It also includes semantic segmentation of the generated refined 3D cargo hold model. Based on the local geometric features and height distribution statistics of the point cloud, the refined 3D cargo hold model is divided into four semantic labels: bulkhead, bottom, material pile and empty area. The segmentation accuracy is greater than or equal to 95%, providing semantic-level environmental perception for the trajectory planning of the unloader's grab bucket.

8. The method for detailed three-dimensional modeling of a ship unloader's cargo hold according to claim 1, characterized in that: It also includes establishing a refined three-dimensional cargo hold model update mechanism. When the Chamfer distance between two consecutive frames of reconstructed refined three-dimensional cargo hold models is greater than the preset threshold of 50 mm, the incremental update process of reconstructed refined three-dimensional cargo hold models is triggered. Only the changed areas are re-executed in steps (3) to (5), and the model update delay is controlled within 200 milliseconds to meet the real-time operation requirements.

9. A method for detailed three-dimensional modeling of a ship unloader's cargo hold according to claim 1, characterized in that: The method is integrated into the central control system of the ship unloader and is linked with the grab bucket posture sensor, material flow meter and anti-collision module. The update frequency of the refined three-dimensional cargo hold model is no less than 5 Hz, the absolute positioning error of the refined three-dimensional cargo hold model is less than 30 mm, and the relative deformation detection sensitivity reaches the order of 10 mm.

10. A method for detailed three-dimensional modeling of a ship unloader's cargo hold according to claim 7, characterized in that: The semantic segmentation extracts local geometric features of each point cloud point, including curvature, rate of change of normal vector, local density, and height distribution statistical features, including height relative to the bottom of the cabin and height gradient. The feature vector dimension is 64. The feature vector is classified through a trained deep neural network model. The model adopts a convolutional neural network structure, which contains three convolutional layers and two fully connected layers. The activation function is ReLU and the loss function is cross-entropy.

Images