Bulk cargo ship hatchway measurement method and system based on ship plan and local point cloud

Abstract

The present application relates to the technical field of three-dimensional measurement and ship, and particularly relates to a bulk cargo ship hatch measuring method and system based on a ship drawing and local point cloud. The measuring method comprises the following steps: based on a 2D CAD ship drawing, automatically extracting information in the ship drawing; based on a parameterized three-dimensional modeling algorithm, acquiring color local point cloud data of a bulk cargo ship by a collecting device composed of a laser radar and a color camera, and outputting a color local point cloud subset corresponding to each feature of the ship body by using a semantic segmentation model; extracting ship outer contour feature points and color local point cloud feature points; performing regional registration to obtain a registered three-dimensional model; extracting line features of each semantic region based on the registered three-dimensional model and the color local point cloud data; performing fine correction on the registered three-dimensional model based on the line features to minimize the deviation between the line features; and automatically measuring key parameters of a hatch of the bulk cargo ship based on the fine corrected three-dimensional model and the color local point cloud data.

Description

Technical Field

[0001] This invention relates to the field of three-dimensional measurement and shipbuilding technology, specifically to a method and system for measuring hatches of bulk carriers based on ship diagrams and local point clouds. Background Technology

[0002] The dimensional accuracy of bulk carrier hatches directly impacts the efficiency of automated cargo loading and the precision of remote operations at bulk cargo ports. Existing measurement methods primarily rely on full-ship scanning with lidar and minimal manual intervention, and can only measure hatch data. This results in low efficiency, poor accuracy, and a limited range of data elements. Furthermore, these methods are susceptible to interference from environmental factors such as wind, waves, and lighting conditions, failing to adequately support the requirements for fully automated, efficient, and safe operations at bulk cargo ports. Simultaneously, existing measurement methods require dedicated time for full-ship scanning, hindering dynamic monitoring of real-time ship operational data. This impacts both the efficiency of normal port loading and unloading operations and the need for remote, high-precision, high-efficiency, and safe cargo transfer and obstacle avoidance operations. Summary of the Invention

[0003] This invention provides a method and system for measuring hatches of bulk carriers based on ship diagrams and local point clouds. It integrates 2D CAD ship diagrams and 3D point cloud data, achieves accurate separation of target areas through semantic segmentation, establishes 2D-3D association through feature registration of different areas, and completes fine correction through line feature extraction. Ultimately, it realizes fully automatic three-dimensional online intelligent measurement of bulk carriers, providing an effective way to solve the shortcomings of traditional measurement methods that mainly measure only hatch data, resulting in low efficiency, poor accuracy, and limited measurement data elements.

[0004] In a first aspect, the present invention provides a method for measuring hatches of bulk carriers based on ship charts and local point clouds, comprising the following steps: Based on 2DCAD ship drawings, the geometric contour information, dimension information and topological relationship information are automatically extracted from the ship drawings, and a three-dimensional model of the ship's outer contour is constructed based on a parametric three-dimensional modeling algorithm. Color local point cloud data of bulk carriers is acquired by a data acquisition device consisting of LiDAR and color camera. After preprocessing the color local point cloud data, a semantic segmentation model is used to output the color local point cloud subsets corresponding to each feature of the ship. Based on the three-dimensional model of the ship's outer contour, feature points of the ship's outer contour are extracted according to semantic categories; Based on the semantic segmentation model, output the color local point cloud subsets corresponding to each feature of the hull, and extract color local point cloud feature points of the same category as the feature points of the ship's outer contour. A registration algorithm is used to register the feature points of the ship's outer contour with the feature points of the local color point cloud of the same semantic region, and the registration transformation matrix of each semantic region is obtained. Based on the registration transformation matrix of each semantic region, the color local point cloud data is registered as a whole, so that the three-dimensional model of the ship's outer contour is globally aligned with the color local point cloud data, and the registered three-dimensional model is obtained. Based on the registered 3D model and color local point cloud data, line features of each semantic region are extracted; Fine-tuning of the registered 3D model is performed based on line features to minimize the deviation between line features; Based on the finely calibrated 3D model and color local point cloud data, key parameters of bulk carrier hatches are automatically measured.

[0005] As one possible implementation, preprocessing of color local point cloud data includes: Statistical filtering algorithm is used for noise reduction. A threshold is set for the number of points in the neighborhood of the color local point cloud data. Points with a number of points in the neighborhood below the threshold are identified as isolated noise points and removed. A voxel grid filtering algorithm is used for downsampling processing. A voxel grid is set up, and the denoised color local point cloud data is resampled. Coordinate normalization processing involves converting the coordinates of the colored local point cloud data to a measurement coordinate system consistent with the 3D model of the ship's outer contour by collecting attitude data from the inertial measurement unit of the equipment.

[0006] As one possible implementation, the semantic segmentation model adds an RGB feature fusion layer and an attention mechanism module to the PointNet++ base network: the RGB feature fusion layer normalizes the RGB color values ​​of the color local point cloud data to a specific range, and then concatenates them with the spatial geometric features of the color local point cloud data to form a feature vector of preset dimensions; the attention mechanism module enhances the recognition of the ship's outer contour by learning the weights of different feature channels.

[0007] As one possible approach, the ship's outer contour feature points can be extracted using the following method: Curvature analysis algorithm is used to calculate the curvature value of each point on the surface of the three-dimensional model of the ship's outer contour, and points with curvature values ​​greater than a preset threshold are identified as corner features; An edge detection algorithm is used to extract discrete points on the edges of the three-dimensional model of the ship's outer contour as edge feature points; The least squares surface fitting algorithm is used to fit the regular curved surface of the three-dimensional model of the ship's outer contour, and the feature points of the center and edge of the surface are extracted as surface feature points. Each extracted feature point is uniquely numbered, and its coordinate information in the 3D model coordinate system and its corresponding semantic category label are recorded.

[0008] As one possible implementation, color local point cloud feature points of the same category as the ship's outer contour feature points are extracted using the following method: The SIFT algorithm and line detection algorithm are combined with spatial gradient information to extract corner and line features; The RANSAC algorithm is used to extract edge feature points. By randomly sampling and fitting a straight line, points whose distance from the straight line is less than a set threshold are identified as edge points. The normal distribution transformation algorithm is used to calculate the normal distribution parameters of the local region of the color local point cloud, and the distribution center and variance abrupt change points are extracted as surface feature points.

[0009] As one possible implementation, the registration transformation matrix can be obtained through the following method: Preliminary alignment of feature points is achieved through a coarse registration algorithm based on feature descriptor matching: a pre-defined feature descriptor is constructed for each feature point; The fast approximate nearest neighbor algorithm is used to calculate the descriptor distance between the feature points of the ship's outer contour and the feature points of the colored local point cloud. Point pairs with a distance less than a threshold are judged as matching point pairs. The initial transformation matrix is ​​solved using the singular value decomposition algorithm based on the matching point pairs to achieve alignment and obtain the registration transformation matrix for each semantic region.

[0010] As one possible implementation, line features of each semantic region can be extracted using the following method: For the registered 3D model, line features are directly extracted based on its geometric parameters; For the color local point cloud data, the PCA transform algorithm is used to extract line features: the color local point cloud is projected onto three orthogonal planes XY, YZ, and XZ, and straight lines are detected on each plane through PCA transform. The straight lines detected on the three planes are projected back into three-dimensional space, and the three-dimensional line features of the color local point cloud data are obtained by fitting the spatial straight lines. The extracted line features are filtered, and line features with a length greater than a preset distance and a number of points supported by the point cloud greater than a preset number are retained, resulting in color local point cloud line features that correspond one-to-one with the line features of the registered three-dimensional model.

[0011] As one possible approach, the registered 3D model can be finely corrected using the following method: Using the distance and angle deviations between line features as optimization objectives, a least squares optimization model is constructed to finely correct the registration results; The distance deviation between line features is defined as the average of the shortest distances from each point on the colored local point cloud line feature to the corresponding model line feature; The angular deviation is the angle between the local point cloud features in the color and the model line features; Optimize the objective function by iteratively solving the optimization model using the gradient descent algorithm, fixing the iteration step size and the number of iterations, until the objective function value converges.

[0012] As one possible implementation, a measurement result verification step is also included: The key parameters of the automatic measurement are compared with the design parameters of the 2D CAD ship drawing to calculate the measurement error. If the error exceeds the preset threshold, the process returns to the step of using a registration algorithm to register the ship's outer contour feature points with the color local point cloud feature points of the same semantic meaning in different regions, and obtains the registration transformation matrix of each semantic region.

[0013] Secondly, the present invention provides a bulk carrier hatch measurement system based on ship charts and local point clouds, comprising: The building unit, based on 2D CAD ship drawings, automatically extracts geometric contour information, dimension annotation information and topological relationship information from the ship drawings, and constructs a 3D model of the ship's outer contour based on parametric 3D modeling algorithms; The point cloud data acquisition unit acquires local color point cloud data of the bulk carrier through an acquisition device composed of LiDAR and a color camera. After preprocessing the local color point cloud data, a semantic segmentation model is used to output the local color point cloud subsets corresponding to each feature of the ship. The extraction unit extracts feature points of the ship's outer contour based on the semantic category of the 3D model of the ship's outer contour. Based on the semantic segmentation model, output the color local point cloud subsets corresponding to each feature of the hull, and extract color local point cloud feature points of the same category as the feature points of the ship's outer contour. The registration unit uses a registration algorithm to register the feature points of the ship's outer contour with the feature points of the colored local point cloud of the same semantic type in different regions, and obtains the registration transformation matrix of each semantic region. Based on the registration transformation matrix of each semantic region, the colored local point cloud data is registered as a whole, so that the 3D model of the ship's outer contour is globally aligned with the colored local point cloud data, and the 2D-3D association is completed to obtain the registered 3D model. The correction unit extracts line features of each semantic region based on the registered 3D model and the color local point cloud data; and performs fine correction on the registered 3D model based on the line features to minimize the deviation between line features. The measurement unit automatically measures key parameters of the hatches of bulk carriers based on a finely calibrated 3D model and color local point cloud data.

[0014] The beneficial effects of this invention are as follows: 1. This invention integrates 2D CAD ship drawings and 3D point cloud data. It acquires local color point cloud data of bulk carriers in real time through a collection device composed of LiDAR and color cameras. It achieves precise separation of target areas through semantic segmentation, establishes 2D-3D association through regional feature registration, and completes fine correction through line feature extraction. This enables fully automatic three-dimensional online intelligent measurement of bulk carriers, with a large number of measurement data elements, resulting in high accuracy.

[0015] 2. This invention establishes a three-dimensional model based on 2D CAD ship drawings and 3D point cloud data to automatically measure key parameters of bulk carrier hatches. The acquisition device, consisting of a lidar and a color camera, acquires local color point cloud data of the bulk carrier for non-contact measurement, eliminating the need for measurement personnel to board the ship or work near the hatch edge, thus completely avoiding safety risks.

[0016] 3. The present invention employs preprocessing including denoising and downsampling: denoising using a statistical filtering algorithm can remove isolated noise points, and downsampling using a voxel grid filtering algorithm, by setting a voxel grid and resampling the denoised color local point cloud data, can reduce the point cloud density.

[0017] 4. This invention uses a semantic segmentation model based on a multimodal point cloud semantic segmentation model improved from PointNet++ or other advanced point cloud segmentation models, which integrates RGB color features and spatial geometric features. The attention mechanism module learns the weights of different feature channels and strengthens the features that are effective for identifying ship structures, thereby improving the segmentation accuracy.

[0018] 5. This invention calculates the measurement error by comparing the key parameters of automatic measurement with the design parameters of 2D CAD ship drawings. If the error exceeds a preset threshold, it returns to the step of using a registration algorithm to register the ship's outer contour feature points with the feature points of the colored local point cloud of the same semantic meaning in different regions, and obtains the registration transformation matrix of each semantic region; a measurement result verification step is set to make the measurement results more accurate. Attached Figure Description

[0019] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this invention, illustrate exemplary embodiments of the invention and are used to explain the invention, but do not constitute an undue limitation of the invention. In the drawings: Figure 1 This is a flowchart of the bulk carrier hatch measurement method based on ship diagrams and local point clouds. Figure 2 These are comparison images showing the construction of a 3D model of a ship's outer contour based on 2D CAD ship drawings according to the present invention. Figure 2 (a) is a side view of a 2D CAD ship drawing. Figure 2 (b) is the top view of a 2D CAD ship drawing. Figure 2 (c) is a 3D model of the ship constructed using a parametric 3D modeling algorithm; Figure 3 This invention outputs a color local point cloud subset image corresponding to each feature of the ship's hull based on a semantic segmentation model. Figure 4 This is a diagram illustrating the process of extracting ship outer contour feature points according to semantic categories in this invention. Detailed Implementation

[0020] To make the technical problems to be solved, the technical solutions, and the beneficial effects of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present invention and are not intended to limit the present invention.

[0021] It should be noted that when a component is referred to as "fixed to" or "set on" another component, it can be directly on or indirectly on that other component. When a component is referred to as "connected to" another component, it can be directly connected to or indirectly connected to that other component.

[0022] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0023] In the description of this invention, it should be understood that the terms "upper" and "lower" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limiting this invention.

[0024] The present invention aims to provide a method and system for measuring hatches of bulk carriers based on ship diagrams and local point clouds. It integrates 2D CAD ship diagrams and 3D point cloud data, achieves accurate separation of target areas through semantic segmentation, establishes 2D-3D association through regional feature registration, and completes fine correction through line feature extraction, ultimately realizing fully automatic three-dimensional online intelligent measurement of bulk carriers.

[0025] In a first aspect, embodiments of the present invention provide a method for measuring hatch openings of bulk carriers based on ship charts and local point clouds, see [link to relevant documentation]. Figures 1-4 This includes the following steps: Based on 2DCAD ship drawings, the geometric contour information, dimension information and topological relationship information are automatically extracted from the ship drawings, and a three-dimensional model of the ship's outer contour is constructed based on a parametric three-dimensional modeling algorithm. One possible approach is to acquire 2D CAD drawings of the target bulk carrier. These drawings must include the general arrangement of the hull, hatch cover arrangement, and superstructure structure, ensuring the inclusion of geometric contours, dimensions, and topological relationships of key structures such as hatches, hatch covers, superstructure, and masts. The CAD parsing algorithm reads the DXF or DWG format files of the drawings. This algorithm automatically identifies graphic elements such as lines, arcs, and polygons, separates contour lines, dimension lines, and annotation text, and extracts geometric contour information such as the hull outline, hatch design outline (including length, width, and chamfer dimensions), hatch cover installation outline, superstructure outline, and mast arrangement outline. Simultaneously, natural language processing technology is used to parse the annotation text, extracting dimensional data such as length, width, spacing, and installation height for each structure, and deriving the topological connections between components (such as the alignment between hatches and hatch covers, and the fixed positional relationship between the superstructure and the hull) through the relationships between graphic elements.

[0026] Based on the extracted geometric contour information, dimensioning information, and topological relationship information, a parametric 3D modeling algorithm is used to construct a 3D ship model. Using the design coordinate system of the 2D CAD ship drawing (usually a right-handed coordinate system composed of the hull centerline plane, baseline plane, and design waterline plane) as a reference, the 2D contours of each structure are extruded according to dimensions (e.g., hatches are extruded vertically to the design height), rotated (e.g., masts are rotated along their axis to generate a cylindrical structure), or lofted (e.g., the 3D shaping of irregular contours of the superstructure) to generate 3D solids. Using topological relationship information, the models of components such as hatches, hatch covers, superstructures, and masts are assembled into a complete 3D ship model according to their actual installation positions. During assembly, dimensional constraint verification ensures that the relative positional error between components is no greater than ±0.1mm, ultimately forming a 3D model consistent with the actual ship structure, providing a precise reference framework for subsequent registration and measurement.

[0027] A comparison of constructing a 3D model of a ship's outer contour based on 2D CAD drawings, such as... Figure 2 As shown, (a) is a side view of the 2D CAD ship drawing, (b) is a top view of the 2D CAD ship drawing, and (c) is a 3D ship model constructed using a parametric 3D modeling algorithm.

[0028] Color local point cloud data of bulk carriers is acquired by a data acquisition device consisting of LiDAR and color camera. After preprocessing the color local point cloud data, a semantic segmentation model is used to output the color local point cloud subsets corresponding to each feature of the ship. One possible approach is to employ a multi-sensor fusion acquisition device consisting of lidar and color cameras. While the ship is docked or anchored, the acquisition device, mounted on port machinery (such as ship loaders and gantry cranes), performs online scanning of the bulk carrier to acquire real-time color local point cloud data. This data includes both spatial geometric location information (X, Y, Z coordinates) and RGB color information (24-bit color resolution), comprehensively reflecting the ship's structural appearance and spatial distribution patterns. A semantic segmentation model is used to output color local point cloud subsets corresponding to hatches, hatch covers, superstructures, and masts.

[0029] Based on the semantic segmentation model, the model outputs a subset of colored local point clouds corresponding to each feature of the hull, such as... Figure 3 As shown.

[0030] Based on the three-dimensional model of the ship's outer contour, feature points of the ship's outer contour are extracted according to semantic categories. For example, feature points of hatches, hatch covers, superstructures, and masts are extracted according to semantic categories. The feature points include corner points, straight line feature points, and planar features. Based on the semantic segmentation model, a subset of colored local point clouds corresponding to each feature of the hull is output, and colored local point cloud feature points of the same category as the feature points of the ship's outer contour are extracted to ensure the consistency of feature types. A registration algorithm is used to register the feature points of the ship's outer contour with the feature points of the local color point cloud of the same semantic region, and the registration transformation matrix of each semantic region is obtained. Based on the registration transformation matrix of each semantic region, the color local point cloud data is registered as a whole, so that the three-dimensional model of the ship's outer contour is globally aligned with the color local point cloud data, completing the 2D-3D association and obtaining the registered three-dimensional model. Based on the registered 3D model and color local point cloud data, line features of each semantic region are extracted; for example, line features include hatch edge lines, hatch cover outline lines, superstructure edge lines and mast centerline, etc. Fine-tuning of the registered 3D model is performed based on line features to minimize the deviation between line features; Based on a finely calibrated 3D model and color local point cloud data, key parameters of the bulk carrier hatch are automatically measured. For example, key parameters include hatch length, width, edge flatness, center coordinates, and the gap between the hatch and the hatch cover.

[0031] As one possible implementation, preprocessing of color local point cloud data includes: Statistical filtering algorithm is used for noise reduction. A threshold is set for the number of points in the neighborhood of the color local point cloud data. Points with a number of points in the neighborhood below the threshold are identified as isolated noise points and removed. For example, the threshold for the number of points in the neighborhood of the point cloud (neighborhood radius 5cm) is set to 10. Points with fewer than 10 points in the neighborhood are identified as isolated noise points and removed to eliminate noise caused by interference factors such as air dust and birds during the scanning process. A voxel grid filtering algorithm is used for downsampling processing. A voxel grid is set up, and the denoised color local point cloud data is resampled. For example, the voxel grid size is set to 1cm×1cm×1cm, and the denoised point cloud is resampled. While ensuring that the point cloud features are not lost, the amount of point cloud data is reduced by 60%-70%, which improves the processing efficiency of subsequent semantic segmentation and registration. Coordinate normalization processing involves converting the coordinates of the color local point cloud data to a measurement coordinate system consistent with the 3D model of the ship's outer contour by collecting attitude data from the inertial measurement unit of the equipment. This eliminates registration errors caused by differences in coordinate systems and ensures that the coordinate references of the two are consistent.

[0032] As one possible implementation, the semantic segmentation model adds an RGB feature fusion layer and an attention mechanism module to the PointNet++ base network: the RGB feature fusion layer normalizes the RGB color values ​​of the color local point cloud data to a specific range, and then concatenates them with the spatial geometric features of the color local point cloud data to form a feature vector of preset dimensions; the attention mechanism module enhances the recognition of the ship's outer contour by learning the weights of different feature channels.

[0033] For example, the RGB feature fusion layer normalizes the RGB color values ​​of the colored point cloud to the [0,1] interval, and then concatenates them with the spatial geometric features of the point cloud (normal vector, curvature, and variance of the distance between neighboring points) to form a 12-dimensional feature vector. The attention mechanism module learns the weights of different feature channels to strengthen features that are effective for identifying ship structures (such as the color difference features between hatches and hatch covers, and the angular geometric features of the superstructure). After the model is trained on a point cloud dataset containing 100 bulk carriers (each ship contains 5 million to 8 million points, covering targets such as hatches, hatch covers, superstructures, and masts under different lighting and weather conditions), the segmentation accuracy of each target region is no less than 95%. It can automatically identify and label each target region, and output point cloud subsets of hatch semantic regions, hatch cover semantic regions, superstructure semantic regions, and mast semantic regions, achieving accurate separation of target regions from the background.

[0034] As one possible approach, the feature points of the ship's outer contour can be extracted using the following method: Feature points are extracted according to semantic categories (hatch, hatch cover, superstructure, mast), and curvature analysis algorithm is used to calculate the curvature value of each point on the surface of the three-dimensional model of the ship's outer contour. Points with curvature values ​​greater than a preset threshold are identified as corner features. An edge detection algorithm is used to extract discrete points on the edges of the three-dimensional model of the ship's outer contour as edge feature points; The least squares surface fitting algorithm is used to fit regular curved surfaces such as planes and cylinders on the surface of the three-dimensional model of the ship's outer contour, and the feature points of the center and edge of the surface are extracted as surface feature points. All extracted feature points are uniquely numbered, and their coordinate information in the 3D model coordinate system and their corresponding semantic category labels (such as "top left corner of hatch" and "right edge of superstructure") are recorded. The number of feature points for each semantic category is no less than 50 to ensure the reliability of registration.

[0035] The process of extracting ship outer contour feature points according to semantic categories, such as Figure 4 As shown.

[0036] As one possible implementation, color local point cloud feature points of the same category as the ship's outer contour feature points are extracted using the following method: For each semantic region's segmented colored local point cloud subset, an algorithm corresponding to the feature point extraction of the 3D model is used to extract feature points of the same category. For the hatch semantic region point cloud and line subset, the SIFT algorithm and PCA line detection algorithm are combined with spatial gradient information to extract corner points and line features. The RANSAC algorithm is used to extract edge feature points. By randomly sampling and fitting a straight line, points whose distance from the straight line is less than a set threshold are identified as edge points. A normal distribution transformation algorithm is used to calculate the normal distribution parameters of local regions in the color local point cloud, and the distribution center and variance abrupt change points are extracted as surface feature points. It is ensured that the type and distribution pattern of the point cloud feature points of each semantic category are consistent with the feature points of the 3D model (e.g., the number and relative position of hatch corner feature points match the corresponding feature points in the 3D model), and the ratio of point cloud feature points to 3D model feature points is 1.2:1, meeting the feature matching requirements in the registration process.

[0037] As one possible implementation, the registration transformation matrix can be obtained through the following method: Preliminary alignment of feature points is achieved through a coarse registration algorithm based on feature descriptor matching: a pre-defined feature descriptor is constructed for each feature point; For example, a 128-dimensional feature descriptor (integrating coordinates, normal vectors, and curvature information) is constructed for each feature point. The Fast Approximate Nearest Neighbor (FLANN) algorithm is used to calculate the descriptor distance between the feature points of the ship's outer contour and the feature points of the colored local point cloud. Point pairs with a distance less than a threshold (empirical threshold of 0.1) are identified as matching point pairs. Based on the matching point pairs, the singular value decomposition (SVD) algorithm is used to solve the initial transformation matrix (including the rotation matrix R and the translation vector T) to achieve preliminary alignment. After coarse registration, the average Euclidean distance between feature points is no greater than a set threshold. Then, fine registration is performed: using the sum of Euclidean distances between feature points as the objective function, an optimization equation is constructed; The optimization equations are solved using the Levenberg-Marquardt iterative algorithm. The convergence threshold is dynamically adjusted during the iteration process until the number of iterations reaches a preset number or the average Euclidean distance between feature points is less than a preset distance, thus obtaining the registration transformation matrix of each semantic region.

[0038] For example, using the sum of Euclidean distances between feature points as the objective function, an optimization equation is constructed: ; in, The coordinates of feature points in the 3D model. Let be the coordinates of the feature points in the point cloud, and n be the number of matching point pairs. The optimization equation is solved using the Levenberg-Marquardt iterative algorithm. During the iteration process, the convergence threshold is dynamically adjusted (initially 1 mm, decreasing to 0.8 times the previous threshold every 5 iterations) until the number of iterations reaches 100 or the average Euclidean distance between feature points is less than 20 mm, thus obtaining the optimal registration transformation matrix for each semantic region.

[0039] As one possible approach, the color local point cloud data is optimized and registered globally based on the registration transformation matrices of each semantic region. A graph optimization algorithm is used to construct a global optimization model, with the registration transformation matrices of each semantic region as nodes and the topological constraints between regions (such as the relative positional constraints between hatches and hatch covers, and the fixed relationship constraints between the superstructure and the hull) as edges. By minimizing the global error function, the registration results of each region are fused, eliminating the accumulation of local registration errors. After global registration, the global alignment error between the ship's 3D model and the color local point cloud is no greater than ±50mm, achieving a precise association between the 2D CAD ship drawing (transferred through the 3D model) and the 3D point cloud. This gives the point cloud data design parameter constraints, providing a reliable foundation for subsequent fine correction and measurement.

[0040] As one possible implementation, line features of each semantic region can be extracted in the following way: For the registered 3D model, line features are directly extracted based on its geometric parameters: the hatch edge lines are the four straight edges of the hatch outline in the model, the hatch cover outline is the closed broken line of the hatch cover shape, the superstructure edge lines are the intersection lines of the superstructure surfaces, and the mast centerline is the central axis of the mast cylindrical structure. All model line features are stored in the form of a set of straight line segments (each straight line segment contains the coordinates of the start and end points).

[0041] For the colored local point cloud data, the PCA transform algorithm is used to extract line features: the colored local point cloud is projected onto three orthogonal planes XY, YZ, and XZ, and straight lines are detected on each plane through PCA transform. The straight lines detected on the three planes are projected back into three-dimensional space, and the three-dimensional line features of the point cloud are obtained by fitting the spatial straight lines. The extracted line features are filtered, and line features with a length greater than a preset distance and a number of points supported by the point cloud greater than a preset number are retained, so as to obtain colored local point cloud line features that correspond one-to-one with the line features of the registered three-dimensional model. For example, the extracted line features are filtered to retain those with a length greater than 100cm and a point cloud support of more than 100, and finally point cloud line features (hatch edge lines, hatch outline lines, etc.) that correspond one-to-one with the model line features are obtained.

[0042] As one possible approach, the registered 3D model can be finely corrected using the following method: Using the distance and angle deviations between line features as optimization objectives, a least squares optimization model is constructed to finely correct the registration results; The distance deviation between line features is defined as the average of the shortest distances from each point on the colored local point cloud line feature to the corresponding model line feature; The angular deviation is the angle between the local point cloud features in the color and the model line features; Optimize the objective function by iteratively solving the optimization model using the gradient descent algorithm, with a fixed iteration step size and number of iterations, until the objective function value converges. For example, the objective function to be optimized is: ; in, Let be the distance deviation of the corresponding line feature in the j-th group. The angle deviation of the corresponding line feature in the j-th group is m, and the number of line feature groups is m=12 in this embodiment, including 4 groups of hatch edge lines, 4 groups of hatch cover outline lines, 2 groups of superstructure edge lines, and 2 groups of mast centerline. (Weight value 10) and These are the weighting coefficients for distance deviation and angle deviation, respectively. The optimization model is iteratively solved using the gradient descent algorithm, with a fixed iteration step size and number of iterations, until the objective function value converges. After fine correction, the distance deviation between line features is no greater than ±20mm, and the angle deviation is no greater than ±1 radian, further improving 2D-3D alignment accuracy.

[0043] As one possible implementation method, it also includes a measurement result verification step: comparing the key parameters of automatic measurement with the design parameters of 2DCAD ship drawings, calculating the measurement error, and if the error exceeds the preset threshold, returning to the step of using a registration algorithm to perform regional registration of the ship's outer contour feature points and the color local point cloud feature points of the same semantic meaning, and obtaining the registration transformation matrix of each semantic region.

[0044] For example, the key parameters of the hatch obtained by automatic measurement are compared with the design parameters in the 2D CAD ship drawing, and the measurement error is calculated. If the measurement error of all key parameters is less than or equal to the preset threshold, the measurement result is deemed qualified, and the final measurement result is output; if the measurement error of any parameter exceeds the preset threshold, the feature descriptor matching threshold of coarse registration and the convergence threshold of fine registration are adjusted, and registration optimization is performed again until the measurement error of all parameters meets the accuracy requirements.

[0045] For example, key parameters of bulk carrier hatches are automatically measured: Hatch length and width: Extract the finely corrected hatch edge lines (point cloud features), and use the least squares method to fit four edge lines. Calculate the distance between two relative edge lines, which are used as the hatch length (along the longitudinal direction of the hull) and width (along the transverse direction of the hull), respectively. Hatch height: For each hatch opening, calculate the edge line height and optimize the measured hatch height value as the output based on the consistency constraint of all hatch heights; Hatch center coordinates: calculated by measuring the geometric center coordinates of the rectangular area formed by the four edge lines of the hatch. Output the center coordinates based on the global coordinate system; Dimensions and coordinates of auxiliary buildings: Extract the outline of the superstructure and mast, calculate the dimensions of the outline bounding box (calculate the length, width and height along the direction perpendicular to the hatch plane, and output the three-dimensional coordinates of the center point).

[0046] The key hatch parameters obtained by automatic measurement are compared with the design parameters in the 2D CAD ship drawing to calculate the measurement error. If the measurement error of all key parameters is less than or equal to the preset threshold, the measurement result is deemed qualified and the final measurement result is output; if the measurement error of any parameter exceeds the preset threshold, the process returns to adjust the feature descriptor matching threshold of coarse registration and the convergence threshold of fine registration, and registration optimization is performed again until the measurement error of all parameters meets the accuracy requirements.

[0047] according to Figures 1-4 As shown, this invention integrates 2D CAD ship diagrams and 3D point cloud data. It acquires local color point cloud data of bulk carriers in real time through a collection device composed of LiDAR and color cameras. It achieves precise separation of target areas through semantic segmentation, establishes 2D-3D association through regional feature registration, and completes fine correction through line feature extraction. This enables fully automatic three-dimensional online intelligent measurement of bulk carriers, with a large number of measurement data elements, resulting in high accuracy.

[0048] Secondly, the present invention provides a bulk carrier hatch measurement system based on ship charts and local point clouds, specifically including: The building unit, based on 2D CAD ship drawings, automatically extracts geometric contour information, dimension annotation information and topological relationship information from the ship drawings, and constructs a 3D model of the ship's outer contour based on parametric 3D modeling algorithms; The point cloud data acquisition unit acquires local color point cloud data of the bulk carrier through an acquisition device composed of LiDAR and a color camera. After preprocessing the local color point cloud data, a semantic segmentation model is used to output the local color point cloud subsets corresponding to each feature of the ship. The extraction unit extracts feature points of the ship's outer contour based on the semantic category of the 3D model of the ship's outer contour. Based on the semantic segmentation model, output the color local point cloud subsets corresponding to each feature of the hull, and extract color local point cloud feature points of the same category as the feature points of the ship's outer contour. The registration unit uses a registration algorithm to register the feature points of the ship's outer contour with the feature points of the colored local point cloud of the same semantic type in different regions, and obtains the registration transformation matrix of each semantic region. Based on the registration transformation matrix of each semantic region, the colored local point cloud data is registered as a whole, so that the 3D model of the ship's outer contour is globally aligned with the colored local point cloud data, and the 2D-3D association is completed to obtain the registered 3D model. The correction unit extracts line features of each semantic region based on the registered 3D model and the color local point cloud data; and performs fine correction on the registered 3D model based on the line features to minimize the deviation between line features. The measurement unit automatically measures key parameters of the hatches of bulk carriers based on a finely calibrated 3D model and color local point cloud data.

[0049] For example, taking a 50,000-ton bulk carrier (180m long, 30m wide, with 5 cargo holds, each hatch approximately 20m × 15m in size) as an example, the method of this invention is used for measurement, and the specific implementation process is as follows: (1) Data preparation: Obtain the 2D CAD ship drawings (PDF format) of the bulk carrier, including the general arrangement drawing of the hull, the hatch cover arrangement drawing and the superstructure structure drawing. The design coordinate system of the ship drawings is a right-handed coordinate system composed of the centerline plane (X=0), the baseline plane (Z=0) and the design waterline plane (Z=8m). (2) Ship 3D model reconstruction: The design outline (length 20m, width 15m, chamfer radius 0.5m), dimension annotation and topological relationship of 5 hatches in the ship drawing are automatically extracted by CAD analysis algorithm. The ship 3D model is reconstructed offline by parametric modeling algorithm, and the data is saved one by one according to the ship model. The size error of the hatch structure in the model is no more than ±2mm compared with the design value, and the coordinate system is consistent with the design coordinate system of the ship drawing. (3) Point cloud preprocessing and semantic segmentation: Using a Lanwo lidar and an IMX214 color camera, the local three-dimensional color point cloud data of the ship is acquired in real time when the ship just enters the dock. At the same time, the ship three-dimensional model reconstructed from the ship map is automatically called according to the ship model. The collected point cloud data is denoised (about 3% of noise points are removed), downsampled (the data volume is reduced to 120 million points) and coordinate normalized. The improved PointNet++ semantic segmentation model is used to segment the preprocessed local point cloud and output the segmented point cloud subsets of each target area such as hatch, hatch cover, superstructure and mast. (4) Feature point extraction and registration: Feature points and feature lines are extracted from the CAD reconstructed 3D model and the point cloud subset respectively (about 80 feature points and 4 lines are extracted from each hatch). The improved ICP algorithm is used for real-time registration in different regions. After coarse registration, the average distance of feature points is 50mm, and after fine registration, it is reduced to 20mm. Global registration is performed by graph optimization algorithm, and the global alignment error is better than 20mm. (5) Line feature extraction and fine correction: 12 sets of line features were extracted, including 4 edge lines of each hatch and 4 outline lines of the hatch cover. The least squares optimization model was constructed for fine correction. After correction, the line feature distance deviation was 10 mm and the angle deviation was 1 radian. (6) Measurement and verification of key parameters: Based on the registration results, the length, width and dimensions of the hatches, auxiliary buildings and masts, as well as the center coordinates of the five hatches are automatically measured, and the measurement results are compared with the CAD design values.

[0050] Although the invention has been described herein in conjunction with various embodiments, those skilled in the art will understand and implement other variations of the disclosed embodiments by reviewing the accompanying drawings, the disclosure, and the description of the drawings, in carrying out the claimed invention. In this specification, the word "comprising" does not exclude other components or steps, and "a" or "an" does not exclude multiple components. A single processor or other unit can implement several of the functions listed in the specification. While certain measures are described in different embodiments, this does not mean that these measures cannot be combined to produce good results.

[0051] Although the invention has been described in conjunction with specific features and embodiments, it is obvious that various modifications and combinations can be made therein without departing from the spirit and scope of the invention. Accordingly, this specification and drawings are merely illustrative of the invention and are considered to cover any and all modifications, variations, combinations, or equivalents within the scope of the invention. Clearly, those skilled in the art can make various alterations and modifications to the invention without departing from its spirit and scope. Thus, if such modifications and modifications fall within the scope of the invention and its equivalents, the invention is also intended to include such modifications and modifications.

Claims

1. A method for measuring hatch openings of bulk carriers based on ship charts and local point clouds, characterized in that, Includes the following steps: Based on 2DCAD ship drawings, the geometric contour information, dimension information and topological relationship information are automatically extracted from the ship drawings, and a three-dimensional model of the ship's outer contour is constructed based on a parametric three-dimensional modeling algorithm. Color local point cloud data of bulk carriers is acquired by a data acquisition device consisting of LiDAR and color camera. After preprocessing the color local point cloud data, a semantic segmentation model is used to output the color local point cloud subsets corresponding to each feature of the ship. Based on the three-dimensional model of the ship's outer contour, feature points of the ship's outer contour are extracted according to the semantic categories output by the semantic segmentation model. Based on the semantic segmentation model, output the color local point cloud subsets corresponding to each feature of the hull, and extract color local point cloud feature points of the same category as the feature points of the ship's outer contour. A registration algorithm is used to register the feature points of the ship's outer contour with the feature points of the local color point cloud of the same semantic region, and the registration transformation matrix of each semantic region is obtained. Based on the registration transformation matrix of each semantic region, the color local point cloud data is registered as a whole, so that the three-dimensional model of the ship's outer contour is globally aligned with the color local point cloud data, and the registered three-dimensional model is obtained. Based on the registered 3D model and color local point cloud data, line features of each semantic region are extracted; Fine-tuning of the registered 3D model is performed based on line features to minimize the deviation between line features; Based on the finely calibrated 3D model and color local point cloud data, key parameters of bulk carrier hatches are automatically measured.

2. The method for measuring hatches of bulk carriers based on ship charts and local point clouds according to claim 1, characterized in that, Preprocessing of color local point cloud data includes: Statistical filtering algorithm is used for noise reduction. A threshold is set for the number of points in the neighborhood of the color local point cloud data. Points with a number of points in the neighborhood below the threshold are identified as isolated noise points and removed. A voxel grid filtering algorithm is used for downsampling processing. A voxel grid is set up, and the denoised color local point cloud data is resampled. Coordinate normalization processing involves converting the coordinates of the colored local point cloud data to a measurement coordinate system consistent with the 3D model of the ship's outer contour by collecting attitude data from the inertial measurement unit of the equipment.

3. The method for measuring hatches of bulk carriers based on ship charts and local point clouds according to claim 2, characterized in that, The semantic segmentation model adds an RGB feature fusion layer and an attention mechanism module to the PointNet++ basic network: the RGB feature fusion layer normalizes the RGB color values ​​of the color local point cloud data to a specific range, and then concatenates them with the spatial geometric features of the color local point cloud data to form a feature vector of preset dimensions. The attention mechanism module enhances the recognition of the ship's outer contour by learning the weights of different feature channels.

4. The method for measuring hatches of bulk carriers based on ship charts and local point clouds according to claim 1, characterized in that, The following method is used to extract the feature points of the ship's outer contour: Curvature analysis algorithm is used to calculate the curvature value of each point on the surface of the three-dimensional model of the ship's outer contour, and points with curvature values ​​greater than a preset threshold are identified as corner features; An edge detection algorithm is used to extract discrete points on the edges of the three-dimensional model of the ship's outer contour as edge feature points; The least squares surface fitting algorithm is used to fit the regular curved surface of the three-dimensional model of the ship's outer contour, and the feature points of the center and edge of the surface are extracted as surface feature points. Each extracted feature point is uniquely numbered, and its coordinate information in the 3D model coordinate system and its corresponding semantic category label are recorded.

5. The method for measuring hatches of bulk carriers based on ship charts and local point clouds according to claim 4, characterized in that, The following method is used to extract color local point cloud feature points of the same category as the feature points of the ship's outer contour: The SIFT algorithm and line detection algorithm are combined with spatial gradient information to extract corner and line features; The RANSAC algorithm is used to extract edge feature points. By randomly sampling and fitting a straight line, points whose distance from the straight line is less than a set threshold are identified as edge points. The normal distribution parameters of the local region of the color local point cloud are calculated by using the normal distribution transformation algorithm, and the distribution center and variance abrupt change points are extracted as surface feature points.

6. The method for measuring hatches of bulk carriers based on ship charts and local point clouds according to claim 1, characterized in that, The registration transformation matrix is ​​obtained using the following method: Preliminary alignment of feature points is achieved through a coarse registration algorithm based on feature descriptor matching: a pre-defined feature descriptor is constructed for each feature point; The fast approximate nearest neighbor algorithm is used to calculate the descriptor distance between the feature points of the ship's outer contour and the feature points of the colored local point cloud. Point pairs with a distance less than a threshold are judged as matching point pairs. The initial transformation matrix is ​​solved using the singular value decomposition algorithm based on the matching point pairs to achieve alignment and obtain the registration transformation matrix for each semantic region.

7. The method for measuring hatches of bulk carriers based on ship charts and local point clouds according to claim 1, characterized in that, Line features of each semantic region are extracted using the following method: For the registered 3D model, line features are directly extracted based on its geometric parameters; For the color local point cloud data, the PCA transform algorithm is used to extract line features: the color local point cloud is projected onto three orthogonal planes XY, YZ, and XZ, and straight lines are detected on each plane through PCA transform. The straight lines detected on the three planes are projected back into three-dimensional space, and the three-dimensional line features of the color local point cloud data are obtained by fitting the spatial straight lines. The extracted line features are filtered, and line features with a length greater than a preset distance and a number of points supported by the point cloud greater than a preset number are retained, resulting in color local point cloud line features that correspond one-to-one with the line features of the registered three-dimensional model.

8. The method for measuring hatches of bulk carriers based on ship charts and local point clouds according to claim 1, characterized in that, The registered 3D model was finely corrected using the following method: Using the distance and angle deviations between line features as optimization objectives, a least squares optimization model is constructed to finely correct the registration results; The distance deviation between line features is defined as the average of the shortest distances from each point on the colored local point cloud line feature to the corresponding model line feature; The angular deviation is the angle between the local point cloud features in the color and the model line features; Optimize the objective function by iteratively solving the optimization model using the gradient descent algorithm, fixing the iteration step size and number of iterations until the objective function value converges.

9. The method for measuring hatches of bulk carriers based on ship charts and local point clouds according to any one of claims 1-8, characterized in that, It also includes a measurement result verification step: The key parameters of the automatic measurement are compared with the design parameters of the 2D CAD ship drawing to calculate the measurement error. If the error exceeds the preset threshold, the process returns to the step of using a registration algorithm to register the ship's outer contour feature points with the color local point cloud feature points of the same semantic meaning in different regions, and obtains the registration transformation matrix of each semantic region.

10. A bulk carrier hatch measurement system based on ship charts and local point clouds, characterized in that, include: The building unit, based on 2D CAD ship drawings, automatically extracts geometric contour information, dimension annotation information and topological relationship information from the ship drawings, and constructs a 3D model of the ship's outer contour based on parametric 3D modeling algorithms; The point cloud data acquisition unit acquires local color point cloud data of the bulk carrier through an acquisition device composed of LiDAR and a color camera. After preprocessing the local color point cloud data, a semantic segmentation model is used to output the local color point cloud subsets corresponding to each feature of the ship. The extraction unit extracts feature points of the ship's outer contour based on the three-dimensional model of the ship's outer contour and according to the semantic categories output by the semantic segmentation model. Based on the semantic segmentation model, output the color local point cloud subsets corresponding to each feature of the hull, and extract color local point cloud feature points of the same category as the feature points of the ship's outer contour. The registration unit uses a registration algorithm to register the feature points of the ship's outer contour with the feature points of the colored local point cloud of the same semantic meaning in different regions, and obtains the registration transformation matrix of each semantic region. Based on the registration transformation matrix of each semantic region, the colored local point cloud data is registered as a whole, so that the 3D model of the ship's outer contour is globally aligned with the colored local point cloud data, and the registered 3D model is obtained. The correction unit extracts line features of each semantic region based on the registered 3D model and the color local point cloud data; and performs fine correction on the registered 3D model based on the line features to minimize the deviation between line features. The measurement unit automatically measures key parameters of the hatches of bulk carriers based on a finely calibrated 3D model and color local point cloud data.

Images