Unmanned ship berth key target identification method based on semantic-geometry-topology fusion

Abstract

The invention discloses an unmanned ship berth key target identification method based on semantic-geometry-topology fusion, and the method integrates the steps of point cloud preprocessing, lightweight semantic segmentation and geometric constraint, berth semantic topology modeling, obstacle detection and three-dimensional modeling, and identification result enhancement and verification. The method is used for realizing robust target identification required by berthing of the unmanned ship in a complex port environment, can accurately identify key targets such as a berth front edge, a buoy, a bollard and a ship, also corrects a detection result through geometric and semantic constraints, ensures identification robustness and availability, and improves the accuracy of target identification. And reliable support is provided for path planning and berthing control of the unmanned ship.

Description

Technical Field

[0001] This invention relates to the field of intelligent unmanned smart ship technology, specifically to a method for identifying key targets in unmanned vessel berths based on semantic-geometric-topology fusion. Background Technology

[0002] Existing network architectures for berth target detection use the PointPillars method with bird's-eye view (BEV) as input and estimate oriented bounding boxes for possible target categories (ships, containers, and buoys in this study). However, existing berth target monitoring technologies still have the following shortcomings: 1. Point cloud recognition relies solely on deep networks and lacks geometric constraints. (1) Existing point cloud semantic segmentation methods (such as PointNet++, SPVCNN, etc.) are easily affected by noise points, point cloud sparsity and occlusion in complex port environments.

[0003] (2) Reasons for defects: Relying solely on semantic segmentation networks and lacking geometric prior constraints leads to deviations or misjudgments in the identification of key targets at berths (such as berth front, buoys, and bollards).

[0004] 2. Semantic relationships between targets are not utilized, resulting in a lack of topological consistency. (1) Most existing methods treat the targets as independent individuals and do not model the spatial and semantic relationships between key targets in the berth.

[0005] (2) Reasons for defects: The topological logic such as "the bollard should be close to the front edge of the berth" or "the buoy should be aligned with the direction of the berth" is not considered, which may lead to isolated points or semantic conflicts in the identification results.

[0006] 3. Obstacle modeling is crude and lacks uncertainty estimation. (1) Most existing detection methods only output 3D bounding boxes and do not model the confidence or uncertainty of the recognition results.

[0007] (2) Reasons for the defect: The lack of characterization of detection reliability makes it impossible for unmanned surface vessels to effectively distinguish between high-risk and low-risk targets during path planning, which poses a safety hazard.

[0008] Therefore, there is an urgent need for a key target identification method for unmanned surface vessels (USVs) based on semantic-geometric-topological fusion, which can accurately identify key targets such as berth fronts, buoys, bollards, and ships, and correct the detection results through geometric and semantic constraints to ensure the robustness and usability of identification, providing reliable support for USV path planning and berthing control. Summary of the Invention

[0009] The purpose of this invention is to overcome the shortcomings of the prior art and provide a key target identification method for unmanned surface vessels (USVs) based on semantic-geometric-topology fusion. This method can accurately identify key targets such as berth fronts, buoys, bollards, and ships. By correcting the detection results through geometric and semantic constraints, the robustness and usability of the identification are ensured, providing reliable support for USV path planning and berthing control.

[0010] To achieve the above-mentioned objectives, the technical solution adopted by the present invention is as follows: A method for identifying key targets in unmanned surface vessel (USV) berths based on semantic-geometric-topological fusion, comprising the following steps: S1. Point cloud preprocessing: Multiple frames of point cloud data are continuously acquired using a shipborne 3D lidar. After data filtering, noise reduction, and motion distortion compensation, high-quality, dense, and uniform temporal sparse point cloud is output. S2. Lightweight Semantic Segmentation and Geometric Constraints: The point cloud is encoded, lightweight semantic features are extracted, and geometric constraints are introduced for berth front targets, buoy targets, and bollard targets. Through semantic-geometric consistency fusion, the category, location, and boundary of key targets in the unmanned surface vessel berthing scenario are output. S3. Berth Semantic Topology Modeling: First, key targets in the berth environment are extracted and used as nodes in the topology graph. Then, relational constraints are established between nodes and added to the topology graph as edges. The constructed topology graph is input into a graph neural network to perform global semantic and geometric consistency modeling among multiple targets. The global topology relationship is used to correct the local detection results, and finally, the corrected key target category and 3D position are output. S4. Obstacle Detection and 3D Modeling: First, the point cloud data is clustered and segmented to obtain multiple independent point clusters. Then, 3D geometric fitting is performed on each point cluster to generate the minimum bounding box. Each obstacle is parametrically modeled in the form of a 3D bounding box. Reliability indicators are added to the detection results of each obstacle to model uncertainty. The semantic segmentation results are combined with the bounding box modeling results to output structured obstacle information. S5. Recognition Result Enhancement and Verification: Perform semantic and topological consistency verification on the target semantic labels and three-dimensional geometric parameters to obtain recognition results that conform to the topological rules of the berth scene. Then, perform confidence and rule filtering on the recognition results, and finally output the semantic category, three-dimensional spatial position and geometric parameters of each target, as well as the final filtering confidence.

[0011] Preferably, the specific process of S1 is as follows: S1-1, Multi-frame point cloud acquisition: Continuously acquire multiple frames of point cloud data using a shipborne 3D LiDAR. S1-2, Data Filtering and Denoising: Outliers introduced by water surface reflection, environmental interference, etc. are removed by voxel grid downsampling and statistical outlier filtering. S1-3 Motion Distortion Compensation: Using the high-frequency pose information of the IMU and the external parameter calibration results, the motion compensation of each point is performed by time interpolation to eliminate the point cloud distortion caused by the movement of the hull and obtain a high-quality, dense and uniform temporally sparse point cloud.

[0012] Preferably, the specific process of S2 is as follows: S2-1 Lightweight Semantic Feature Extraction: The lightweight Transformer module is used to encode the point cloud, extract multi-scale semantic features, and output the category probability distribution corresponding to each point. The main categories include at least berth front, shore wall, buoy, bollard and ship. S2-2, Introduction of Geometric Constraints (1) For the berth front target: On the candidate point cloud of the semantic segmentation result, the straight line / plane fitting method is used to constrain its shape to be a continuous straight line / plane; through geometric consistency correction, the accuracy of berth boundary recognition is improved.

[0013] (2) For buoy targets: fit the semantic segmentation candidate regions to cylinders or spheres to ensure that buoy recognition not only depends on semantic features but also satisfies geometric constraints.

[0014] (3) For cable pile targets: small-scale convex body fitting is adopted, combined with point cloud local convexity detection method to enhance the robustness of identification of small structures such as cable piles; S2-3, Semantic-Geometric Consistency Fusion: Geometric fitting error is introduced into the loss function as a consistency constraint, while geometric correction is performed on the semantic segmentation output to eliminate misidentified points or unreasonable boundaries; S2-4. Output Results: Output the categories, locations, and boundaries of key targets in the unmanned surface vessel (USV) berthing scenario; the key targets are the berth front, buoys, bollards, and the vessel.

[0015] Preferably, the specific process of S3 is as follows: S3-1. Topology Node Construction: Based on semantic segmentation and geometric constraint results, key targets in the berth environment are extracted; and the identified objects such as the berth front, buoys, bollards, and ships are used as nodes in the topology graph. Each node includes semantic category, spatial location, geometric parameters, and confidence level; the geometric parameters include shape and orientation. S3-2, Definition of Topological Edge Relationships: Based on prior knowledge of the port berthing scenario, establish relationship constraints between nodes and add these relationships to the topological graph in the form of edges, with each edge having constraints or weights. S3-3, Topological Relationship Modeling: The node features and edge features of the constructed topological graph are input into the GNN graph neural network. Through the node-edge interaction propagation mechanism of the GNN, global semantic and geometric consistency modeling among multiple targets is achieved. The node features include the semantic category, geometric shape, spatial location, and confidence of the target. The edge features include distance, direction, and alignment, which represent relationship parameters. S3-4, Reasoning and Error Correction: During the GNN graph neural network reasoning process, global topological relationships are used to correct local detection results; when a target is misclassified or missed, compensation or correction is performed through neighboring nodes and edge relationships; when the detection result does not conform to the topological constraints, i.e., when the buoy deviates from the berth direction, it is readjusted or removed. S3-5. Output Results: Output the corrected key target categories and 3D positions.

[0016] Preferably, the relational constraints are specifically established as follows: a. Bollards – Berth Edge: Bollards should be located near the berth edge; b. Buoys – Berth Lead-off: The buoy sequence should be aligned with the direction of the berth lead-off; c. Vessel – Berth Front: The vessel should be positioned close to the berth area and its orientation should be parallel or nearly parallel to the berth line; d. Buoys – Buoys: Buoys of the same type are usually distributed on straight lines or curves with equal spacing.

[0017] Preferably, the specific process of S4 is as follows: S4-1 Point Cloud Clustering and Segmentation S4-1-1, Input: Point cloud data processed by semantic-geometric-topological fusion; S4-1-2, Operation: Use the Euclidean distance clustering algorithm to divide the clusters of points belonging to the same obstacle; S4-1-3, Results: Multiple independent point clusters were obtained, and each point cluster corresponds to a potential obstacle; S4-2, 3D Bounding Box Generation: Perform 3D geometric fitting on each point cluster to generate the minimum bounding box. Each obstacle is parametrically modeled in the form of a 3D bounding box. The parameters of the 3D geometric fitting include center coordinates, size, and orientation. The center coordinates are the geometric center or centroid of the point cluster. The size is the projected length of the point cluster in the three principal directions. The orientation is determined by the minimum volume bounding box algorithm. S4-3 Uncertainty Modeling: A reliability index is attached to the detection result of each obstacle. The reliability index includes confidence, location variance, and uncertainty expression. The confidence is derived from the classification probability of the semantic segmentation network and measures the credibility of the obstacle category determination. The location variance is calculated through Kalman filter residuals and describes the uncertainty of the detected target's position and size. The uncertainty expression represents the error range of the target in three-dimensional space in the form of an ellipsoid or confidence interval. S4-4, Semantic Category and Attribute Fusion: The results of the semantic segmentation in the previous stage are combined with the bounding box modeling results. Each 3D bounding box is accompanied by a semantic category. The semantic categories include berth front, buoy, bollard, ship, and other obstacles. The unified output is the target model, which includes semantic category, three-dimensional position and attitude parameters, and confidence and uncertainty estimates. S4-5. Output: Output structured obstacle information; the obstacle information includes target category, spatial geometric parameters, confidence level and uncertainty range.

[0018] Preferably, the semantic and topological consistency verification process is as follows: (1) Input: Target semantic labels and three-dimensional geometric parameters output from the previous stage; (2) Topology constraint check: Bollard nodes must be within a certain distance threshold of the berth front edge, otherwise it is judged as a false detection; the buoy sequence should be approximately parallel to or in the same direction as the berth front edge in space, otherwise its category or position should be corrected; the vessel should be located in the water area rather than on the berth front edge; (3) Correction and elimination: If a target does not meet the topological logic, its position or category is adjusted if there is a reasonable cluster of neighboring points; if it cannot be corrected, it is eliminated directly.

[0019] Preferably, the confidence level and rule filtering process is as follows: (1) Input: The target set and its detection confidence obtained through semantic-geometric-topological modeling; (2) Confidence screening: Remove detection results below the set threshold and retain only targets with high reliability; (3) Topology rationality filtering: Even if the detection confidence is high, if it is seriously inconsistent with the topology logic, it will be eliminated or downweighted. (4) Multi-rule fusion: The final target selection is completed through a comprehensive rule of "semantic confidence + geometric rationality + topological consistency"; (5) Results: A set of key targets for berths with high reliability and high logical consistency was obtained.

[0020] Beneficial effects Compared with the prior art, the beneficial effects achieved by the present invention are as follows: (1) Improved recognition accuracy. Compared with traditional schemes that rely solely on depth segmentation, this invention effectively corrects the boundary and position errors of key targets in berths through semantic-geometric fusion, thereby improving recognition accuracy by approximately 15%–25%.

[0021] (2) Global consistency guarantee. By using semantic topology modeling, isolated identification points and logical conflicts can be eliminated, ensuring that the identification results of berth scenes are consistent with the actual spatial structure, and significantly improving the usability of the identification results.

[0022] (3) Enhanced safety. Through uncertainty modeling and safety margin verification, this invention can provide risk classification information for unmanned surface vessel path planning, making the berthing process safer and more robust.

[0023] (4) Improved robustness. In complex port environments (such as obstruction, sparse point clouds, water surface reflection, etc.), the present invention can still stably identify key targets at berths, and its adaptability is stronger than that of existing methods. Attached Figure Description

[0024] Figure 1 This is a flowchart illustrating the method for identifying key targets in unmanned surface vessel (USV) berths based on semantic-geometric-topology fusion according to the present invention. Detailed Implementation

[0025] To make the objectives, technical solutions, and advantages of the present invention clearer and more complete, the present invention will be further described in detail below with reference to the embodiments. Obviously, the embodiments described below are some embodiments of the present invention, but the scope of protection claimed by the present invention is not limited to the specific embodiments below.

[0026] Example 1 like Figure 1 As shown, a method for identifying key targets in unmanned surface vessel (USV) berths based on semantic-geometric-topological fusion is presented. This method includes the following steps: S1, Point Cloud Preprocessing S1-1, Multi-frame point cloud acquisition: Continuously acquire multiple frames of point cloud data using a shipborne 3D LiDAR. S1-2, Data Filtering and Denoising: Outliers introduced by water surface reflection, environmental interference, etc. are removed by voxel grid downsampling and statistical outlier filtering. Voxel grid downsampling can reduce point cloud density and computational cost while ensuring the shape characteristics of the point cloud.

[0027] S1-3 Motion Distortion Compensation: Using the high-frequency pose information and extrinsic parameter calibration results of the IMU (Inertial Measurement Unit, a sensor used to measure the attitude angles (roll, pitch, yaw) and linear acceleration and angular velocity of a moving vehicle), motion compensation is performed on each point using time interpolation to eliminate the point cloud distortion caused by the movement of the hull, resulting in a high-quality, dense and uniform temporally sparse point cloud.

[0028] Because a mechanically rotating LiDAR requires a certain amount of time (typically 100ms) to scan a point cloud frame, and the hull is in motion during this period, points in the same frame are not acquired at the same time, resulting in motion distortion. Therefore, it is necessary to use the high-frequency pose information of the IMU to perform temporal interpolation compensation for each point, unifying them to the same reference time coordinate system. The specific compensation process is as follows: (1) Time synchronization During point cloud acquisition, a precise timestamp is assigned to each point:

[0029] in, Indicates the start time of this frame. Indicates the first The relative scanning time of each point (calculated based on the lidar line number and rotation angle).

[0030] (2) Find the corresponding IMU pose The IMU outputs high-frequency attitude and velocity information. The settings are as follows: IMU in time position Find two IMU timestamps that contain that time point. (3) Pose linear interpolation Assuming that the linear velocity and angular velocity of the hull change smoothly over a short period of time, linear interpolation can be performed on the IMU pose: Translation interpolation: , Rotational interpolation uses quaternion spherical linear interpolation (Slerp): , Finally, time The hull pose transformation matrix at time t: , (4) Spatiotemporal compensation of point clouds Unify each point to the reference time of this frame. :

[0031] in, Represents the original acquisition point coordinates (LiDAR coordinate system). This represents the coordinates of the point after motion compensation.

[0032] S2, Lightweight Semantic Segmentation and Geometric Constraints S2-1 Lightweight Semantic Feature Extraction: The lightweight Transformer module is used to encode the point cloud, extract multi-scale semantic features, and output the category probability distribution corresponding to each point. The categories include at least berth front, shore wall, buoy, bollard, and ship. The number of model parameters is reduced while ensuring accuracy, thereby improving the real-time performance and applicability of the algorithm.

[0033] S2-2, Introduction of Geometric Constraints (1) For the berth front target: On the candidate point cloud of the semantic segmentation result, the straight line / plane fitting method is used to constrain its shape to be a continuous straight line / plane; through geometric consistency correction, the accuracy of berth boundary recognition is improved.

[0034] (2) For buoy targets: fit the semantic segmentation candidate regions to cylinders or spheres to ensure that buoy recognition not only depends on semantic features but also satisfies geometric constraints.

[0035] (3) For cable pile targets: small-scale convex body fitting is adopted, combined with point cloud local convexity detection method to enhance the robustness of identification of small structures such as cable piles; S2-3, Semantic-Geometric Consistency Fusion (1) During the training phase: the geometric fitting error is introduced into the loss function as a consistency constraint, so that the model can optimize semantic accuracy and geometric rationality at the same time during the learning process; (2) In the reasoning stage: geometric correction is performed on the semantic segmentation output to eliminate misidentified points or unreasonable boundaries; The final output is the target category label and boundary enhanced with semantic and geometric consistency.

[0036] Specifically, the geometric fitting error is introduced into the semantic segmentation loss to achieve semantic-geometric consistency, as follows: (1) Explanation of background symbols Input point cloud: .

[0037] Truth value tags: (one-hot), category .

[0038] The segmentation network outputs the class probability for each point: (softmax).

[0039] Temperature / Power Factor: (Controlling soft weight sensitivity).

[0040] Minimum number of valid points threshold: (e.g., 30).

[0041] Robust kernel function: (For example, Huber).

[0042] The point-level probabilities output by the network are used as soft weights to perform weighted geometric fitting on the candidate point set (which can be solved in a differentiable way). The weighted robust residual from the point to the geometric prototype is calculated and used as a geometric consistency loss term. This residual is then jointly optimized with the semantic segmentation loss, so that the network can simultaneously optimize semantic accuracy and geometric rationality during training.

[0043] The specific steps are as follows: (1) Obtain semantic probabilities from network forward pass training sample point cloud Inputting into the lightweight Transformer module yields... .

[0044] (2): Calculate soft weights Define soft weights for each point and each class:

[0045] (default (It can be set to 0.6–1.0) Calculate the weighted effective number of points for this class:

[0046] like Skip categories The geometric terms (to avoid instability in low-sample conditions).

[0047] (3) Weighted statistics (for differentiable fitting) Calculate the weighted mean:

[0048] Calculate the weighted covariance matrix:

[0049] (4) Differentiable geometric fitting (closed-loop / SVD implementation) Select the geometric prototype type based on the category mapping (e.g., berth front → plane / straight line, buoy → sphere / cylinder, bollard → small cylinder / convex body).

[0050] Plane fitting (example): for Perform SVD or eigenvalue decomposition:

[0051] Find the smallest eigenvector As a plane normal (SVD / Eig is differentiable in modern depth frameworks; SVD stands for Singular Value Decomposition, and Eig refers to Eigen-decomposition, i.e., matrix eigenvalue decomposition). Plane constants:

[0052] For spheres, cylinders, and straight lines, the corresponding weighted least squares closed-form solutions can be used, or weighted PCA can be used to estimate the radius from the axial direction. Both are implemented using linear algebra operations to ensure differentiability.

[0053] PCA is an abbreviation for Principal Component Analysis. It is a commonly used method for data dimensionality reduction and feature extraction. The core idea is to extract the principal directions of the data through the eigenvalue decomposition of the covariance matrix: the first principal component: the direction of the greatest data variation; the minimum principal component: the direction of the least data variation (e.g., the plane normal).

[0054] In point cloud processing, PCA is often used to estimate the structure orientation of point clouds or fit geometric models, such as fitting the principal axes of lines or plane normals.

[0055] Step 5: Calculate the point-to-model residuals (signed / absolute value depends on the model). Taking a plane as an example, the signed distance from a point to the plane is:

[0056] Radial error for spheres / cylinders .

[0057] Step 6: Robustly calculate and weight the residuals Robust kernel for residuals (For example, Huber), the weighted geometric error is obtained:

[0058] Step 7: Geometric Loss and Joint Training Loss Sum the geometric errors of all categories using a weighted average:

[0059] in Weights for categories (key categories such as berth frontage) (relatively large)

[0060] Basic semantic loss:

[0061] Final joint loss (used for backpropagation): , After a semantic segmentation network outputs the class probability for each point, there may be issues such as misidentified points, noise points, discontinuous boundaries, or unreasonable geometric shapes. The goal of geometric correction is to use geometric prior constraints to correct the network output, ensuring that the boundaries of key targets (such as berth fronts, buoys, and bollards) are continuous, their shapes are reasonable, and their positions are accurate. The geometric correction process is as follows: (1) Candidate point cloud extraction For each category (berth front, buoys, bollards, etc.): , in, It represents the class probability output by the segmentation network. It is the confidence threshold (e.g., 0.5).

[0062] (2) Geometric prototype fitting Select a geometric model based on the target category, and perform differentiable fitting or least squares fitting on the candidate point cloud.

[0063] Berth leading edge (planar / linear fitting) Perform PCA or SVD on the candidate point cloud: , in, It is the normal vector. The center of mass.

[0064] Plane equation: , Buoy (sphere / cylinder fitting) Seeking the center of the ball With radius : , Cable bollards (small cylindrical / convex fitting) The central axis is determined using local convexity detection combined with least squares fitting. With radius .

[0065] (3) Geometric correction For each candidate point Calculate the distance from the point to the geometric prototype. : flat:

[0066] Sphere / Cylinder:

[0067] Based on distance threshold Classification correction: , in, It can be set according to LiDAR point cloud accuracy or target size, such as berth leading edge. ,buoy .

[0068] (4) Boundary smoothing and continuity correction Project or cluster the corrected category point cloud to ensure continuity: Planar / linear targets: Smooth the boundaries using RANSAC or local linear fitting.

[0069] Cylindrical / spherical targets: Apply radius-mean filtering to the point cloud on the surface of a sphere / cylinder.

[0070] Noise isolated points (too far from the prototype and isolated) are directly removed.

[0071] (5) Output geometric correction results The final semantic labels are obtained after geometric correction: , By combining geometric prototype parameters, the position, shape, and boundary of key targets are output, which can be directly used for docking planning and collision avoidance algorithms.

[0072] S2-4. Output Results: Output the categories, locations, and boundaries of key targets in the unmanned surface vessel (USV) berthing scenario; the key targets are the berth front, buoys, bollards, and the vessel.

[0073] Compared to traditional solutions that rely solely on depth segmentation, this invention achieves semantic-geometric consistency by overlaying geometric fitting constraints (lines, planes, cylinders, convex bodies) onto the semantic results output by the deep network. This effectively corrects the boundary and positional errors of key berth targets, significantly improves the recognition accuracy of structured targets such as berth fronts, buoys, and bollards, and reduces the impact of noise.

[0074] S3, Berth Semantic Topology Modeling S3-1. Topology Node Construction: Based on semantic segmentation and geometric constraint results, key targets in the berth environment are extracted; and the identified objects such as the berth front, buoys, bollards, and ships are used as nodes in the topology graph. Each node includes semantic category, spatial location, geometric parameters, and confidence level; the geometric parameters include shape and orientation. S3-2. Definition of Topological Edge Relationships: Based on prior knowledge of the port berthing scenario, relationship constraints are established between nodes, and these relationships are added to the topological graph in the form of edges. Each edge carries a constraint condition or weight; the relationship constraints are specifically established as follows: a. Bollards – Berth Edge: Bollards should be located near the berth edge; b. Buoys – Berth Lead-off: The buoy sequence should be aligned with the direction of the berth lead-off; c. Vessel – Berth Front: The vessel should be positioned close to the berth area and its orientation should be parallel or nearly parallel to the berth line; d. Buoys – Buoys: Buoys of the same type are usually distributed on straight lines or curves with equal spacing.

[0075] S3-3, Topological Relationship Modeling (1) Input the constructed topology graph into the GNN graph neural network: a. Node features: including the target's semantic category, geometric shape, spatial location, and confidence level; b. Edge features: including relational parameters such as distance, direction, and alignment; (2) Global semantic and geometric consistency modeling among multiple objectives is achieved through the node-edge interaction propagation mechanism of GNN; In berth semantic topology modeling, a well-constructed topology graph Contains a set of nodes Sum of edges Each node Indicates key targets (berth front, buoys, bollards, ships, etc.), each side This represents the semantic or geometric relationship constraints between nodes (such as distance, orientation, alignment, weight, etc.). GNNs achieve joint optimization of local detection results and global topological constraints through a node-edge interaction propagation mechanism, as follows: (1) Initialization of node and edge features Node features initialization: , in, For spatial location, geometric parameters include shape, orientation, etc.

[0076] Edge features initialization: ; (2) Message Passing For the Layer iteration, nodes Receive neighbor nodes Information is modulated through edge features to enable node-edge interaction: Message calculation: , in, For message functions, MLPs or attention mechanisms can be used: , The edge weight attention can be calculated from the neighbor relationships: .

[0077] (3) Node status update Update node characteristics after aggregating neighbor messages:

[0078] in, For update functions (such as GRU / MLP). This indicates message aggregation operations (such as summation, mean, or attention-weighted summation).

[0079] (4) Edge feature update (optional) If edge features also need to be adjusted with node updates, edge state updates can be defined: , (5) Iterative propagation and output Repeat steps (2)-(4) for a total of Layer iteration enables node features to be integrated with neighbor information and topological constraints, achieving global consistency.

[0080] Final node output features It can be used for target category correction (compensating for missed detections / false detections), location / geometry optimization (ensuring spatial topological rationality), and confidence update (globally consistent weighted).

[0081] (6) Topology correction application For node categories Perform remarking:

[0082] Fine-tune the node positions and geometric parameters to ensure that adjacent nodes meet port berthing constraints: bollard-berth front distance constraint, buoy sequence alignment constraint, and vessels approaching the berth in the same direction.

[0083] In this way, GNN achieves global semantic and geometric consistency correction of local target detection results through node-edge interaction mechanism, ensuring that the berth semantic topology model is consistent with the actual port environment.

[0084] S3-4, Reasoning and Error Correction: During the GNN inference process, global topological relationships are used to correct local detection results; when a target is misclassified or missed, compensation or correction is performed through neighboring nodes and edge relationships; when the detection result does not conform to the topological constraints, i.e., when the buoy deviates from the berth direction, it is readjusted or removed. S3-5. Output Results: Output the corrected key target categories and 3D positions.

[0085] This invention utilizes semantic topology modeling to construct key targets as semantic topology graphs and introduces graph neural networks (GNNs) for relational reasoning. By correcting abnormal results through topological logic, it ensures the global consistency of recognition results, eliminates isolated recognition points and logical conflicts, ensures that the recognition results of berth scenes match the actual spatial structure, solves the problem of easy misjudgment in single-point recognition, and significantly improves the usability of recognition results.

[0086] S4, Obstacle Detection and 3D Modeling S4-1 Point Cloud Clustering and Segmentation S4-1-1, Input: Point cloud data processed by semantic-geometric-topological fusion; S4-1-2, Operation: Use the Euclidean distance clustering algorithm to divide the clusters of points belonging to the same obstacle; The Euclidean distance clustering algorithm uses Euclidean Cluster Extraction (ECE) based on Euclidean distance, and the main process is as follows: (1) For each point Traverse the unmarked cluster of points; for the current point Find the set of points in its neighborhood:

[0087] in, This is the Euclidean distance threshold, used to control the density and size of the point cluster; (2) Place the point Add it to the same cluster as its neighbors, and recursively perform the same operation on the neighbors until all neighbors have been classified. (3) Repeat the above process until all points are assigned to a cluster or marked as isolated points (noise points); to obtain several independent point clusters. Each point cluster corresponds to a potential obstacle.

[0088] Formula expression European distance: , Definition of neighborhood point set: , Clustering: S4-1-3, Results: Multiple independent point clusters were obtained, and each point cluster corresponds to a potential obstacle; S4-2, Generation of 3D Bounding Boxes S4-2-1. Perform 3D geometric fitting on each point cluster to generate the minimum bounding box: a. Central coordinates (x, y, z): the geometric center or centroid of the point cluster; b. Dimensions (l, w, h): The projected lengths of the point cluster in the three principal directions; c. Orientation (θ): Determined using the minimum volume bounding box algorithm; The specific details of the minimum volume bounding box algorithm are as follows: (1) For a given point cloud cluster Construct a three-dimensional cuboid (box) that satisfies: a. Includes all points ; b. Minimal three-dimensional volume; c. It can rotate in any direction (i.e., it is not limited to coordinate axis alignment).

[0089] Output parameters: , : Coordinates of the bounding box center; Length, width, and height; : Rotation angles around three axes (describing directions).

[0090] (2) The steps of the minimum volume bounding box algorithm are as follows: Step 1: Point Cloud Centralization First, shift the point cloud to the vicinity of the centroid: , Step 2: Covariance Matrix and Principal Direction Calculate the point cloud covariance matrix: , Eigenvalue decomposition (EVD) of the covariance matrix: , Three eigenvectors were obtained As the main direction of the point cloud, the corresponding eigenvalues Describe the variance of the point cloud in each direction.

[0091] The direction of the eigenvector is the axis of the optimal rotating coordinate system.

[0092] Step 3: Rotate the point cloud to the principal axis coordinate system Construct the rotation matrix: , Rotate the point cloud to this coordinate system: , At this point, the axis-aligned bounding box of the point cloud in the new coordinate system is approximately the smallest.

[0093] Step 4: Calculate axis alignment minimum bounding box In the rotated coordinate system, find the maximum and minimum values ​​of the point cloud on the three axes:

[0094]

[0095] , Box dimensions: , Center of the box (in a rotating coordinate system): , Step 5: Rotate back to the original coordinate system Center of the enclosure: , Box orientation: The principal axis of the bounding box is the rotation matrix. Column vectors; Rotation angles can be represented using Euler angles or quaternions. .

[0096] (3) Minimum volume constraint Volume formula: , In theory, to obtain the truly global minimum volume, it is necessary to search the point cloud at all rotation angles: , In practice, due to the huge amount of computation, the principal direction of PCA is generally used as an approximate solution, and then a small amount of rotation optimization is performed (such as rotation ±5° or rotation based on convex hull).

[0097] S4-2-2, Output: Each obstacle is parametrically modeled in the form of a 3D bounding box; Parametric modeling means using a set of mathematical parameters to fully describe the spatial position, size, and orientation of each obstacle, thereby transforming the originally sparse point cloud clusters into structured geometric objects, which facilitates subsequent path planning, collision detection, simulation, or state estimation.

[0098] The parametric modeling is as follows: (1) The core idea of ​​parametric modeling The original point cloud consists of a large number of discrete three-dimensional points. The shape and size information is implied in the distribution of points.

[0099] Parametric modeling uses a small number of parameters to describe the geometric features of these points, such as position, size, and orientation, without having to store every single point.

[0100] By parametrically modeling obstacles, a compact mathematical model can replace the entire cluster of points, improving efficiency and tractability.

[0101] (2) Specific parameters In your method, the 3D bounding box of an obstacle typically includes the following parameters: center coordinates. ,size Orientation ; center coordinates Indicates the position and size of the obstacle in the world coordinate system. The size of the obstacle in the length, width, and height directions corresponds to the projection length of the point cluster onto the three principal axes, and the orientation... It represents the angle of rotation of the obstacle along a reference direction.

[0102] Therefore, the 3D bounding box of an obstacle can be represented by a parameter vector as follows: .

[0103] S4-3. Uncertainty Modeling: A reliability index is added to the detection results of each obstacle. This reliability index includes confidence level, position variance, and uncertainty expression. The confidence level is derived from the classification probability of the semantic segmentation network, measuring the reliability of the obstacle category determination. The position variance is calculated using Kalman filter residuals, describing the uncertainty of the detected target's position and size. The uncertainty expression represents the target's error range in three-dimensional space using an ellipsoid or confidence interval. Introducing confidence level and position variance estimation on top of 3D detection provides an uncertainty range for target recognition results, enabling the unmanned surface vessel to adjust its berthing path based on risk level and improve safety.

[0104] The confidence level reflects the credibility of each target belonging to a certain semantic category, and is usually derived from the category probability output by the semantic segmentation network.

[0105] Assuming a certain point or target The semantic category is The probability corresponding to the network output is: , Then the confidence level of the target It can be defined as: , If a target is a cluster of points (an obstacle composed of multiple points), then the average or weighted average confidence score of the points within the cluster can be taken: , in This indicates the number of points in a point cluster.

[0106] Confidence can also be optimized by using weighted geometric features (such as the weight of the distance from the centroid). , in, Point coordinate, Indicates the cluster center. This indicates that the radius controls the weight decay.

[0107] The location variance is used to quantify the uncertainty in estimating the location of point clusters or obstacles. It can be calculated using the geometric properties of the point clusters or the filtered residuals. Specifically: (1) Based on the centroid of point clusters Assuming the obstacle cluster is composed of Points Composition, the centroid of the cluster is: , Then the position variance matrix It can be represented as a covariance matrix: , The diagonal elements are respectively Directional variance, with off-diagonal elements describing the correlation between directions.

[0108] (2) Based on filter residuals (such as Kalman filtering) In dynamic scenes, obstacle poses are obtained through Kalman filtering. With measurement : , The location variance can then be updated as follows: , in, Indicates Kalman gain, Represents the observation matrix. This represents the predicted covariance.

[0109] The uncertainty is typically represented by an ellipsoid or confidence interval, indicating the uncertainty of the obstacle's position and size in three-dimensional space. Specifically: (1) Represented by the covariance ellipsoid Based on the location covariance matrix Perform eigenvalue decomposition: , This represents the eigenvalues ​​of the covariance matrix, corresponding to the lengths of the ellipsoidal axes. The corresponding eigenvector matrix represents the ellipsoidal direction. ellipsoidal semi-axis length Corresponding confidence level : , Represents the critical value (confidence level) of the 3D chi-square distribution. For example, 95% corresponds to ) (2) Expressed as the error range A 3D bounding box can be represented using position and size ± uncertainty:

[0110] in, , Related to confidence level.

[0111] The confidence level, position variance, and uncertainty expression can be used for obstacle screening, topological logic verification, and final target fusion to ensure the robustness and reliability of the unmanned surface vessel berthing perception system.

[0112] S4-4, Semantic Category and Attribute Fusion: Combining the results of semantic segmentation from the previous stage with the bounding box modeling results: a. Each 3D bounding box includes a semantic category; the semantic categories include berth frontage, buoys, bollards, ships, and other obstacles; b. The unified output is the target model, including: semantic category, 3D position and pose parameters, and confidence and uncertainty estimates; S4-5. Output: Output structured obstacle information; the obstacle information includes target category, spatial geometric parameters, confidence level and uncertainty range.

[0113] S5. Enhancement and Verification of Recognition Results S5-1, Semantic and Topological Consistency Verification S5-1-1, Input: Input the target semantic label and three-dimensional geometric parameters output from the previous stage; S5-1-2 Topology constraint check: Bollard nodes must be within a certain distance threshold of the berth front edge; otherwise, it is considered a false detection. The buoy sequence should be approximately parallel to or aligned with the berth front edge in space; otherwise, its category or position should be corrected. The vessel should be located within the water area rather than on the berth front edge. S5-1-3 Correction and Removal: For targets that do not meet the topological logic, if there is a reasonable cluster of neighboring points, adjust the position or category; if correction is not possible, remove them directly to ensure that the recognition results conform to the topological rules of the berth scene and improve global consistency.

[0114] S5-2, Confidence and Rule Filtering S5-2-1, Input: The target set and its detection confidence obtained through semantic-geometric-topological modeling; S5-2-2, Confidence screening: Remove detection results below the set threshold and retain only targets with high reliability; S5-2-3, Topology rationality filtering: Even if the detection confidence is high, if it is seriously inconsistent with the topology logic, it will be eliminated or downgraded. S5-2-4, Multi-rule fusion: The final target selection is completed through a combination of rules including "semantic confidence + geometric rationality + topological consistency"; The comprehensive rule can be expressed as a target scoring function, which applies to each detected target. Calculate the overall score : , in, This represents the semantic confidence index. Indicators representing geometric rationality This represents the topology consistency index. The weighting coefficient is adjustable and satisfies... .

[0115] when If the overall score is greater than or equal to the threshold, the target is retained; otherwise, it is removed or downgraded.

[0116] The semantic confidence index Output from the semantic segmentation network: , If the target is a point cluster, the weighted average of the confidence scores of the points within the cluster can be used, and the semantic confidence index is... The value range of is [0,1], for example .

[0117] The geometric rationality index This is used to measure whether the geometric characteristics of the target conform to prior rules (e.g., the berth leading edge should be a straight line, the buoy should be approximately cylindrical, etc.). Error quantification: berth leading edge straight line fitting residual. buoy cylinder fitting residual and the fitting residual of the cable pile protrusion .

[0118] The geometric rationality index It can be normalized to [0,1]: , in, Represents the fitting residual. This indicates geometric tolerance; the smaller the geometric error, the better. The closer it is to 1, the higher the geometric rationality.

[0119] The topology consistency index This is used to measure whether the spatial arrangement between the target and surrounding key targets conforms to the berth prior logic.

[0120] Bollard – Berth Front Distance Consistency: , otherwise: , Buoy sequence direction consistency:

[0121] , Ship-berth orientation consistency: Similarly, the deviation angle or distance is calculated and mapped to [0,1].

[0122] As In summary, considering all topological constraints: when When this is the case, it indicates high topological rationality. Comprehensive Rules Diagram

[0123] Overall rating:

[0124] like If, then retain the target, if If the weight is low, it will be removed or downgraded for final decision-making.

[0125] S5-2-5, Result: A set of key berth targets with high reliability and high logical consistency was obtained.

[0126] S5-3. Final Result Output: Output the semantic category (berth front, buoy, bollard, ship, etc.), three-dimensional spatial location and geometric parameters, and final screening confidence score for each target.

[0127] This invention introduces a result enhancement and verification mechanism, adding a "semantic and topological consistency verification + confidence filtering" mechanism before the output result to ensure that the final identification result is not only accurate but also reliable and can be directly used for unmanned surface vessel berthing control.

[0128] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions within the technical scope disclosed in this application should be included within the scope of protection of this application. Based on the disclosure and teachings of the above specification, those skilled in the art can also make changes and modifications to the above embodiments. Therefore, this invention is not limited to the specific embodiments disclosed and described above, and some modifications and changes to the invention should also fall within the scope of protection of the claims of this invention. Furthermore, although some specific terms are used in this specification, these terms are only for convenience of explanation and do not constitute any limitation on this invention.

Claims

1. A method for identifying key targets in unmanned surface vessel (USV) berths based on semantic-geometric-topological fusion, characterized in that, The method includes the following steps: S1. Point cloud preprocessing: Multiple frames of point cloud data are continuously acquired using a shipborne 3D lidar. After data filtering, noise reduction, and motion distortion compensation, high-quality, dense, and uniform temporal sparse point cloud is output. S2. Lightweight Semantic Segmentation and Geometric Constraints: The point cloud is encoded, lightweight semantic features are extracted, and geometric constraints are introduced for berth front targets, buoy targets, and bollard targets. Through semantic-geometric consistency fusion, the category, location, and boundary of key targets in the unmanned surface vessel berthing scenario are output. S3. Berth Semantic Topology Modeling: First, key targets in the berth environment are extracted and used as nodes in the topology graph. Then, relational constraints are established between nodes and added to the topology graph as edges. The constructed topology graph is input into a graph neural network to perform global semantic and geometric consistency modeling among multiple targets. The global topology relationship is used to correct the local detection results, and finally, the corrected key target category and 3D position are output. S4. Obstacle Detection and 3D Modeling: First, point cloud data is clustered and segmented to obtain multiple independent point clusters. Then, 3D geometric fitting is performed on each point cluster to generate the minimum bounding box. Each obstacle is parametrically modeled in the form of a 3D bounding box. Reliability indicators are added to the detection results of each obstacle to model uncertainty. The semantic segmentation results are combined with the bounding box modeling results to output structured obstacle information. S5. Recognition Result Enhancement and Verification: Perform semantic and topological consistency verification on the target semantic labels and three-dimensional geometric parameters to obtain recognition results that conform to the topological rules of the berth scene. Then, perform confidence and rule filtering on the recognition results, and finally output the semantic category, three-dimensional spatial position and geometric parameters of each target, as well as the final filtering confidence.

2. The method for identifying key targets in unmanned surface vessel (USV) berths based on semantic-geometric-topological fusion according to claim 1, characterized in that: The specific process of S1 is as follows: S1-1, Multi-frame point cloud acquisition: Continuously acquire multiple frames of point cloud data using a shipborne 3D LiDAR. S1-2, Data Filtering and Denoising: Outliers introduced by water surface reflection, environmental interference, etc. are removed by voxel grid downsampling and statistical outlier filtering. S1-3 Motion Distortion Compensation: Using the high-frequency pose information of the IMU and the external parameter calibration results, the motion compensation of each point is performed by time interpolation to eliminate the point cloud distortion caused by the movement of the hull and obtain a high-quality, dense and uniform temporally sparse point cloud.

3. The method for identifying key targets in unmanned surface vessel (USV) berths based on semantic-geometric-topological fusion according to claim 2, characterized in that: The specific process of S2 is as follows: S2-1 Lightweight Semantic Feature Extraction: The lightweight Transformer module is used to encode the point cloud, extract multi-scale semantic features, and output the category probability distribution corresponding to each point. The main categories include at least berth front, shore wall, buoy, bollard and ship. S2-2, Introduction of Geometric Constraints (1) For the target at the berth front: On the candidate point cloud of the semantic segmentation result, the straight line / plane fitting method is used to constrain its shape to be a continuous straight line / plane; (2) For buoy targets: fit the semantic segmentation candidate regions with cylinders or spheres; (3) For the target of the cable pile: a small-scale convex body fitting is used; S2-3, Semantic-Geometric Consistency Fusion: Geometric fitting error is introduced into the loss function as a consistency constraint, while geometric correction is performed on the semantic segmentation output to eliminate misidentified points or unreasonable boundaries; S2-4. Output Results: Output the categories, locations, and boundaries of key targets in the unmanned surface vessel (USV) berthing scenario; the key targets are the berth front, buoys, bollards, and the vessel.

4. The method for identifying key targets in unmanned surface vessel (USV) berths based on semantic-geometric-topological fusion according to claim 3, characterized in that: The specific process of S3 is as follows: S3-1. Topology Node Construction: Based on semantic segmentation and geometric constraint results, key targets in the berth environment are extracted; and the identified objects such as the berth front, buoys, bollards, and ships are used as nodes in the topology graph. Each node includes semantic category, spatial location, geometric parameters, and confidence level; the geometric parameters include shape and orientation. S3-2, Definition of Topological Edge Relationships: Based on prior knowledge of the port berthing scenario, establish relationship constraints between nodes and add these relationships to the topological graph in the form of edges, with each edge having constraints or weights. S3-3, Topological Relationship Modeling: The node features and edge features of the constructed topological graph are input into the GNN graph neural network. Through the node-edge interaction propagation mechanism of the GNN, global semantic and geometric consistency modeling among multiple targets is achieved. The node features include the semantic category, geometric shape, spatial location, and confidence of the target. The edge features include distance, direction, and alignment, which represent relationship parameters. S3-4, Reasoning and Error Correction: In the reasoning process of GNN graph neural network, global topological relationships are used to correct local detection results; When a target is misclassified or missed, compensation or correction is made through the relationship between neighboring nodes and edges; when the detection result does not meet the topological constraints, that is, when the buoy deviates from the berth direction, readjustment or rejection is performed. S3-5. Output Results: Output the corrected key target categories and 3D positions.

5. The method for identifying key targets in unmanned surface vessel (USV) berths based on semantic-geometric-topological fusion according to claim 4, characterized in that: The specific relationship constraints are established as follows: a. Bollards – Berth Edge: Bollards should be located near the berth edge; b. Buoys – Berth Lead-off: The buoy sequence should be aligned with the direction of the berth lead-off; c. Vessel – Berth Front: The vessel should be positioned close to the berth area and its orientation should be parallel or nearly parallel to the berth line; d. Buoys – Buoys: Buoys of the same type are usually distributed on straight lines or curves with equal spacing.

6. The method for identifying key targets in unmanned surface vessel (USV) berths based on semantic-geometric-topological fusion according to claim 5, characterized in that: The specific process of S4 is as follows: S4-1 Point Cloud Clustering and Segmentation S4-1-1, Input: Point cloud data processed by semantic-geometric-topological fusion; S4-1-2, Operation: Use the Euclidean distance clustering algorithm to divide the clusters of points belonging to the same obstacle; S4-1-3, Results: Multiple independent point clusters were obtained, and each point cluster corresponds to a potential obstacle; S4-2, 3D Bounding Box Generation: Perform 3D geometric fitting on each point cluster to generate the minimum bounding box. Each obstacle is parametrically modeled in the form of a 3D bounding box. The parameters of the 3D geometric fitting include center coordinates, size, and orientation. The center coordinates are the geometric center or centroid of the point cluster. The dimension is the projected length of the point cluster in the three main directions; the orientation is determined by the minimum volume bounding box algorithm. S4-3 Uncertainty Modeling: A reliability index is attached to the detection result of each obstacle. The reliability index includes confidence, location variance, and uncertainty expression. The confidence is derived from the classification probability of the semantic segmentation network and measures the credibility of the obstacle category determination. The location variance is calculated through Kalman filter residuals and describes the uncertainty of the detected target's position and size. The uncertainty expression represents the error range of the target in three-dimensional space in the form of an ellipsoid or confidence interval. S4-4, Semantic Category and Attribute Fusion: The results of the semantic segmentation in the previous stage are combined with the bounding box modeling results. Each 3D bounding box is accompanied by a semantic category. The semantic categories include berth front, buoy, bollard, ship, and other obstacles. The unified output is the target model, including semantic category, 3D position and pose parameters, and confidence and uncertainty estimates; S4-5. Output: Output structured obstacle information; The obstacle information includes target category, spatial geometric parameters, confidence level, and uncertainty range.

7. The method for identifying key targets in unmanned surface vessel (USV) berths based on semantic-geometric-topological fusion according to claim 6, characterized in that: The process of semantic and topological consistency verification is as follows: (1) Input: Target semantic labels and three-dimensional geometric parameters output from the previous stage; (2) Topology constraint check: Bollard nodes must be within a certain distance threshold of the berth front edge, otherwise it is judged as a false detection; the buoy sequence should be approximately parallel to or in the same direction as the berth front edge in space, otherwise its category or position should be corrected; the vessel should be located in the water area rather than on the berth front edge; (3) Correction and elimination: If a target does not meet the topological logic, its position or category is adjusted if there is a reasonable cluster of neighboring points; if it cannot be corrected, it is eliminated directly.

8. The method for identifying key targets in unmanned surface vessel berths based on semantic-geometric-topological fusion according to claim 7, characterized in that: The confidence level and rule filtering process is as follows: (1) Input: The target set and its detection confidence obtained through semantic-geometric-topological modeling; (2) Confidence screening: Remove detection results below the set threshold and retain only targets with high reliability; (3) Topology rationality filtering: Even if the detection confidence is high, if it is seriously inconsistent with the topology logic, it will be eliminated or downweighted. (4) Multi-rule fusion: The final target selection is completed through a comprehensive rule of "semantic confidence + geometric rationality + topological consistency"; (5) Results: A set of key targets for berths with high reliability and high logical consistency was obtained.

Images