Bulk carrier berthing laser radar point cloud data AI processing method and system

Abstract

The application relates to the technical field of laser radar point cloud data processing, in particular to a bulk carrier berthing laser radar point cloud data AI processing method and system; the method comprises the following steps: extracting the motion vector of a target point based on obtained laser radar point cloud data, and calculating the motion parameters of a ship; performing preliminary screening on the target point based on the motion vector and the motion parameters, and setting the weight of each target point after preliminary screening; based on the weight of each target point, performing abnormal detection and time sequence aggregation on the laser radar point cloud data to obtain rain and snow abnormal points and aggregated point clouds; constructing a rain and snow physical model based on the rain and snow abnormal points, and performing point cloud repair and compensation based on the rain and snow physical model to obtain compensated point clouds; analyzing the berthing state of the ship based on the compensated point clouds, and controlling the berthing of the ship based on the berthing state; the application realizes accurate de-noising and complete reconstruction of laser radar point clouds under rain and snow interference, and improves the environmental adaptability and safety efficiency of bulk carrier berthing operation.

Description

Technical Field

[0001] This application relates to the field of lidar point cloud data processing technology, specifically to an AI processing method and system for lidar point cloud data of bulk carriers berthing. Background Technology

[0002] As the shipping industry evolves towards larger and more intelligent vessels, the tonnage of bulk carriers continues to rise, significantly increasing the difficulty of controlling the ship's attitude during berthing and keeping the risk of collisions at the dock consistently high.

[0003] Traditional berthing operations rely on the experience and judgment of the captain and pilot, making it difficult to accurately capture the dynamic positional relationship between the ship and the dock. LiDAR, with its centimeter-level ranging accuracy and active scanning capabilities, can generate 3D point clouds and construct environmental models in real time, making it a core sensing device in bulk carrier berthing environment perception systems. However, bulk carrier terminals are often located in areas with complex climates. In rainy or snowy weather, the laser beam is easily affected by the scattering of water droplets and snowflakes, introducing a large amount of noise into the point cloud data, directly reducing the accuracy of berthing parameter calculations. Therefore, point cloud processing technology with rain and snow interference resistance has become a core technical requirement for ensuring safe berthing in all weather conditions.

[0004] Current lidar point cloud denoising methods are mostly designed for static or general dynamic scenes, such as outlier removal methods based on statistical filtering and noise separation methods based on cluster analysis. These methods have obvious limitations: First, they are difficult to effectively distinguish between rain and snow noise and dynamic valid points at the edge of ships, and dynamic valid points of ships are easily misjudged as noise and removed. Second, they are not adaptable to rain and snow interference, lacking targeted design for interference characteristics under different rain and snow intensities, and cannot dynamically adjust the processing strategy according to the actual interference scenario, resulting in poor stability of denoising effect. Third, they can only remove some explicit rain and snow noise, and lack an effective repair mechanism for problems such as missing ship target point clouds caused by rain and snow interference. The data processing efficiency is greatly reduced in moderate to heavy rain and snow weather, making it difficult to provide reliable environmental perception data support for berthing operations.

[0005] For example, Chinese patent CN121500325A discloses a target vehicle tracking method and system under curve conditions, which uses RANSAC fitting and Kalman filtering for vehicle tracking. However, it is geared towards static road scenarios on land and is difficult to adapt to the complex working conditions of ships berthing, where dynamic deformation of the hull and rain and snow interference coexist. For example, Chinese patent CN122131318A discloses a real-time calculation method and system for six degrees of freedom of ships based on lidar point clouds. It is based on an iterative nearest point point cloud matching algorithm, which solves the optimal transformation moments through nearest point search and singular value decomposition. The method enables real-time calculation of the six degrees of freedom motion of a ship. However, it is mainly aimed at ship motion measurement in normal environments and does not consider the noise interference in laser point clouds under rain and snow. Its point cloud matching accuracy will be significantly reduced in rain and snow environments. In addition, although Chinese patents with publication numbers CN121454483A, CN121564026A and CN121374660A all involve point cloud denoising and reconstruction, they are aimed at different application scenarios such as mine monitoring, ship tracking and robotic arm obstacle avoidance. Their interference types and physical mechanisms are fundamentally different from those of ship berthing in rain and snow.

[0006] The information disclosed in this background section is intended only to enhance the understanding of the overall background of this application and should not be construed as an admission or in any way implying that the information constitutes prior art known to those skilled in the art. Summary of the Invention

[0007] The purpose of this application is to provide an AI processing method and system for lidar point cloud data of bulk carrier berthing, which to a certain extent solves the problems mentioned in the background technology, realizes the denoising and reconstruction of lidar point clouds under rain and snow interference, and improves the environmental adaptability and safety efficiency of bulk carrier berthing operations.

[0008] To achieve the above objectives, this application provides the following technical solution:

[0009] Firstly, this application provides an AI processing method for lidar point cloud data of bulk carriers berthing, comprising the following steps:

[0010] Simultaneously acquire lidar point cloud data and ship attitude sensor data;

[0011] Set target points; calculate the motion vector of each target point based on the lidar point cloud data, and calculate the ship's motion parameters in combination with the ship attitude sensor data;

[0012] The target points are initially screened based on the motion vectors and motion parameters, and weights are assigned to each target point after initial screening. Based on the weights of each target point, anomaly detection and temporal aggregation are performed on the lidar point cloud data to obtain rain and snow anomaly points and aggregated point cloud data.

[0013] Calculate the characteristic parameters of the rain and snow anomalies, and construct a rain and snow physical model based on the characteristic parameters; perform point cloud repair and compensation based on the rain and snow physical model and the aggregated point cloud to obtain the compensated point cloud;

[0014] The berthing status of the ship is analyzed based on the compensated point cloud, and the ship is controlled to berth based on the berthing status.

[0015] As a preferred embodiment of the AI ​​processing method for lidar point cloud data of bulk carrier berthing described in this application, the steps of anomaly detection and temporal aggregation specifically include:

[0016] The dynamic velocity range is divided into M sub-ranges, and a state space is constructed based on the sub-ranges and the rain and snow motion parameters; where M is a positive integer; the length of the aggregation window is calculated based on the state space.

[0017] Based on the length of the aggregation window, the target points corresponding to the current aggregation window are initially screened to obtain the initial screened point cloud;

[0018] A KD-tree index is constructed based on the last frame of the point cloud after initial screening, and a radius neighborhood search is performed on each frame of the point cloud after initial screening to obtain the temporal coordinate sequence of each target point.

[0019] The search radius of the radius neighborhood search is positively correlated with the length of the aggregation window.

[0020] As a preferred embodiment of the AI ​​processing method for lidar point cloud data of bulk carrier berthing described in this application, the steps of anomaly detection and temporal aggregation further include:

[0021] The time-series coordinate sequence of each target point is standardized, and the standardized time-series coordinate sequence is used as input to obtain the predicted coordinates of the corresponding target point using a pre-trained LSTM network.

[0022] Calculate the physical distance between the predicted coordinates and the current coordinates of the target point to obtain the coordinate deviation, and set a deviation threshold based on the rain and snow motion parameters;

[0023] If the coordinate deviation is greater than the deviation threshold, the corresponding target point is regarded as an outlier.

[0024] Extract the weight of outliers. If the weight is less than 1, mark the outlier as a rain / snow outlier; otherwise, mark the outlier as a data outlier.

[0025] The data anomalies are compensated to obtain the aggregated point cloud.

[0026] As a preferred embodiment of the AI ​​processing method for lidar point cloud data of bulk carrier berthing described in this application, the specific method for compensating for data anomalies is as follows:

[0027] Centered on each data anomaly point, the neighborhood search range is set according to the ship's structural dimensions and the spatial resolution of the lidar;

[0028] A spatial neighborhood search is performed based on the neighborhood search range to obtain valid neighborhood points; the valid neighborhood points are the valid ship points in the current aggregation window within the neighborhood.

[0029] The coordinates and corresponding times of each neighborhood valid point are extracted, and the spatial motion trajectory of the neighborhood valid points is constructed by polynomial fitting; the theoretical coordinates of the data anomaly points in the current frame are calculated based on the spatial motion trajectory.

[0030] Using the theoretical coordinates as initial values ​​and the average velocity of the point cloud in the current frame as a constraint, a Kalman filter equation is constructed to obtain the compensation coordinates of the data outliers.

[0031] As a preferred embodiment of the AI ​​processing method for lidar point cloud data of bulk carrier berthing described in this application, the motion parameters include motion feature intervals, and their calculation method is as follows:

[0032] The lidar point cloud data is matched with corresponding points, and the trajectory continuity index of the target point is calculated based on the matching results.

[0033] Extract target points with a trajectory continuity index of 1 from the point cloud of the current frame as ship candidate points, and calculate the average velocity and velocity variance of the point cloud of the current frame using the velocities of all ship candidate points as samples.

[0034] An absolute velocity interval is generated with the average velocity as the center and the velocity variance as the range;

[0035] The ship's berthing phase is identified based on the ship's attitude sensor data, and the absolute speed range is adjusted based on the berthing phase to obtain the dynamic speed range.

[0036] The ship's body dynamics parameters and loading status parameters are collected, and the acceleration safety limit threshold is calculated; the acceleration safety limit threshold is adjusted based on the berthing phase to obtain the acceleration dynamic range.

[0037] As a preferred embodiment of the AI ​​processing method for lidar point cloud data of bulk carrier berthing described in this application, the step of initial screening of the target points specifically includes:

[0038] The state equations are based on velocity, acceleration, and trajectory continuity indices, and the observation equations are based on the motion vector of the target point. A state transition matrix is ​​set according to the ship's berthing stage. Based on the state equations, observation equations, and state transition matrix, the state estimation covariance matrix is ​​obtained through Kalman filtering.

[0039] The confidence level of the corresponding target point is calculated based on the trace of the state estimation covariance matrix; the confidence level is negatively correlated with the trace of the state estimation covariance matrix.

[0040] If the confidence level is greater than or equal to the preset confidence level threshold, the corresponding target point is marked as a valid ship point, and the weight of the valid ship point is set to 1.

[0041] Otherwise, retain the coordinates of the corresponding target point and set the weight of the corresponding target point to n; where n is less than 1.

[0042] As a preferred embodiment of the AI ​​processing method for lidar point cloud data of bulk carrier berthing described in this application, the rain and snow physical model is constructed as follows:

[0043] LiDAR point cloud data, rain and snow characteristic data, and reference point cloud data under different rain and snow intensity levels are collected simultaneously under different berthing scenarios to obtain a related dataset; an initial model is constructed based on lidar scattering theory, and the model parameters of the initial model are fitted using the related dataset as a sample;

[0044] Density clustering algorithm is used to cluster the rain and snow anomalies to obtain different rain and snow point clusters; the neighborhood radius of the cluster is positively correlated with the length of the aggregation window;

[0045] Fit the motion trajectory of the rain and snow spot cluster, and calculate the falling velocity of rain and snow based on the vertical component of the motion trajectory; calculate the mean physical distance between rain and snow anomaly points within the rain and snow spot cluster to obtain the rain and snow particle size;

[0046] The rain and snow physical model is obtained by updating the model parameters of the initial model based on the falling speed and particle size of the rain and snow.

[0047] As a preferred embodiment of the AI ​​processing method for lidar point cloud data of bulk carrier berthing described in this application, the specific steps of point cloud repair and compensation are as follows:

[0048] The ship's structural constraints are determined based on the ship's body dynamics parameters and loading state parameters, and the rain and snow physical model, ship structural constraints, and motion characteristic range are used as constraint conditions.

[0049] A physical-guided generative adversarial network is constructed based on the aforementioned constraints, and the associated dataset is used as training samples. The physical-guided generative adversarial network is pre-trained using a model-independent meta-learning algorithm.

[0050] Using the aggregated point cloud as the base input, a pre-trained physical-guided generative adversarial network is used to generate a pre-compensated point cloud.

[0051] Calculate the similarity index between the preliminary compensated point cloud and the effective points of the ship. If the similarity index is greater than a preset index threshold, then the preliminary compensated point cloud is taken as the final compensated point cloud.

[0052] Otherwise, adjust the model parameters of the rain and snow physics model and regenerate the preliminary compensated point cloud based on the pre-trained physics-guided generative adversarial network.

[0053] As a preferred embodiment of the AI ​​processing method for lidar point cloud data of bulk carrier berthing described in this application, the similarity index is calculated as follows:

[0054] Feature extraction was performed on the point cloud after preliminary compensation and the ship point cloud composed of all valid ship points, and the similarity coefficient of each feature was calculated.

[0055] A similarity index is obtained by weighting the similarity coefficients of each feature based on the berthing stage and then summing the similarity coefficients of each feature based on the weighting coefficients.

[0056] The features include motion features, which are calculated as follows: calculate the motion vector of the point cloud after preliminary compensation, and divide the dynamic velocity range into N velocity sub-ranges; where N is a positive integer;

[0057] The percentage of points whose velocities lie in each velocity sub-interval in the motion vector of the point cloud after preliminary compensation and the percentage of points whose velocities lie in each velocity sub-interval in the motion vector of the ship point cloud are respectively counted to obtain the velocity distribution vectors of the point cloud and the ship point cloud after preliminary compensation.

[0058] The cosine similarity between the velocity distribution vector of the point cloud after preliminary compensation and the velocity distribution vector of the ship's point cloud is calculated to obtain the similarity coefficient of the motion features.

[0059] Secondly, this application provides an AI processing system for lidar point cloud data of bulk carriers berthing, including a data acquisition module, a point cloud filtering module, a point cloud compensation module and a ship control module;

[0060] The data acquisition module is used to simultaneously acquire lidar point cloud data and ship attitude sensor data; the data acquisition module is also used to set target points; calculate the motion vector of each target point based on the lidar point cloud data, and calculate the ship's motion parameters in combination with the ship attitude sensor data;

[0061] The point cloud filtering module performs initial screening of the target points based on the motion vectors and motion parameters, and sets weights for each target point after initial screening; based on the weights of each target point, it performs anomaly detection and temporal aggregation on the lidar point cloud data to obtain rain and snow anomaly points and aggregated point clouds;

[0062] The point cloud compensation module is used to calculate the feature parameters of the rain and snow anomaly points and construct a rain and snow physical model based on the feature parameters; the point cloud compensation module is also used to perform point cloud repair and compensation based on the rain and snow physical model and the aggregated point cloud to obtain the compensated point cloud;

[0063] The ship control module analyzes the ship's berthing status based on the compensated point cloud and controls the ship to berth based on the berthing status.

[0064] Compared with the prior art, the beneficial effects achieved by the present invention are as follows:

[0065] Motion vectors of target points are extracted from the acquired LiDAR point cloud data, and the ship's motion parameters are calculated based on ship attitude sensor data. The target points are initially screened based on the motion vectors and parameters, and weights are assigned to each screened target point. Based on the weights of each target point, anomaly detection and temporal aggregation are performed on the LiDAR point cloud data to obtain rain and snow anomalies and aggregated point clouds. This not only effectively removes a large number of extreme noise points but also improves the consistency and reliability of the point cloud data. Feature parameters of the rain and snow anomalies are calculated, and a rain and snow physical model is constructed based on these parameters, improving the targeting of rain and snow noise removal. Point cloud repair and compensation are performed based on the rain and snow physical model and the aggregated point cloud to obtain compensated point clouds. This compensates for point cloud gaps and coordinate offsets caused by rain and snow interference, improving the accuracy and completeness of the point cloud data. The ship's berthing status is analyzed based on the compensated point cloud, and berthing is controlled based on the berthing status, reducing human judgment errors and improving the safety and efficiency of ship berthing operations. Attached Figure Description

[0066] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. Wherein:

[0067] Figure 1 A flowchart of an AI processing method for lidar point cloud data of bulk carriers during berthing, provided in this application;

[0068] Figure 2This application provides an architecture diagram of an AI processing system for lidar point cloud data of bulk carrier berthing. Detailed Implementation

[0069] The technical solution of this application will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the embodiments and specific features in the embodiments are detailed descriptions of the technical solution of this application, rather than limitations thereof. In the absence of conflict, the embodiments and technical features in the embodiments can be combined with each other.

[0070] Example 1

[0071] like Figure 1 As shown in the figure, this embodiment introduces an AI processing method for lidar point cloud data of bulk carriers berthing, including:

[0072] Simultaneously acquire lidar point cloud data and ship attitude sensor data; the lidar point cloud data includes a set of three-dimensional coordinate points generated by lidar scanning of the ship and its surrounding environment; define each independent coordinate point in the set of three-dimensional coordinate points as a target point; calculate the motion vector of each target point based on the lidar point cloud data, and calculate the ship's motion parameters in combination with the ship attitude sensor data;

[0073] The motion vector of the target point is extracted as follows:

[0074] Three consecutive frames of point cloud data are extracted from the lidar point cloud data as an analysis sequence, denoted as frame k-2, frame k-1, and frame k. Voxel filtering is used to remove duplicate points within a single frame. Frame k represents the point cloud of the current frame, and the voxel grid size is set according to the lidar's spatial resolution. In this embodiment, the lidar's horizontal angular resolution is α, its vertical angular resolution is β, and its maximum effective detection range is R_max. Therefore, the voxel grid size is max(R_max × tan(α), R_max × tan(β)) × γ, such that the voxel grid size is consistent with... The spatial spacing between adjacent laser beams at the maximum detection range of the lidar is matched to ensure that no non-repeating points from different laser beams are included in the same voxel, while avoiding excessive compression of the effective point cloud due to excessively large grid size; where tan(α) is the tangent of the horizontal angular resolution α, tan(β) is the tangent of the vertical angular resolution β, and γ is an adjustment coefficient with a value ranging from 0.5 to 1.5. The specific value is determined according to the point cloud density requirements: when more details need to be retained, a smaller value such as 0.5 to 0.8 is used, and when the amount of data needs to be compressed significantly, a larger value such as 1.2 to 1.5 is used.

[0075] A KD tree is constructed using the k-frame point cloud in the analysis sequence as the reference frame, and a radius neighborhood search is performed on the k-2 frame point cloud and the k-1 frame point cloud in turn to achieve matching of corresponding points; the inter-frame displacement between successfully matched corresponding points is calculated, and the frame interval is determined in combination with the frame rate of the lidar; the velocity and acceleration of the corresponding target point are calculated based on the inter-frame displacement and frame interval.

[0076] The trajectory continuity index of the target point is determined based on the result of the matching of the same point. If the same target point in the k-frame point cloud is successfully matched with the same point in both the k-2 frame point cloud and the k-1 frame point cloud, the trajectory continuity index of the corresponding target point is set to 1; otherwise, the trajectory continuity index of the corresponding target point is set to 0.

[0077] The motion vector of the corresponding target point is constructed based on the velocity, acceleration, and trajectory continuity indices.

[0078] The motion parameters include motion characteristic intervals and rain / snow motion parameters, and their calculation methods are as follows:

[0079] Extract target points with a trajectory continuity index of 1 from the point cloud of the current frame as ship candidate points, and use the velocities of all ship candidate points as samples to learn the overall motion state of the ship in the current frame through recursive least squares method to obtain the average velocity and velocity variance of the point cloud of the current frame.

[0080] Based on ship attitude sensor data, a pre-trained temporal convolutional network is used to identify the ship's berthing phase. The berthing phase includes an initial phase, a deceleration phase, and a docking phase. In the initial phase, the ship approaches the dock at a relatively high speed. In the deceleration phase, the ship gradually reduces its speed and adjusts its attitude. In the docking phase, the ship completes docking at an extremely low speed.

[0081] In this embodiment, the temporal convolutional network adopts a residual connection structure, consisting of an input layer, a first residual module, a second residual module, a third residual module, a global average pooling layer, a fully connected layer, and a Softmax output layer connected in sequence. Each residual module consists of two dilated causal convolutional layers with dilation rates c and 2c, where c is the base dilation rate for each residual module; c is 1 for the first residual module, c is 2 for the second residual module, and c is 4 for the third residual module. Each dilated causal convolutional layer is followed by a batch normalization layer, a ReLU activation function, and a D0D ... The process involves a ropout layer; the input of the residual module is dimension-matched through a 1×1 convolutional layer and then added to the output to form a residual connection; the number of convolutional kernels in each residual module is 64, 128, and 256 respectively, and the kernel size is set to 3; the global average pooling layer averages the output of the third residual module along the time dimension to obtain a 256-dimensional feature vector; the fully connected layer maps the 256-dimensional feature vector to a 3-dimensional output, which is then converted into a probability distribution of the three berthing stages by a Softmax layer, and the category with the highest probability is taken as the recognition result of the current berthing stage.

[0082] An absolute speed range is generated with the average speed as the center and the speed variance as the range, and the absolute speed range is adjusted based on the berthing stage to obtain the dynamic speed range.

[0083] The ship's body dynamics parameters and loading status parameters are collected, and the ship's acceleration safety limit threshold is calculated based on the body dynamics parameters and loading status parameters; the acceleration safety limit threshold is adjusted based on the berthing stage to obtain the acceleration dynamic range; the acceleration dynamic range includes the longitudinal acceleration dynamic range and the lateral acceleration dynamic range;

[0084] Optionally, the acceleration safety limit threshold is calculated as follows: the ship dynamics parameters and loading state parameters are simulated and calculated using a ship dynamics simulation model to obtain the maximum longitudinal acceleration value and the maximum lateral acceleration value that the ship can withstand. The maximum longitudinal acceleration value and the maximum lateral acceleration value are then multiplied by a preset adjustment coefficient to obtain the corresponding acceleration safety limit threshold. The adjustment coefficient ranges from 0.7 to 0.9 and is set based on the sensor's measurement accuracy and the model estimation error. When the sensor's measurement accuracy is higher and the model estimation error is smaller, the adjustment coefficient takes a larger value; when the sensor's measurement accuracy is lower and the model estimation error is larger, the adjustment coefficient takes a smaller value.

[0085] In this embodiment, the specific method for adjusting the acceleration safety limit threshold based on the berthing stage is as follows: If the berthing stage is the initial stage, the acceleration safety limit threshold is multiplied by a preset first adjustment coefficient to obtain the acceleration dynamic range corresponding to the initial stage; if the berthing stage is the deceleration stage, the longitudinal acceleration safety limit threshold is multiplied by a preset second adjustment coefficient to obtain the longitudinal acceleration dynamic range, and the lateral acceleration dynamic range is set to be less than or equal to a preset acceleration threshold; the acceleration threshold is set according to the lateral safety control requirements of the ship's berthing operation, specifically as the product of the maximum allowable lateral displacement on the berthing side of the ship and the square of the maximum allowable lateral oscillation angular velocity during berthing, and is less than the product of the lateral acceleration safety limit threshold and the second adjustment coefficient; if the berthing stage is the docking stage, the acceleration safety limit threshold is multiplied by a preset third adjustment coefficient to obtain the acceleration dynamic range corresponding to the docking stage; the first adjustment coefficient, the second adjustment coefficient, and the third adjustment coefficient are all less than 1, and the first adjustment coefficient is greater than the second adjustment coefficient, and the second adjustment coefficient is greater than the third adjustment coefficient;

[0086] Extract target points with a trajectory continuity index of 0 from the point cloud of the current frame as rain and snow noise, and count the number of rain and snow noise points; count the number of target points in the point cloud of the current frame, and use the ratio of the number of rain and snow noise points to the number of target points in the point cloud of the current frame as the rain and snow noise density; the larger the rain and snow noise density, the higher the proportion of rain and snow noise in the current frame, and the more frequent the rain and snow movement.

[0087] Based on historical experience, a density threshold is set. If the density of rain and snow points is greater than or equal to the density threshold, the rain and snow motion parameter is 1; otherwise, the rain and snow motion parameter is 0. In this embodiment, the density threshold is set as follows: under clear weather conditions without rain or snow interference, at least 10 sets of lidar point cloud data for berthing operations are collected. The rain and snow noise density of each set of lidar point cloud data is calculated using the same method, and the average value and standard deviation of each set of calculation results are taken. The sum of the average value and three times the standard deviation is used as the density threshold.

[0088] The target points are initially screened based on the motion vectors and motion feature intervals, and weights are assigned to each of the initially screened target points. Based on the weights of each target point, anomaly detection and temporal aggregation are performed on the lidar point cloud data to obtain rain and snow anomaly points and the aggregated point cloud. The specific method of the initial screening is as follows:

[0089] Target points whose speed is greater than the upper limit of the speed dynamic range and whose acceleration is greater than the upper limit of the acceleration dynamic range are directly filtered out;

[0090] The state equations are based on velocity, acceleration, and trajectory continuity indices, the observation equations are based on the motion vector of the target point, and the state transition matrix is ​​set according to the ship's berthing phase.

[0091] Based on the state equation, observation equation, and state transition matrix, the state estimation covariance matrix is ​​obtained through Kalman filtering, and the confidence level of the corresponding target point is calculated based on the trace of the state estimation covariance matrix.

[0092] The confidence level is negatively correlated with the trace of the state estimation covariance matrix; the smaller the trace of the state estimation covariance matrix, the closer the confidence level is to 1.

[0093] If the confidence level is greater than or equal to the preset confidence threshold, the corresponding target point is marked as a valid ship point, and the weight of the valid ship point is set to 1; otherwise, the corresponding target point is marked as a candidate noise point, and the candidate noise point is soft-deleted, that is, the coordinates of the candidate noise point are retained, and the weight of the candidate noise point is set to n; where n is less than 1, and is the product of the corresponding confidence level and the basic weight value; the confidence threshold is set as follows: under clear weather conditions without rain or snow interference, no less than 10 sets of lidar point cloud data for berthing operations are collected, the confidence level is calculated for each target point, the confidence level distribution of all valid ship points is statistically analyzed, and the lower quartile is taken as the confidence threshold; the basic weight value is dynamically determined according to the ratio of the number of valid ship points to the number of candidate noise points, specifically the ratio of the number of valid ship points to the sum of the number of valid ship points and candidate noise points.

[0094] The specific steps for anomaly detection and temporal aggregation of the lidar point cloud data are as follows:

[0095] The length of the aggregation window is calculated based on motion vectors: the dynamic velocity interval is divided into M sub-intervals to characterize the differences in motion states during the berthing phase, and a discrete state space is constructed based on the sub-intervals and rain / snow motion parameters; where M is a positive integer; in this embodiment, M is set to 3; an action space containing different aggregation window lengths is defined, and a Q-table is established based on the action space and the state space; based on the Q-table, the length of the aggregation window corresponding to the action with the largest Q value is calculated using the Q-learning algorithm;

[0096] For example, the dynamic speed range is uniformly divided into M sub-ranges, corresponding to three discrete states: low speed, medium speed, and high speed. The motion parameters for rain and snow are 0 or 1, thus constructing a state space S={s1,s2,s3,s4,s5,s6} containing 6 discrete states; and defining an action space A={a1,a2,...,a...} containing K discrete actions. K}, each action a j The corresponding aggregation window length value L j; j∈[1,K], K is 5, then L1=3 frames, L2=5 frames, L3=7 frames, L4=10 frames, L5=15 frames; construct a 6×5 Q-table, and initialize all Q values ​​to 0; when action a is selected j Then, the reward value is calculated and the Q-table is updated using the temporal difference method. When selecting actions, an ε-greedy strategy is adopted, which selects the action with the largest Q value in the current state with a 90% probability and explores other actions randomly with a 10% probability. As the docking process proceeds, the Q-table is updated once after processing each frame of point cloud, so that the aggregation window length gradually converges to the optimal value in the current environment.

[0097] Based on the length of the aggregation window, the lidar point cloud data corresponding to the current aggregation window is obtained and initially screened to obtain the initial screening point cloud. Taking the last frame of the initial screening point cloud as the benchmark, a KD tree index is constructed, and a radius neighborhood search is performed on the remaining frames of the initial screening point cloud to achieve matching of the same physical target with corresponding points across multiple frames, thus obtaining the temporal coordinate sequence of each target point. The search radius is positively correlated with the length of the aggregation window.

[0098] The time-series coordinate sequence of each target point is standardized, and the standardized time-series coordinate sequence is used as input. A pre-trained LSTM network, i.e., a long short-term memory network, is used to obtain the predicted coordinates of the corresponding target point.

[0099] Specifically, the training samples of the LSTM network are historical lidar point cloud datasets of ship berthing processes under rain- or snow-free scenarios; the LSTM network captures the temporal dependencies between the coordinates of the same target point in each frame of the temporal coordinate sequence through hidden layer memory units, and learns the temporal evolution pattern of the corresponding target point under the normal berthing motion state of the ship based on the temporal dependencies; based on the temporal evolution pattern, the predicted coordinates of the target point in the current frame are output through the output layer of the LSTM network.

[0100] In this embodiment, the LSTM network is a three-layer stacked structure, consisting of an input layer, a first LSTM layer, a second LSTM layer, a third LSTM layer, a Dropout layer, and a fully connected output layer connected in sequence. The input dimension of the input layer is T×3, where T is the length of the temporal coordinate sequence, i.e., the number of frames contained in the aggregation window. The first LSTM layer has 64 hidden units, the second LSTM layer has 128 hidden units, and the third LSTM layer has 64 hidden units. Each layer uses the tanh activation function. The dropout rate of the Dropout layer is set to 0.2 to prevent overfitting. The output dimension of the fully connected output layer is 3, corresponding to the predicted three-dimensional coordinates of the target point in the current frame.

[0101] The physical distance between the predicted coordinates and the coordinates of the target point in the current frame is calculated to obtain the coordinate deviation, and a deviation threshold is set based on the rain and snow motion parameters. Specifically, when the rain and snow motion parameter is 1, the deviation threshold is set as a first deviation threshold; when the rain and snow motion parameter is 0, the deviation threshold is set as a second deviation threshold; the first deviation threshold is greater than the second deviation threshold; if the coordinate deviation is greater than the deviation threshold, the corresponding target point is designated as an anomaly; the weight of the anomaly is extracted, and if the weight is less than 1, the anomaly is marked as a rain and snow anomaly; otherwise, the anomaly is marked as a data anomaly.

[0102] In this embodiment, under clear weather conditions without rain or snow interference, at least 10 sets of lidar point cloud data for berthing operations are collected. For each target point, the coordinate deviation between its predicted coordinates and the coordinates of the current frame is calculated, and the average and standard deviation of all coordinate deviations are calculated. The second deviation threshold is set as the sum of the average value and twice the standard deviation, and the first deviation threshold is set as the product of the second deviation threshold and the rain and snow interference amplification factor. The rain and snow interference amplification factor is the ratio of the rain and snow noise density of the current frame to the density threshold.

[0103] The data anomalies are compensated to obtain the aggregated point cloud; the specific compensation method is as follows:

[0104] Centered on each data anomaly point, a neighborhood search range is set according to the ship's structural dimensions and the spatial resolution of the lidar; a spatial neighborhood search is performed based on the neighborhood search range to obtain valid neighborhood points; the valid neighborhood points are the valid ship points in the current aggregation window within the neighborhood.

[0105] The coordinates and corresponding times of each neighborhood valid point are extracted, and the spatial motion trajectory of the neighborhood valid points is constructed by polynomial fitting; the theoretical coordinates of the data anomaly points in the current frame are calculated based on the spatial motion trajectory.

[0106] Using the theoretical coordinates as initial values ​​and the average velocity of the point cloud in the current frame as a constraint, a Kalman filter equation is constructed to obtain the compensation coordinates of the data outliers.

[0107] Calculate the characteristic parameters of the rain and snow anomaly points, and construct a rain and snow physical model based on the characteristic parameters; perform point cloud repair and compensation based on the rain and snow physical model and the aggregated point cloud to obtain the compensated point cloud;

[0108] The physical model of rain and snow is constructed as follows:

[0109] LiDAR point cloud data, rain and snow characteristic data, and reference point cloud data are simultaneously collected under different rain and snow intensity levels in different berthing scenarios to obtain a correlated dataset. The different berthing scenarios include different berthing stages, different ship types, and different dock environments. The LiDAR point cloud data includes the coordinates of the point cloud, the point cloud intensity value, and the LiDAR operating parameters. The rain and snow characteristic data includes the particle size distribution, propagation distance, spatial concentration, and optical refractive index of rain and snow particles. The reference point cloud data is LiDAR point cloud data collected under the same berthing scenario without rain and snow interference.

[0110] An initial model is constructed based on lidar scattering theory, and the model parameters of the initial model are fitted using the associated dataset as samples. The initial model includes a scattering intensity sub-model, an intensity attenuation sub-model, and a coordinate offset sub-model. The scattering intensity sub-model is used to correlate rain and snow particle size, refractive index, and laser wavelength to quantify the scattering intensity of the laser by rain and snow particles. It is constructed based on MIE scattering theory, and its expression is: I_sca=(λ 2 / (8π 2 ·r 2 ))×((m 2 -1) / (m 2 +2)) 2 ×(2π·d / λ) 6 Where I_sca is the scattering intensity of the laser by rain and snow particles, λ is the laser wavelength, r is the distance from the laser to the rain and snow particles, m is the refractive index of the rain and snow particles, and d is the rain and snow particle size; the intensity attenuation sub-model is used to establish the correlation between rain and snow particle concentration, propagation distance, and point cloud intensity attenuation. Based on the Beer-Lambert law, its expression is: I_atten = I0 × exp(-μ × C × D), where I_atten is the laser intensity after passing through the rain and snow medium, I0 is the laser emission intensity, and μ is the attenuation coefficient of the rain and snow particles, calculated based on the rain and snow particle size and the laser wavelength. The calculation formula is μ=(3×Q_ext) / (2×d×ρ_w), where Q_ext is the extinction efficiency factor, obtained from a table using MIE scattering based on the ratio of rain / snow particle size to laser wavelength; ρ_w is the material density of rain / snow particles, typically the density of water; C is the rain / snow particle concentration; D is the laser propagation distance in the rain / snow medium, equal to r; and the ratio of I_atten to I0 is the point cloud intensity attenuation. The coordinate offset sub-model is used to correct the spurious offset of point cloud coordinates by incorporating the motion trajectory of rain / snow particles. It is constructed based on the motion trajectory of rain / snow particles and is expressed as: ΔP=Σ i w i ×(v i ×Δt i ); where ΔP is the spurious coordinate offset vector of the laser beam caused by scattering from rain and snow particles, i is the index of the rain and snow particles on the laser propagation path, and v iLet Δt be the velocity vector of the i-th rain / snow particle. i Let w be the interaction time between the laser and the i-th rain / snow particle. i The weighting coefficient is the i-th rain / snow particle; the interaction time is the ratio of the rain / snow particle diameter to the laser beam diameter; the weighting coefficient is proportional to the scattering intensity of the rain / snow particle, and the proportionality coefficient is the reciprocal of the sum of the scattering intensities of all rain / snow particles on the current laser propagation path.

[0111] The characteristic parameters of rain and snow anomalies in the aggregated point cloud are calculated, including rain and snow falling velocity and rain and snow particle size; the calculation method of the characteristic parameters is as follows:

[0112] Density clustering algorithm is used to cluster the rain and snow anomalies to obtain different rain and snow point clusters; the neighborhood radius of the cluster is positively correlated with the length of the aggregation window;

[0113] For example, the DBSCAN (Density-Based Spatial Clustering of Applications with Noise) algorithm is used, with the neighborhood radius set to 0.02 times the length of the aggregation window and the minimum number of points set to 3. For each rain / snow anomaly point, the number of rain / snow anomaly points contained in its neighborhood radius is counted. If the number of points is greater than the minimum number of points, the rain / snow anomaly point is marked as a core point. Starting from any unvisited core point, all rain / snow anomaly points in its neighborhood are grouped into the same cluster, and this process is recursively expanded until all density-reachable points are grouped into the cluster. Rain / snow anomaly points that are not grouped into any cluster are marked as noise points and discarded. Finally, several rain / snow point clusters are obtained, and each rain / snow point cluster contains multiple spatially adjacent and temporally continuous rain / snow anomaly points.

[0114] The coordinates of the rain and snow anomaly points in the rain and snow point cluster are sorted in chronological order, and the motion trajectory of the rain and snow point cluster is fitted using the least squares method; the falling velocity of rain and snow is calculated based on the vertical component of the motion trajectory, and the average physical distance between the rain and snow anomaly points in the rain and snow point cluster is calculated to obtain the rain and snow particle size.

[0115] The model parameters of the initial model are updated based on the falling speed and particle size of the rain and snow to obtain the rain and snow physical model. Specifically, the particle size of the rain and snow in the scattering intensity sub-model is replaced with the particle size of the rain and snow, and the calculation coefficient of the scattering intensity is corrected by combining the wavelength measured by the lidar and the refractive index of the rain and snow particles. That is, the laser wavelength λ is set to the wavelength measured by the lidar, and the refractive index of the rain and snow particles is set to the value obtained by looking up the table based on the current air temperature and water droplet temperature. The particle size of the rain and snow in the intensity attenuation sub-model is replaced with the rain and snow particle size calculated in real time to recalculate the attenuation coefficient μ. The falling speed of the rain and snow is replaced with the motion speed in the coordinate offset sub-model, and the interaction time and scattering intensity of each rain and snow particle are recalculated based on the particle size of the rain and snow, thereby updating the weight coefficients and improving the correction accuracy of false offsets.

[0116] The specific methods for point cloud repair and compensation are as follows:

[0117] The ship's structural constraints are determined based on the ship's body dynamics parameters and loading state parameters, including deck plane constraints and hatch outline proportion constraints, and the rain and snow physical model, ship structural constraints and motion characteristic range are used as constraint conditions.

[0118] A physical-guided generative adversarial network is constructed based on the aforementioned constraints, and the associated dataset is used as training samples. The physical-guided generative adversarial network is pre-trained using a model-independent meta-learning algorithm.

[0119] In this embodiment, the associated dataset is divided into multiple meta-tasks according to the combination of berthing scenario and rain / snow intensity. Each meta-task contains a small number of training samples and validation samples for that combination. The parameters θ of the physics-guided generative adversarial network are initialized. For each meta-task, a batch of data is sampled from the training samples, a small number of gradient updates are performed under the current parameters θ, and the loss of the meta-task on the validation samples is calculated. The losses of all meta-tasks on the validation samples are summed to obtain the meta-loss. The parameters θ are updated through backpropagation to obtain the optimal parameters. ,make It achieves good performance on all tasks after a small number of gradient updates; when the aggregated point cloud data of the current berthing scenario is obtained, it is used as the training sample for the new meta-task, and the optimal parameters are obtained in the meta-training. Based on this, a small number of gradient updates are performed to quickly adapt to the current specific rain and snow conditions and berthing scenarios.

[0120] Using the aggregated point cloud as the base input, a preliminary compensated point cloud is generated based on a pre-trained physics-guided generative adversarial network. The similarity index between the preliminary compensated point cloud and the effective points of the ship is calculated. If the similarity index is greater than a preset index threshold, the preliminary compensated point cloud is used as the final compensated point cloud. Otherwise, the model parameters of the rain and snow physics model are adjusted, and a preliminary compensated point cloud is generated again based on the pre-trained physics-guided generative adversarial network.

[0121] The index threshold is set as follows: Under clear weather conditions without rain or snow interference, at least 10 sets of lidar point cloud data for berthing operations are collected. For each set of lidar point cloud data, the effective point cloud of the ship is obtained directly without going through a rain / snow physical model and a generative adversarial network, and this is used as the reference point cloud. Simulated rain / snow noise is added to the reference point cloud to generate corresponding simulated interference point clouds. The similarity index between each set of reference point clouds and itself is calculated to obtain the distribution of the first similarity index. The similarity index between each reference point cloud and the corresponding simulated interference point cloud is calculated to obtain the distribution of the second similarity index. The mean and standard deviation of all first similarity indices are calculated, and the mean and standard deviation of the second similarity index are also calculated. The difference between the mean and standard deviation of the first similarity index is used as the lower boundary of the similarity index for high-quality reconstruction, and the sum of the mean and standard deviation of the second similarity index is used as the upper boundary of the similarity index for low-quality reconstruction. The mean of the lower boundary and the upper boundary is used as the index threshold.

[0122] In this embodiment, the generator of the physics-guided generative adversarial network adopts a progressive encoder-decoder structure. The encoder extracts the spatial and temporal features of the aggregated point cloud through three-dimensional convolutional layers. The encoder consists of four sequentially connected three-dimensional convolutional layers, with the number of convolutional kernels in each layer being 32, 64, 128, and 256, respectively. The kernel size is 3×3×3, and the stride is 2. Each convolutional layer is followed by a batch normalization layer and a LeakyReLU activation function. The input of the encoder is the aggregated point cloud, and the output is a 256-dimensional feature map. The decoder consists of four sequentially connected three-dimensional deconvolutional layers, with the number of convolutional kernels in each layer being 128, 64, 32, and 3, respectively. The kernel size is 3×3×3, and the stride is 2. Each deconvolutional layer is followed by a batch normalization layer and a ReLU activation function. The last deconvolutional layer does not use an activation function. During the deconvolution process, each layer of the decoder calls a rain and snow physics model to calculate the matching between each generated point and the scattering characteristics of rain and snow particles. The degree of matching is the ratio of the corrected scattering intensity to the laser intensity after passing through the rain and snow medium. Points with a matching degree higher than a preset threshold are suppressed to remove rain and snow noise. The preset threshold is dynamically determined based on the rain and snow motion parameters of the current frame and the berthing stage: when the scattering echo intensity of a point is equal to 50% of the attenuated intensity, the probability of that point being a rain and snow noise point or a valid ship point is 50 / 50, placing it in a critical state; therefore, the base value of the preset threshold is set to 0.5. When rain and snow interference is strong, the overall scattering intensity of rain and snow particles increases. If the preset threshold is not increased, a large number of valid ship points will be misjudged as rain and snow noise points and suppressed. Increasing the preset threshold can reduce the false deletion rate; therefore, when the rain and snow motion parameter is 1, the preset threshold is increased by 0.1. In the initial stage, the ship is far from the dock, and the tolerance for point cloud accuracy is high, so the suppression conditions can be appropriately relaxed; therefore, the preset threshold is decreased by 0.05. In the berthing stage, the ship is close to the dock, and the point cloud accuracy requirement is the highest; therefore, the preset threshold is increased by 0.05.

[0123] Optionally, the discriminator of the physical-guided generative adversarial network is used to distinguish between the generated point cloud and the ship berthing reference point cloud without rain or snow interference, and to verify whether the velocity and acceleration of the generated point are within the motion characteristic range during the discrimination process.

[0124] The similarity index is calculated as follows: features are extracted from the point cloud after preliminary compensation and the ship point cloud composed of all valid ship points, and the similarity coefficient of each feature is calculated. The features include spatial distribution features, structural geometric features and motion features. Weight coefficients are set based on the berthing stage, and the similarity coefficients of each feature are weighted and summed based on the weight coefficients to obtain the similarity index.

[0125] In this embodiment, the weighting coefficients are set as follows: During berthing, the ship's structural geometry is the most stable and reliable, so the basic weight of the structural geometry is set to the highest weight of 0.4, and the basic weights of the spatial distribution features and motion features are both set to 0.3; In the initial stage of berthing, the ship is far from the dock, the point cloud distribution is relatively scattered, and the spatial distribution features have low distinguishability, so the weighting coefficient of the spatial distribution features is reduced to 0.2, the weighting coefficient of the motion features is increased to 0.4, and the weighting coefficient of the structural geometry features is kept as the basic weight; In the deceleration stage, the ship gradually approaches the dock, and all features are relatively reliable, so the basic weights remain unchanged; In the docking stage, the ship has approached the dock, and the structural geometry is the most critical, so the weighting coefficient of the structural geometry features is increased to 0.5, and the weighting coefficients of the spatial distribution features and motion features are both reduced to 0.25.

[0126] Optionally, the similarity coefficient is calculated as follows: The point cloud after preliminary compensation and the ship point cloud composed of all valid ship points are voxelized, and the number of points within each voxel is counted to generate a corresponding point density histogram; the Barthold distance between the point density histograms of the point cloud after preliminary compensation and the ship point cloud is calculated to obtain the similarity coefficient of spatial distribution features; the geometric parameters of the deck and hatches are extracted from the point cloud after preliminary compensation and the ship point cloud respectively to obtain the corresponding parameter sets; the Euclidean distance between the parameter sets of the point cloud after preliminary compensation and the ship point cloud is calculated, and the difference between 1 and the normalized Euclidean distance is calculated. That is, subtract the normalized Euclidean distance from 1 to obtain the similarity coefficient of the structural geometric features; calculate the motion vector of the point cloud after preliminary compensation, and divide the dynamic velocity interval into N velocity sub-intervals; respectively count the proportion of points whose velocity is located in each velocity sub-interval in the motion vector of the point cloud after preliminary compensation, and the proportion of points whose velocity is located in each velocity sub-interval in the motion vector of the ship point cloud, to obtain the velocity distribution vectors of the point cloud after preliminary compensation and the ship point cloud; where N is a positive integer; calculate the cosine similarity between the velocity distribution vector of the point cloud after preliminary compensation and the velocity distribution vector of the ship point cloud to obtain the similarity coefficient of the motion features.

[0127] The berthing status of the ship is analyzed based on the compensated point cloud data, and the ship is controlled to berth based on the berthing status. The analysis method for the berthing status is as follows:

[0128] Ship structural features are extracted from the compensated point cloud, including the deck plane equation, hatch outline boundary point set, and ship heading angle. Specifically, the deck plane is fitted from the compensated point cloud using a random sampling consensus algorithm, and hatch corner points are extracted based on hatch outline proportion constraints. The principal direction of the compensated point cloud is calculated through principal component analysis, and the heading angle is obtained by combining it with the heading angle from the ship attitude sensor data.

[0129] Based on the aforementioned ship structural characteristics, the relative position parameters between the ship and the dock are calculated, including berthing distance, berthing angle, and berthing speed; the berthing distance is the Euclidean distance between the bow of the ship and the dock reference; the berthing angle is the angle between the bow of the ship and the normal direction of the dock; the berthing speed is the component of the average speed of the effective points of the ship in the berthing direction in the point cloud after compensation.

[0130] Based on the berthing stage, motion characteristic range, and ship structural constraints, berthing safety assessment indicators are calculated. These indicators include distance safety factor, angle safety factor, and speed safety factor. Specifically, the ratio of the berthing distance to a preset safe berthing distance threshold is calculated, and the distance safety factor is calculated based on this ratio. The safe berthing distance threshold is proportional to the total length of the ship, and the proportionality coefficient is the ratio of the total length of the ship to the total length of the berth design. The distance safety factor is calculated by raising the natural constant to the power of the negative of the ratio of the berthing distance to the safe berthing distance threshold. The sine of the berthing angle is calculated to obtain the angle safety factor. The matching degree between the berthing speed and the dynamic speed range is calculated to obtain the speed safety factor. The matching degree is calculated by using a Gaussian function to calculate the closeness between the berthing speed and the center value of the dynamic speed range.

[0131] Based on the aforementioned berthing safety assessment indicators and ship attitude sensor data, a fuzzy control algorithm is used to generate control commands. The input variables of the fuzzy control algorithm include berthing distance, berthing angle, berthing speed, distance safety factor, angle safety factor, speed safety factor, and the current berthing stage. The output variables are main thruster thrust command, side thruster thrust command, and rudder angle command. The main thruster thrust command is used to control the ship's forward or backward propulsion, the side thruster thrust command is used to control the lateral propulsion of the bow, and the rudder angle command is used to control the ship's course.

[0132] In this embodiment, the fuzzy control algorithm maps the input variables to fuzzy sets through a predefined membership function and performs fuzzy inference based on the fuzzy rule base corresponding to the current berthing stage. The fuzzy rule base defines the logical mapping relationship between the input fuzzy sets and the output fuzzy sets under different berthing stages. Through defuzzification processing, the output fuzzy sets obtained by fuzzy inference are converted into main thruster thrust commands, side thruster thrust commands, and rudder angle commands, thereby driving the ship to perform berthing actions. The fuzzy rule base is dynamically configured according to the berthing stage. In the initial stage, it prioritizes optimizing the berthing angle and berthing speed. In the deceleration stage, it focuses on controlling the berthing speed and longitudinal acceleration. In the stopping stage, it ensures that the berthing distance and berthing angle tend to zero, thereby achieving berthing control.

[0133] Example 2

[0134] This embodiment is the second embodiment of this application; it is based on the same inventive concept as Embodiment 1, and refers to... Figure 2 This embodiment introduces an AI processing system for lidar point cloud data of bulk carriers berthing, including a data acquisition module, a point cloud filtering module, a point cloud compensation module, and a ship control module;

[0135] The data acquisition module is used to simultaneously acquire lidar point cloud data and ship attitude sensor data;

[0136] The data acquisition module is also used to set target points; extract the motion vector of the target points based on the lidar point cloud data, and calculate the motion parameters of the ship in combination with the ship attitude sensor data; the motion parameters include motion feature intervals and rain / snow motion parameters;

[0137] The point cloud filtering module performs initial screening of the target points based on the motion vectors and motion parameters, and sets weights for each target point after initial screening; based on the weights of each target point, it performs anomaly detection and temporal aggregation on the lidar point cloud data to obtain rain and snow anomaly points and aggregated point clouds;

[0138] The point cloud compensation module is used to calculate the characteristic parameters of the rain and snow anomaly points and construct a rain and snow physical model based on the characteristic parameters; the characteristic parameters include the rain and snow falling velocity and the rain and snow particle size;

[0139] The point cloud compensation module is also used to repair and compensate point clouds based on the rain and snow physical model and the aggregated point cloud to obtain a compensated point cloud.

[0140] The ship control module analyzes the ship's berthing status based on the compensated point cloud and generates control commands based on the berthing status to control the ship to berth; the control commands include main thruster thrust commands, side thruster thrust commands, and rudder angle commands.

[0141] The specific functions of each step described above are explained in the relevant content of the AI ​​processing method for lidar point cloud data of bulk carrier berthing described in Example 1, and will not be repeated here.

[0142] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0143] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of protection of this application, and these forms are all within the protection scope of this application.

Claims

1. A method for AI processing of lidar point cloud data during the berthing of bulk carriers, characterized in that, Includes the following steps: Simultaneously acquire lidar point cloud data and ship attitude sensor data; Set target points; calculate the motion vector of each target point based on the lidar point cloud data, and calculate the ship's motion parameters in combination with the ship attitude sensor data; The target points are initially screened based on the motion vectors and motion parameters, and weights are assigned to each target point after initial screening. Based on the weights of each target point, anomaly detection and temporal aggregation are performed on the lidar point cloud data to obtain rain and snow anomaly points and aggregated point cloud data. Calculate the characteristic parameters of the rain and snow anomalies, and construct a rain and snow physical model based on the characteristic parameters; perform point cloud repair and compensation based on the rain and snow physical model and the aggregated point cloud to obtain the compensated point cloud; The berthing status of the ship is analyzed based on the compensated point cloud, and the ship is controlled to berth based on the berthing status.

2. The AI ​​processing method for lidar point cloud data of bulk carrier berthing as described in claim 1, characterized in that, The anomaly detection and time-series aggregation steps specifically include: The dynamic velocity range is divided into M sub-ranges, and a state space is constructed based on the sub-ranges and the rain and snow motion parameters; where M is a positive integer; the length of the aggregation window is calculated based on the state space. Based on the length of the aggregation window, the target points corresponding to the current aggregation window are initially screened to obtain the initial screened point cloud; A KD-tree index is constructed based on the last frame of the point cloud after initial screening, and a radius neighborhood search is performed on each frame of the point cloud after initial screening to obtain the temporal coordinate sequence of each target point. The search radius of the radius neighborhood search is positively correlated with the length of the aggregation window.

3. The AI ​​processing method for lidar point cloud data of bulk carrier berthing as described in claim 2, characterized in that, The anomaly detection and time-series aggregation steps also include: The time-series coordinate sequence of each target point is standardized, and the standardized time-series coordinate sequence is used as input to obtain the predicted coordinates of the corresponding target point using a pre-trained LSTM network. Calculate the physical distance between the predicted coordinates and the current coordinates of the target point to obtain the coordinate deviation, and set a deviation threshold based on the rain and snow motion parameters; If the coordinate deviation is greater than the deviation threshold, the corresponding target point is regarded as an outlier. Extract the weight of outliers. If the weight is less than 1, mark the outlier as a rain / snow outlier; otherwise, mark the outlier as a data outlier. The data anomalies are compensated to obtain the aggregated point cloud.

4. The AI ​​processing method for lidar point cloud data of bulk carrier berthing as described in claim 3, characterized in that, The specific method for compensating for the data anomalies is as follows: Centered on each data anomaly point, the neighborhood search range is set according to the ship's structural dimensions and the spatial resolution of the lidar; A spatial neighborhood search is performed based on the neighborhood search range to obtain valid neighborhood points; the valid neighborhood points are the valid ship points in the current aggregation window within the neighborhood. The coordinates and corresponding times of each neighborhood valid point are extracted, and the spatial motion trajectory of the neighborhood valid points is constructed by polynomial fitting; the theoretical coordinates of the data anomaly points in the current frame are calculated based on the spatial motion trajectory. Using the theoretical coordinates as initial values ​​and the average velocity of the point cloud in the current frame as a constraint, a Kalman filter equation is constructed to obtain the compensation coordinates of the data outliers.

5. The AI ​​processing method for lidar point cloud data of bulk carrier berthing as described in claim 4, characterized in that, The motion parameters include motion characteristic intervals, and their calculation method is as follows: The lidar point cloud data is matched with corresponding points, and the trajectory continuity index of the target point is calculated based on the matching results. Extract target points with a trajectory continuity index of 1 from the point cloud of the current frame as ship candidate points, and use the velocities of all ship candidate points as samples to calculate the average velocity and velocity variance of the point cloud of the current frame. An absolute velocity interval is generated with the average velocity as the center and the velocity variance as the range; The ship's berthing phase is identified based on the ship's attitude sensor data, and the absolute speed range is adjusted based on the berthing phase to obtain the dynamic speed range. The ship's body dynamics parameters and loading status parameters are collected, and the acceleration safety limit threshold is calculated; the acceleration safety limit threshold is adjusted based on the berthing phase to obtain the acceleration dynamic range.

6. The AI ​​processing method for lidar point cloud data of bulk carrier berthing as described in claim 5, characterized in that, The specific steps for initial screening of the target points include: The state equations are based on velocity, acceleration, and trajectory continuity indices, and the observation equations are based on the motion vector of the target point. A state transition matrix is ​​set according to the ship's berthing stage. Based on the state equations, observation equations, and state transition matrix, the state estimation covariance matrix is ​​obtained through Kalman filtering. The confidence level of the corresponding target point is calculated based on the trace of the state estimation covariance matrix; the confidence level is negatively correlated with the trace of the state estimation covariance matrix. If the confidence level is greater than or equal to the preset confidence level threshold, the corresponding target point is marked as a valid ship point, and the weight of the valid ship point is set to 1. Otherwise, retain the coordinates of the corresponding target point and set the weight of the corresponding target point to n; where n is less than 1.

7. The AI ​​processing method for lidar point cloud data of bulk carrier berthing as described in claim 6, characterized in that, The physical model of rain and snow is constructed as follows: LiDAR point cloud data, rain and snow characteristic data, and reference point cloud data under different rain and snow intensity levels are collected simultaneously under different berthing scenarios to obtain a related dataset; an initial model is constructed based on lidar scattering theory, and the model parameters of the initial model are fitted using the related dataset as a sample; Density clustering algorithm is used to cluster the rain and snow anomalies to obtain different rain and snow point clusters; the neighborhood radius of the cluster is positively correlated with the length of the aggregation window; Fit the motion trajectory of the rain and snow droplets, and calculate the falling speed of the rain and snow based on the vertical component of the motion trajectory; The average physical distance between rain and snow anomalies within a rain and snow cluster is calculated to obtain the rain and snow particle size. The rain and snow physical model is obtained by updating the model parameters of the initial model based on the falling speed and particle size of the rain and snow.

8. The AI ​​processing method for lidar point cloud data of bulk carrier berthing as described in claim 7, characterized in that, The specific steps for point cloud repair and compensation are as follows: The ship's structural constraints are determined based on the ship's body dynamics parameters and loading state parameters, and the rain and snow physical model, ship structural constraints, and motion characteristic range are used as constraint conditions. A physical-guided generative adversarial network is constructed based on the aforementioned constraints, and the associated dataset is used as training samples. The physical-guided generative adversarial network is pre-trained using a model-independent meta-learning algorithm. Using the aggregated point cloud as the base input, a pre-trained physical-guided generative adversarial network is used to generate a pre-compensated point cloud. Calculate the similarity index between the preliminary compensated point cloud and the effective points of the ship. If the similarity index is greater than a preset index threshold, then the preliminary compensated point cloud is taken as the final compensated point cloud. Otherwise, adjust the model parameters of the rain and snow physics model and regenerate the preliminary compensated point cloud based on the pre-trained physics-guided generative adversarial network.

9. The AI ​​processing method for lidar point cloud data of bulk carrier berthing as described in claim 8, characterized in that, The similarity index is calculated as follows: Feature extraction was performed on the point cloud after preliminary compensation and the ship point cloud composed of all valid ship points, and the similarity coefficient of each feature was calculated. A similarity index is obtained by weighting the similarity coefficients of each feature based on the berthing stage and then summing the similarity coefficients of each feature based on the weighting coefficients. The features include motion features, which are calculated as follows: calculate the motion vector of the point cloud after preliminary compensation, and divide the dynamic velocity range into N velocity sub-ranges; where N is a positive integer; The percentage of points whose velocities lie in each velocity sub-interval in the motion vector of the point cloud after preliminary compensation and the percentage of points whose velocities lie in each velocity sub-interval in the motion vector of the ship point cloud are respectively counted to obtain the velocity distribution vectors of the point cloud and the ship point cloud after preliminary compensation. The cosine similarity between the velocity distribution vector of the point cloud after preliminary compensation and the velocity distribution vector of the ship's point cloud is calculated to obtain the similarity coefficient of the motion features.

10. An AI processing system for lidar point cloud data of bulk carrier berthing, used to implement the AI ​​processing method for lidar point cloud data of bulk carrier berthing as described in any one of claims 1-9, characterized in that, It includes a data acquisition module, a point cloud filtering module, a point cloud compensation module, and a ship control module; The data acquisition module is used to simultaneously acquire lidar point cloud data and ship attitude sensor data; the data acquisition module is also used to set target points; calculate the motion vector of each target point based on the lidar point cloud data, and calculate the ship's motion parameters in combination with the ship attitude sensor data; The point cloud filtering module performs initial screening of the target points based on the motion vectors and motion parameters, and sets weights for each target point after initial screening; based on the weights of each target point, it performs anomaly detection and temporal aggregation on the lidar point cloud data to obtain rain and snow anomaly points and aggregated point clouds; The point cloud compensation module is used to calculate the feature parameters of the rain and snow anomaly points and construct a rain and snow physical model based on the feature parameters; the point cloud compensation module is also used to perform point cloud repair and compensation based on the rain and snow physical model and the aggregated point cloud to obtain the compensated point cloud; The ship control module analyzes the ship's berthing status based on the compensated point cloud and controls the ship to berth based on the berthing status.

Images