Method, system, equipment and storage medium for monitoring ship posture during berthing and unberthing operations

Abstract

The embodiments of the present application provide a method, system, device and storage medium for monitoring the posture of ships in berthing and unberthing operations, which belongs to the field of ship driving technology. The method collects the first point cloud data of the current target ship through a shore-based laser radar array, pre-processes the first point cloud data to obtain the second point cloud data, performs feature extraction processing on the second point cloud data to obtain the hull feature point cloud data, determines the current posture of the ship based on the hull feature point cloud data, determines the target posture and target speed of the ship based on the current posture of the ship, ship motion parameters, environmental parameters and tugboat maneuvering parameters through a posture prediction model, determines the collision probability based on the target posture and target speed of the ship, and sends an abnormal warning to the tugboat side terminal when the collision probability is greater than the first threshold value. The present application can remind tugboats in berthing and unberthing operations of the collision risk of ships, thereby improving the safety of berthing and unberthing operations.

Description

Technical Field

[0001] The present application relates to the field of ship driving technology, and in particular to a method, system, device and storage medium for monitoring the posture of a ship during berthing and unberthing operations. Background Art

[0002] As the tonnage and density of ships at port terminals continue to increase, ships with large tonnage and high inertia have shortcomings such as poor stability and poor operability when berthing and unberthing, resulting in low efficiency in berthing and unberthing operations and the susceptibility to safety accidents. Tugboats are special operation vessels that provide professional water services such as towing, pushing, escorting, and emergency rescue for large cargo ships. They are an important safety guarantee for cargo ships entering and leaving the port and berthing and unberthing, and are an indispensable part of port production. During the berthing and unberthing process of large ships, multiple tugboats may be required to assist, and the tugboats are operated by different drivers. During collaborative berthing and unberthing, due to differences in driver experience and operation levels, the drivers cannot promptly perceive the risk of ship collision, resulting in a high safety risk in berthing and unberthing operations. Summary of the Invention

[0003] The main purpose of the embodiments of the present application is to propose a method, system, device and storage medium for monitoring the posture of ships in berthing and unberthing operations, aiming to provide collision risk reminders to tugboats in berthing and unberthing operations, thereby improving the safety of berthing and unberthing operations.

[0004] To achieve the above objectives, one aspect of an embodiment of the present application provides a method for monitoring the posture of a vessel during berthing and unberthing operations, comprising the following steps:

[0005] Collect the first point cloud data of the current target ship through the shore-based lidar array;

[0006] Preprocessing the first point cloud data to obtain second point cloud data;

[0007] performing feature extraction processing on the second point cloud data to obtain hull feature point cloud data;

[0008] Determining the current posture of the ship based on the hull feature point cloud data;

[0009] Determining the target attitude and target speed of the ship according to the current attitude of the ship, ship motion parameters, environmental parameters and tugboat maneuvering parameters through an attitude prediction model;

[0010] The collision probability is determined according to the target posture of the ship and the target speed of the ship, and when the collision probability is greater than a first threshold, an abnormal warning is sent to the tugboat-side terminal.

[0011] In some embodiments, preprocessing the first point cloud data to obtain second point cloud data includes the following steps:

[0012] Calculating the average distance between the discrete points in the first point cloud data and each of the neighborhood points of the discrete points, and determining a global mean and a standard deviation based on all the average distances;

[0013] Determining whether the discrete point is a noise point according to the average distance, the global mean, and the standard deviation;

[0014] filtering out noise points in the first point cloud data to obtain third point cloud data;

[0015] Water surface plane points are identified and filtered out from the third point cloud data to obtain second point cloud data.

[0016] In some embodiments, performing feature extraction processing on the second point cloud data to obtain hull feature point cloud data includes the following steps:

[0017] Based on different scales of the Gaussian pyramid, Gaussian blur processing is performed on the second point cloud data to obtain Gaussian blurred point cloud data at different scales;

[0018] Construct a differential Gaussian pyramid based on the difference of Gaussian blurred point cloud data at adjacent scales;

[0019] Performing extreme value detection on the differential Gaussian pyramid within a preset image area to determine feature key points;

[0020] Calculating the gradient information of each pixel in the neighborhood of the feature key point;

[0021] Constructing a gradient direction histogram of the feature key point according to the gradient information of each pixel, and determining a feature vector of the feature key point according to the gradient direction histogram;

[0022] The hull feature point cloud data is determined according to the feature vectors of each of the feature key points.

[0023] In some embodiments, determining the current posture of the ship based on the hull feature point cloud data includes the following steps:

[0024] Registering the hull feature point cloud data to a ground coordinate system using an iterative closest point algorithm;

[0025] determining, based on the registered hull feature point cloud data, a set of inner points of a first straight line formed by any two points in the hull feature point cloud data;

[0026] Selecting the interior point set with the largest number of interior points from the interior point sets of each of the first straight lines as the maximum consistent set;

[0027] With the goal of minimizing the sum of squares of distances from all interior points to the straight line, the interior points in the maximum consistent set are fitted to determine the target straight line;

[0028] The current posture of the ship is determined according to the target straight line.

[0029] In some embodiments, determining, based on the registered hull feature point cloud data, a set of inner points of a first straight line formed by any two points in the hull feature point cloud data comprises the following steps:

[0030] Select any two points in the registered hull feature point cloud data to form a first straight line;

[0031] Calculate first distances from all points in the hull feature point cloud data to the first straight line;

[0032] Points whose first distance is greater than a distance threshold are determined as interior points of the first straight line, to obtain an interior point set of the first straight line.

[0033] In some embodiments, determining the target ship attitude and target ship speed based on the current ship attitude, ship motion parameters, environmental parameters, and tugboat maneuvering parameters using an attitude prediction model comprises the following steps:

[0034] Inputting the current attitude of the ship, ship motion parameters, environmental parameters and tugboat maneuvering parameters into an attitude prediction model to obtain candidate ship attitudes and candidate ship speeds at various positions;

[0035] Determine the target location based on the shore-based location and docking range;

[0036] The candidate ship postures and the candidate ship speeds at various positions are screened according to the target position to obtain a target ship posture and a target ship speed.

[0037] In some embodiments, determining the collision probability according to the ship target posture and the ship target speed includes the following steps:

[0038] Determining a ship normal velocity according to the ship target posture and the ship target velocity;

[0039] The collision probability is determined based on the normal velocity of the ship.

[0040] To achieve the above objectives, another aspect of the present application provides a system for monitoring the posture of a vessel during berthing and unberthing operations, comprising:

[0041] The first module is used to collect the first point cloud data of the current target ship through a shore-based laser radar array;

[0042] A second module is used to preprocess the first point cloud data to obtain second point cloud data;

[0043] A third module is used to perform feature extraction processing on the second point cloud data to obtain hull feature point cloud data;

[0044] The fourth module is used to determine the current posture of the ship based on the hull feature point cloud data;

[0045] A fifth module is configured to determine a target ship attitude and a target ship speed according to the current attitude of the ship, ship motion parameters, environmental parameters, and tugboat maneuvering parameters through an attitude prediction model;

[0046] The sixth module is used to determine the collision probability according to the target posture of the ship and the target speed of the ship, and send an abnormal warning to the tugboat side terminal when the collision probability is greater than a first threshold.

[0047] To achieve the above-mentioned purpose, another aspect of an embodiment of the present application proposes an electronic device, which includes a memory, a processor, a program stored on the memory and executable on the processor, and a data bus for realizing connection and communication between the processor and the memory. When the program is executed by the processor, the method described in the above embodiment is implemented.

[0048] To achieve the above-mentioned purpose, another aspect of an embodiment of the present application proposes a storage medium, which is a computer-readable storage medium used for computer-readable storage. The storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the method described in the above embodiment.

[0049] The present application proposes a method, system, device and storage medium for monitoring the attitude of a ship in berthing and unberthing operations. The method collects first point cloud data of the current target ship through a shore-based laser radar array, pre-processes the first point cloud data to obtain second point cloud data, performs feature extraction processing on the second point cloud data to obtain hull feature point cloud data, determines the current attitude of the ship based on the hull feature point cloud data, determines the target attitude and target speed of the ship based on the current attitude of the ship, ship motion parameters, environmental parameters and tugboat maneuvering parameters through an attitude prediction model, determines the collision probability based on the target attitude and target speed of the ship, and sends an abnormal warning to the tugboat side terminal when the collision probability is greater than a first threshold. The present application can predict the future target attitude and target speed of the ship, and then predict the future collision probability, thereby reminding the tugboat driver in advance and improving the safety of berthing and unberthing operations. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 This is a flow chart of a method for monitoring the posture of a ship in berthing and unberthing operations provided by an embodiment of the present application;

[0051] Figure 2This is a schematic diagram of a berthing and unberthing operation scenario provided by an embodiment of the present application;

[0052] Figure 3 This is a logic diagram for implementing the method for monitoring the posture of a ship during berthing and unberthing operations provided in an embodiment of the present application;

[0053] Figure 4 This is a schematic diagram of the hardware structure of the electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0054] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0055] It should be noted that although the system is divided into functional modules and the flowcharts illustrate a logical sequence, in some cases, the steps shown or described may be performed in a different order than the module division in the system or the order in the flowcharts. The terms "first," "second," and so on in the specification, claims, and drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or precedence.

[0056] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this application pertains. The terms used herein are for the purpose of describing the embodiments of this application only and are not intended to limit this application.

[0057] The embodiments of the present application provide a method, system, device and storage medium for monitoring the posture of a ship during berthing and unberthing operations, which are intended to provide a collision risk reminder to a tugboat during berthing and unberthing operations, thereby improving the safety of berthing and unberthing operations.

[0058] The method, system, device and storage medium for monitoring the posture of a ship in berthing and unberthing operations provided in the embodiments of the present application are specifically illustrated through the following embodiments. First, the method for monitoring the posture of a ship in berthing and unberthing operations in the embodiments of the present application is described.

[0059] The method for monitoring the posture of a ship in berthing and unberthing operations provided in the embodiment of the present application relates to the field of ship driving technology. The method for monitoring the posture of a ship in berthing and unberthing operations provided in the embodiment of the present application can be applied to a terminal, can be applied to a server side, or can be software running in a terminal or a server side. In some embodiments, the terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, etc.; the server side can be configured as an independent physical server, or as a server cluster or distributed system composed of multiple physical servers, or as a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms; the software can be an application that implements the method for monitoring the posture of a ship in berthing and unberthing operations, etc., but is not limited to the above forms.

[0060] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, and the like. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, and the like that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments in which tasks are performed by remote processing devices connected via a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media, including storage devices.

[0061] Figure 1 This is an optional flow chart of the method for monitoring the posture of a ship in berthing and unberthing operations provided in an embodiment of the present application. Figure 1 The method may include but is not limited to steps S101 to S106.

[0062] Step S101, collecting first point cloud data of the current target ship through a shore-based laser radar array;

[0063] Step S102, preprocessing the first point cloud data to obtain second point cloud data;

[0064] Step S103, performing feature extraction processing on the second point cloud data to obtain hull feature point cloud data;

[0065] Step S104, determining the current posture of the ship based on the hull feature point cloud data;

[0066] Step S105, determining the target attitude and target speed of the ship according to the current attitude of the ship, the ship motion parameters, the environmental parameters and the tugboat maneuvering parameters through the attitude prediction model;

[0067] Step S106: determining the collision probability based on the target ship posture and the target ship speed, and when the collision probability is greater than a first threshold, sending an abnormal warning to the tugboat terminal.

[0068] According to some embodiments of this application, please refer to Figure 2 This embodiment provides a port berthing and unberthing operation scenario, specifically:

[0069] Data measurement is performed on the shore, and the shore-based data measurement includes a shore-based lidar array 200 and a shore-based communication module 100. Data processing is performed on the ship side, and the ship-side data processing may include a ship-borne communication module and a central processing unit module. Enhanced display modules can be set at the ship side (ship 300), the shore side and the tugboat side (tugboat 500). During berthing and unberthing operations, the shore-based ranging module processes the collected information and transmits it to the shore-based communication module, which transmits it to the tugboat communication module 400 at the tugboat side for early warning. Alternatively, the shore-based ranging module summarizes the collected information and transmits it to the shore-based communication module, which transmits it to the ship-borne communication module at the ship side for analysis and processing, and then transmits the processing results to the tugboat communication module at the tugboat side for early warning.

[0070] A low-latency, high-reliability communication solution can be implemented between the shore-based, ship-borne, and tugboat communication modules. For example, a dual communication channel solution can be employed. The primary channel utilizes the ultra-high-bandwidth, low-latency 5G network, primarily for high-frequency, high-data-volume, real-time communications. The secondary channel utilizes VHF / UHF radio frequencies, widely used in maritime communications for wide coverage and strong anti-interference capabilities, for low-frequency control commands, status information, and emergency communications. This ensures that basic communications can be maintained even when the 5G network fails or coverage is insufficient. Advanced encryption technologies (such as AES-256) can be used to encrypt transmitted data, and a two-way authentication mechanism can be used to ensure device legitimacy. Multi-path transmission and load balancing can be used to improve transmission reliability and anti-interference capabilities. Auxiliary technologies include deploying edge computing nodes on both the ship and shore sides for preliminary data processing and filtering, reducing the amount of transmitted data and improving communication efficiency, as well as data compression to reduce latency. Real-time monitoring of the communication system's operational status allows the system to automatically switch to the secondary channel if the primary channel fails, providing stable and reliable communication.

[0071] In step S101 of some embodiments, the target ship is a ship that needs to perform berthing and unberthing operations and is assisted by a tugboat. Point cloud data refers to a vector set consisting of a set of discrete points in a three-dimensional coordinate system. These points are usually captured by a laser scanner, camera or other three-dimensional sensor, and the three-dimensional coordinate information (X, Y, Z) of each point is recorded. In addition to the basic three-dimensional coordinates, point cloud data may also contain additional attributes such as color information (RGB) and reflection intensity information (Intensity). Point cloud data can provide high-precision geometric information on the surface of an object, and can intuitively represent details such as the shape, surface and texture of the object, with the characteristics of high precision and high resolution. Point cloud data is acquired by a laser radar. The laser radar transmits a laser radar and measures the time from the transmission to the reflection of the transmitted signal. Combined with the position and posture information of the laser radar device, the returned laser signal and radar signal are integrated to calculate the three-dimensional coordinates of each reflection point.

[0072] In step S102 of some embodiments, the first point cloud data is preprocessed to filter out abnormal data or invalid data. The preprocessing process can use a statistical outlier removal algorithm to filter out noise data, and a RANSAC algorithm to filter out invalid data.

[0073] In some embodiments, step S102 may include but is not limited to steps S201 to S204:

[0074] Step S201, calculating the average distance between the discrete points in the first point cloud data and each area point of the discrete points, and determining the global mean and standard deviation based on all the average distances;

[0075] Step S202, determining whether the discrete point is a noise point based on the average distance, global mean and standard deviation;

[0076] Step S203, filtering out noise points in the first point cloud data to obtain third point cloud data;

[0077] Step S204 : identifying and filtering out water surface plane points from the third point cloud data to obtain second point cloud data.

[0078] In this embodiment, the Statistical Outlier Removal (SOR) algorithm is a statistical method for removing noise points from point cloud data. The core idea is to identify and remove points that are significantly out of the normal range by analyzing the distance distribution between each point in the point cloud and its neighboring points. For each point p in the first point cloud data received by the lidar, i, define its neighborhood point set N(p i ) is a set of its nearest k neighbor points. The specific neighborhood definition can be implemented using the nearest neighbor search algorithm (such as kd tree). Calculate the point p i The average distance d to its neighboring points i The calculation method is:

[0079]

[0080] Among them, ||p i -p j || represents point p i and neighborhood point p j The Euclidean distance between .

[0081] The average distance d to all points i Perform statistical analysis and calculate the global mean μ and standard deviation σ of all average distances as follows:

[0082]

[0083]

[0084] Where n is the total number of points in the point cloud.

[0085] For each point p i , according to its average distance d i The relationship between the global mean μ and the standard deviation σ determines whether the point is a noise point. i The average distance d i If the value exceeds a certain multiple α (usually set to 1 or 3) of the mean μ, it will be marked as a noise point:

[0086] d i >μ+ασ;

[0087] The RANSAC (random sampling consensus) algorithm is used to identify and remove surface points from point cloud data. RANSAC identifies surface points by iteratively fitting a model (such as a plane) and evaluating the consistency of the data with the model. In the plane segmentation problem, a plane model is selected. The mathematical representation of a plane is:

[0088] ax+by+cz+d=0

[0089] Where (a, b, c) is the plane's normal vector, and d is the offset of the plane equation. Three points p1, p2, and p3 are randomly selected from the point cloud data to define a plane. The plane's normal vector and offset are calculated based on the three randomly selected points. Assuming the coordinates of the three points are p1 = (x1, y1, z1), p2 = (x2, y2, z2), and p3 = (x3, y3, z3), the normal vector n = (a, b, c) can be obtained by calculating the cross product of the vectors u = p2 - p1 and v = p3 - p1, as follows:

[0090] n=u×v=((y2-y1)(z3-z1)-(z2-z1)(y3-y1),(z2-z1)(x3-x1)-(x2-x1)(z3-z1),(x2-x1)(y3-y1)-(y2-y1)(x3-x1));

[0091] The d in the plane equation can be found by substituting a known point into the plane equation:

[0092] d=ax1+by1+cz1;

[0093] For each point p = (x, y, z) in the point cloud, calculate its distance to the fitting plane:

[0094]

[0095] If the distance is less than a predetermined threshold ∈, the point is considered to be an inlier (belongs to the plane). Repeat the random sampling and model fitting steps, and record the plane model with the most inliers in each fitting. The number of iterations is Where P is the desired confidence level (e.g., 99%), w is the probability that a point belongs to a plane, and k is the number of points sampled each time (3 for a plane). After N iterations, the plane model with the most inliers is selected as the final model, and these inliers are removed from the point cloud data. The remaining points are non-ground points.

[0096] In some embodiments, in step S103, after collecting point cloud data around the moored vessel in real time and filtering out invalid data, a feature extraction algorithm based on the scale-invariant feature transform (SIFT) principle can be used to identify hull feature points from the point cloud data. In another embodiment, a deep learning algorithm can also be used for feature extraction to obtain hull features.

[0097] In some embodiments, step S103 may include but is not limited to steps S301 to S306:

[0098] Step S301, performing Gaussian blur processing on the second point cloud data based on different scales of the Gaussian pyramid to obtain Gaussian blurred point cloud data at different scales;

[0099] Step S302, constructing a differential Gaussian pyramid based on the differences of Gaussian blurred point cloud data at adjacent scales;

[0100] Step S303: performing extreme value detection on the differential Gaussian pyramid within a preset image area to determine feature key points;

[0101] Step S304, calculating the gradient information of each pixel in the neighborhood of the feature key point;

[0102] Step S305, constructing a gradient direction histogram of the feature key point based on the gradient information of each pixel, and determining a feature vector of the feature key point based on the gradient direction histogram;

[0103] Step S306: determining the hull feature point cloud data according to the feature vectors of each feature key point.

[0104] In this embodiment, the Scale-Invariant Feature Transform (SIFT) is a widely used feature extraction algorithm that is scale and rotation invariant. The SIFT algorithm extracts the ship's feature points as follows:

[0105] First, a Gaussian pyramid is constructed to perform Gaussian blur on the second point cloud data at different scales. The Gaussian function is defined as:

[0106]

[0107] Use the Gaussian kernel to blur the second point cloud data image as follows:

[0108] L(x,y,σ)=G(x,y,ε)*I(x,y);

[0109] Where L(x,y,σ) is the blurred result of image I(x,y) at scale σ, and the symbol * represents the convolution operation. By taking the difference of Gaussian blurred images at adjacent scales, we construct a Difference of Gaussian (DoG) pyramid as follows:

[0110] D(x,y,σ)=L(x,y,kσ)*L(x,y,σ);

[0111] In the DoG space, each pixel point is checked for neighborhood extreme values ​​in a preset image area (e.g., 3x3x3) (i.e., checked at the current scale and adjacent scales). If the point is a maximum or minimum value in its neighborhood, it is considered to be a candidate feature key point.

[0112] Then, the Taylor expansion is used to accurately locate the key points of the features and remove points with low contrast and strong edge response. The partial derivative form of the Taylor expansion is:

[0113]

[0114] Where H is the Hessian matrix, is the gradient of the DoG function. Calculate the extreme points after the shift and discard them if their DoG value is below a predetermined threshold. Use the trace and determinant of the Hessian matrix to distinguish edge responses from corner points, discarding points with strong edge responses.

[0115] Based on the above processing, the gradient magnitude and direction of each pixel in the neighborhood of the feature key point are calculated:

[0116]

[0117] Within the neighborhood of a keypoint, a gradient direction histogram is constructed with the keypoint as the center. Each bin of the histogram represents a direction (usually eight directions), and the gradient magnitude of each pixel is added to the bin of the corresponding direction as a weight. The dominant direction of the gradient direction histogram is used as the primary direction of the keypoint. If the peak value of a particular direction exceeds a preset proportion (e.g., 80%) of the primary direction, a new keypoint is generated and assigned to that direction.

[0118] Finally, the neighborhood of the keypoint is divided into 4x4 subregions centered on the keypoint. A gradient direction histogram (typically eight directions) is constructed for each subregion, resulting in a total of 4x4x8 = 128-dimensional feature vectors for the keypoints. Given the dramatic changes in illumination in port operations, the feature vectors should be normalized to increase robustness to illumination variations. After obtaining the feature vectors for each keypoint, data classification is performed on the point cloud image based on the feature vectors to determine the hull feature point cloud data.

[0119] In step S104 of some embodiments, the hull feature point cloud data may represent the outer contour of the hull, and an iterative closest point (ICP) algorithm may be applied to the hull feature point cloud data to accurately calculate the position and posture of the berthing and unberthing ship.

[0120] In some embodiments, step S104 may include but is not limited to steps S401 to S405:

[0121] Step S401, registering the hull feature point cloud data to the ground coordinate system through an iterative closest point algorithm;

[0122] Step S402, determining, based on the registered hull feature point cloud data, a set of inner points of a first straight line formed by any two points in the hull feature point cloud data;

[0123] Step S403, selecting the inlier point set with the largest number of inliers from the inlier point sets of each first straight line as the maximum consistent set;

[0124] Step S404, with the goal of minimizing the sum of squares of distances from all inliers to the straight line, fitting the inliers in the maximum consistent set to determine the target straight line;

[0125] Step S405: determining the current posture of the ship according to the target straight line.

[0126] In some embodiments, step S402 may include but is not limited to steps S501 to S503:

[0127] Step S501, selecting any two points in the registered hull feature point cloud data to form a first straight line;

[0128] Step S502, calculating the first distances from all points in the hull feature point cloud data to the first straight line;

[0129] Step S503: Determine the points whose first distance is greater than the distance threshold as the interior points of the first straight line, and obtain the interior point set of the first straight line.

[0130] In this embodiment, the ICP algorithm is a classic method for point cloud registration, which minimizes the distance between corresponding point sets of the source point cloud and the target point cloud by iteratively optimizing the transformation matrix.

[0131] Assume there are two point clouds: source point cloud P = {p1,p2,...,p n} and target point cloud Q={q1,q2,...,q n The goal of the ICP algorithm is to find a rigid body transformation (R, t) so that the transformed source point cloud is aligned with the target point cloud as much as possible. The rigid body transformation includes the rotation matrix R and the translation vector t. The transformation formula is:

[0132] p' i =Rp i +t;

[0133] ICP is mainly implemented through iterative calculation. The main steps include:

[0134] (1) Initialization: Select the initial transformation matrix (R0, t0).

[0135] (2) Closest point pairing: For each source point p i , find the nearest point q in the target point cloud i .

[0136] (3) Optimize transformation: Calculate the new transformation matrix (R, t) by minimizing the square error between paired points.

[0137] (4) Update point cloud: Apply the new transformation matrix to update the source point cloud.

[0138] (5) Check convergence: If the change of the transformation matrix is ​​less than the set threshold or the maximum number of iterations is reached, the iteration is stopped.

[0139] After projecting and registering the point cloud coordinates to the ground coordinates using the ICP algorithm, the RANSAC algorithm is used to identify the contour of the docking and unberthing ship. The main purpose of the RANSAC algorithm used in this embodiment is to calculate the outer contour of the docking and unberthing ship through point cloud fitting, thereby inferring the normal plane of the ship. The specific algorithm steps include:

[0140] 1) Randomly select point pairs from the outer contour of the wheel (i.e., the hull feature point cloud data after registration) and calculate the line parameters. n} randomly select two points p i =(x i ,y i ,z i ) and p j =(x j ,y j ,z j ), the parametric equation of the straight line formed by these two points can be expressed as the parametric equation of the vector: r(t)=p i +t(p i -p j ), where t is a parameter, then the direction vector of a straight line on the contour surface is d = p i -p j .

[0141] 2) Calculate the distance from other points to the line and count the inner points. For each point p in the point cloud k =(x k ,y k ,z k ), calculate its distance to the straight line where d×(p k -p i ) represents the direction vector d and p k -p i The cross product of the cross product represents the point p k Distance to the line. Compare the distance d of each point k With the set distance threshold ∈, if d k <∈, then p kIt is considered as an inlier and included in the inlier set I.

[0142] 3) Repeat steps 1 and 2 for multiple times (set as N times), randomly select two points each time, calculate the straight line and count the number of inliers. In all iterations, find the point set I that has the largest number of inliers. max , denoted as the maximum consensus set.

[0143] 4) For the maximum consistent set I max All the interior points in are fitted using the least squares method to obtain the final straight line parameters. The goal of the least squares method is to find a straight line that minimizes the sum of the squares of the distances from all interior points to the line. Assuming that the equation of the fitted line is: ax+by+cz+d=0, the objective function to be minimized is: By solving the above objective function, the parameters a, b, c, and d of the best fitting line are obtained, thereby obtaining the target line.

[0144] Through several steps, the point cloud data can be effectively fitted to the ship's attitude line (i.e., target line) during the berthing and unberthing process, and then the distance between the ship and the berth front and the point cloud P of the bow position can be calculated. 艏 and the point cloud P at the stern position 艉 On this basis, the current posture of the ship is further calculated, including the berthing angle θ and the offshore distance d between the ship and the dock 艏 and d 艉 wait.

[0145] In step S105 of some embodiments, the attitude prediction model is a trajectory prediction model, and the predicted trajectory tangent represents the vessel's attitude at various locations. Vessel motion parameters include the vessel's current position, current speed, and acceleration, as well as parameters such as the vessel's weight, length, and width. Environmental parameters include natural factors such as wind, waves, currents, and tides. Tugboat maneuvering parameters include the tugboat's current direction, output power, and power utilization.

[0146] In some embodiments, step S105 may include but is not limited to steps S601 to S603:

[0147] Step S601: Input the current attitude of the ship, ship motion parameters, environmental parameters and tugboat maneuvering parameters into an attitude prediction model to obtain candidate ship attitudes and candidate ship speeds at various positions;

[0148] Step S602, determining the target location based on the shore-based location and the docking range;

[0149] Step S603 , screening the candidate ship postures and candidate ship speeds at various positions according to the target position to obtain the target ship posture and target ship speed.

[0150] In this embodiment, the current ship attitude, ship motion parameters, environmental parameters, and tugboat maneuvering parameters are input into the attitude prediction model to generate a predicted trajectory. This trajectory includes candidate ship attitudes and speeds at discrete locations. In a docking scenario, the target location is determined based on the shore-based location and the preset docking range included in the predicted trajectory. Specifically, the target location is determined by the intersection of a circle centered on the shore-based location and with a radius within the docking range. The candidate ship attitudes and speeds at each location are then screened based on the target location. The attitude and speed corresponding to the location closest to the target location are selected to obtain the target ship attitude and speed.

[0151] In step S106 of some embodiments, the target posture of the ship includes the berthing angle. The normal berthing speed of the ship within the berthing range can be obtained based on the target posture and target speed of the ship obtained according to the model. The normal berthing speed determines the collision probability. When the collision probability is greater than the first threshold, indicating that the ship has a greater risk of berthing collision, an abnormal warning is sent to the tugboat side terminal.

[0152] During the berthing and unberthing process, the normal approaching speed is one of the key parameters directly related to the safety of the ship's berthing operation. Traditional methods make it difficult to accurately determine the ship's normal approaching speed. This embodiment uses a model to predict the ship's target posture and target speed and further determine the normal approaching speed, which can calculate the above key parameters more accurately and in real time.

[0153] In some embodiments, step S106 may include but is not limited to steps S701 to S702:

[0154] Step S701, determining the ship normal velocity according to the ship target posture and the ship target velocity;

[0155] Step S702: determining the collision probability according to the normal velocity of the ship.

[0156] In this embodiment, the target posture of the ship includes the berthing angle. Based on the geometric relationship, the normal speed of the ship (i.e., the normal berthing speed) can be determined according to the target posture of the ship and the target speed of the ship. Multiplying the normal speed of the ship by a certain coefficient can represent the collision probability, that is, the greater the normal speed of the ship, the greater the collision probability.

[0157] According to some embodiments of this application, please refer to Figure 3 The implementation logic of the method for monitoring the posture of a ship in berthing and unberthing operations according to the embodiment of the present application is described in detail as follows:

[0158] First, each cycle reads wind, wave, and current data, vessel parameters, tugboat parameters, and other data. It also receives LiDAR data i and determines whether it is correct. If so, it caches the data into memory. If not, it waits for the next cycle of LiDAR data i. It then determines whether all LiDAR data have been received. If not, it returns to the step of receiving LiDAR data i. If so, it calculates and predicts the ship's (large ship's) posture. It determines whether the docked ship's posture is normal. If not, it issues an abnormality warning. If so, it determines whether the docking and unberthing operations are complete. If so, it issues a task completion command. If not, it issues a task continuation command.

[0159] The present application also provides a system for monitoring the posture of a vessel during berthing and unberthing operations, comprising:

[0160] The first module is used to collect the first point cloud data of the current target ship through a shore-based laser radar array;

[0161] The second module is used to preprocess the first point cloud data to obtain second point cloud data;

[0162] The third module is used to perform feature extraction processing on the second point cloud data to obtain hull feature point cloud data;

[0163] The fourth module is used to determine the current posture of the ship based on the hull feature point cloud data;

[0164] The fifth module is used to determine the target attitude and target speed of the ship according to the current attitude of the ship, ship motion parameters, environmental parameters and tugboat maneuvering parameters through the attitude prediction model;

[0165] The sixth module is used to determine the collision probability based on the ship's target posture and target speed. When the collision probability is greater than a first threshold, an abnormal warning is sent to the tugboat terminal.

[0166] It can be understood that the contents of the above-mentioned embodiments of the method for monitoring the attitude of ships during berthing and unberthing operations are applicable to the embodiments of this system. The functions specifically implemented by the embodiments of this system are the same as those in the above-mentioned embodiments of the method for monitoring the attitude of ships during berthing and unberthing operations, and the beneficial effects achieved are also the same as the beneficial effects achieved by the above-mentioned embodiments of the method for monitoring the attitude of ships during berthing and unberthing operations.

[0167] The present application also provides an electronic device comprising: a memory, a processor, a program stored in the memory and executable on the processor, and a data bus for enabling communication between the processor and the memory. When the program is executed by the processor, the aforementioned method for monitoring the posture of a vessel during berthing and unberthing operations is implemented. The electronic device may be any intelligent terminal, including a tablet computer and an in-vehicle computer.

[0168] See also Figure 4 , Figure 4 The hardware structure of an electronic device according to another embodiment is shown. The electronic device includes:

[0169] The processor 401 may be implemented as a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application.

[0170] The memory 402 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 402 can store an operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 402 and is called by the processor 401 to execute the method for monitoring the posture of a ship in berthing and unberthing operations in the embodiments of this application.

[0171] Input / output interface 403, used to implement information input and output;

[0172] Communication interface 404, used to implement communication interaction between this device and other devices, which can be achieved through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WiFi, Bluetooth, etc.);

[0173] Bus 405 , which transmits information between various components of the device (e.g., processor 401 , memory 402 , input / output interface 403 , and communication interface 404 );

[0174] The processor 401 , the memory 402 , the input / output interface 403 and the communication interface 404 are connected to each other in communication within the device via a bus 405 .

[0175] An embodiment of the present application also provides a storage medium, which is a computer-readable storage medium used for computer-readable storage. The storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the above-mentioned method for monitoring the posture of ships in berthing and unberthing operations.

[0176] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely arranged relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0177] The embodiments described in the embodiments of this application are intended to more clearly illustrate the technical solutions of the embodiments of this application and do not constitute a limitation on the technical solutions provided by the embodiments of this application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.

[0178] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.

[0179] The system embodiment described above is merely illustrative. The units described as separate components may or may not be physically separate, i.e., they may be located in one place or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of this embodiment.

[0180] Those skilled in the art will appreciate that all or some of the steps in the methods, systems, and functional modules / units in the devices disclosed above may be implemented as software, firmware, hardware, or appropriate combinations thereof.

[0181] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0182] It should be understood that in this application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0183] In the several embodiments provided in this application, it should be understood that the disclosed systems and methods can be implemented in other ways. For example, the system embodiments described above are merely illustrative. For example, the division of the above units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of systems or units, which can be electrical, mechanical or other forms.

[0184] The units described above as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0185] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0186] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes multiple instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present application. The aforementioned storage medium includes: various media that can store programs, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0187] The preferred embodiments of the present invention are described above with reference to the accompanying drawings, but are not intended to limit the scope of the present invention. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and essence of the present invention should be within the scope of the present invention.

Claims

1. A method for monitoring the posture of a ship during berthing and unberthing operations, characterized in that: The following steps are involved: Collect the first point cloud data of the current target ship through the shore-based lidar array; Preprocessing the first point cloud data to obtain second point cloud data; performing feature extraction processing on the second point cloud data to obtain hull feature point cloud data; Determining the current posture of the ship based on the hull feature point cloud data; Determining the target attitude and target speed of the ship according to the current attitude of the ship, ship motion parameters, environmental parameters and tugboat maneuvering parameters through an attitude prediction model; determining a collision probability based on the target ship posture and the target ship speed, and sending an abnormality warning to the tugboat terminal when the collision probability is greater than a first threshold; Determining the current posture of the ship based on the hull feature point cloud data includes the following steps: Registering the hull feature point cloud data to a ground coordinate system using an iterative closest point algorithm; determining, based on the registered hull feature point cloud data, a set of inner points of a first straight line formed by any two points in the hull feature point cloud data; Selecting the interior point set with the largest number of interior points from the interior point sets of each of the first straight lines as the maximum consistent set; With the goal of minimizing the sum of squares of distances from all interior points to the straight line, the interior points in the maximum consistent set are fitted to determine the target straight line; The current posture of the ship is determined according to the target straight line.

2. The method for monitoring the posture of a ship during berthing and unberthing operations according to claim 1, wherein: The preprocessing of the first point cloud data to obtain second point cloud data comprises the following steps: Calculating the average distance between the discrete points in the first point cloud data and each of the neighborhood points of the discrete points, and determining a global mean and a standard deviation based on all the average distances; Determining whether the discrete point is a noise point according to the average distance, the global mean, and the standard deviation; filtering out noise points in the first point cloud data to obtain third point cloud data; Water surface plane points are identified and filtered out from the third point cloud data to obtain second point cloud data.

3. The method for monitoring the posture of a ship during berthing and unberthing operations according to claim 1, wherein: The step of performing feature extraction on the second point cloud data to obtain hull feature point cloud data comprises the following steps: Based on different scales of the Gaussian pyramid, Gaussian blur processing is performed on the second point cloud data to obtain Gaussian blurred point cloud data at different scales; Construct a differential Gaussian pyramid based on the difference of Gaussian blurred point cloud data at adjacent scales; Performing extreme value detection on the differential Gaussian pyramid within a preset image area to determine feature key points; Calculating the gradient information of each pixel in the neighborhood of the feature key point; Constructing a gradient direction histogram of the feature key point according to the gradient information of each pixel, and determining a feature vector of the feature key point according to the gradient direction histogram; The hull feature point cloud data is determined according to the feature vectors of each of the feature key points.

4. The method for monitoring the posture of a ship during berthing and unberthing operations according to claim 1, wherein: The step of determining, based on the registered hull feature point cloud data, a set of inner points of a first straight line formed by any two points in the hull feature point cloud data comprises the following steps: Select any two points in the registered hull feature point cloud data to form a first straight line; Calculate first distances from all points in the hull feature point cloud data to the first straight line; Points whose first distance is greater than a distance threshold are determined as interior points of the first straight line, to obtain an interior point set of the first straight line.

5. The method for monitoring the posture of a ship during berthing and unberthing operations according to claim 1, wherein: The method of determining the target ship attitude and target ship speed according to the current ship attitude, ship motion parameters, environmental parameters and tugboat maneuvering parameters through the attitude prediction model includes the following steps: Inputting the current attitude of the ship, ship motion parameters, environmental parameters and tugboat maneuvering parameters into an attitude prediction model to obtain candidate ship attitudes and candidate ship speeds at various positions; Determine the target location based on the shore-based location and docking range; The candidate ship postures and the candidate ship speeds at various positions are screened according to the target position to obtain a target ship posture and a target ship speed.

6. The method for monitoring the posture of a ship during berthing and unberthing operations according to claim 1, wherein: Determining the collision probability according to the ship target posture and the ship target speed includes the following steps: Determining a ship normal velocity according to the ship target posture and the ship target velocity; The collision probability is determined based on the normal velocity of the ship.

7. A ship posture monitoring system for berthing and unberthing operations, characterized in that: include: The first module is used to collect the first point cloud data of the current target ship through a shore-based laser radar array; A second module is used to preprocess the first point cloud data to obtain second point cloud data; A third module is used to perform feature extraction processing on the second point cloud data to obtain hull feature point cloud data; The fourth module is used to determine the current posture of the ship based on the hull feature point cloud data; A fifth module is configured to determine a target ship attitude and a target ship speed according to the current attitude of the ship, ship motion parameters, environmental parameters, and tugboat maneuvering parameters through an attitude prediction model; a sixth module, configured to determine a collision probability based on the target ship posture and the target ship speed, and send an abnormality warning to the tugboat terminal when the collision probability is greater than a first threshold; The fourth module is specifically configured to perform the following steps: Registering the hull feature point cloud data to a ground coordinate system using an iterative closest point algorithm; determining, based on the registered hull feature point cloud data, a set of inner points of a first straight line formed by any two points in the hull feature point cloud data; Selecting the interior point set with the largest number of interior points from the interior point sets of each of the first straight lines as the maximum consistent set; With the goal of minimizing the sum of squares of distances from all interior points to the straight line, the interior points in the maximum consistent set are fitted to determine the target straight line; The current posture of the ship is determined according to the target straight line.

8. An electronic device, characterized in that: The electronic device includes a memory, a processor, a program stored in the memory and executable on the processor, and a data bus for implementing connection and communication between the processor and the memory. When the program is executed by the processor, the steps of the method described in any one of claims 1 to 6 are implemented.

9. A storage medium, which is a computer-readable storage medium and is used for computer-readable storage, characterized in that: The storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps of any one of claims 1 to 6.

Images