A ship berthing monitoring method, device, equipment and storage medium

Abstract

Embodiments of the present application provide a ship berthing monitoring method, device, equipment and storage medium, relating to the technical field of ship guiding and port intelligent monitoring, the method comprising: acquiring point cloud data corresponding to the ship, converting the point cloud data into point cloud polar coordinates and filtering to obtain ship main plane point cloud; performing point cloud density screening and point cloud continuous frame screening on the ship main plane point cloud to obtain ship target plane point cloud; calculating the local curvature of the ship target plane point cloud, determining the boundary point according to the local curvature, and determining the ship candidate region point cloud according to the boundary point and the ship target plane point cloud; calculating the covariance matrix of the ship candidate region point cloud, determining the main direction of the ship and the ship parameters according to the covariance matrix, and monitoring the berthing of the ship based on the main direction of the ship and the ship parameters. The present application can improve the accuracy and reliability of the ship berthing monitoring result.

Description

Technical Field

[0001] This invention relates to the field of ship guidance and port intelligent monitoring technology, and more specifically, to a ship berthing monitoring method, device, equipment, and storage medium. Background Technology

[0002] Ship berthing monitoring technology is one of the core technologies of port automation. This technology automates and intelligently transforms the traditional process of monitoring and guiding ships to berth, which relies on human eyes and experience, in order to improve the efficiency and safety of ship berthing. However, there are many influencing factors involved in the ship berthing process, and existing ship berthing monitoring technologies often only take into account some of these factors, resulting in significant errors in the berthing monitoring results. Summary of the Invention

[0003] In view of this, the purpose of the present invention is to provide a method, apparatus, equipment and storage medium for monitoring ship berthing, so as to solve the problem that the results of existing berthing monitoring have large errors.

[0004] To achieve the above objectives, the technical solution adopted by the present invention is as follows: In a first aspect, the present invention provides a method for monitoring ship berthing, comprising: Obtain the point cloud data corresponding to the ship, convert the point cloud data into point cloud polar coordinates and filter it to obtain the ship's main plane point cloud; The point cloud density and continuous frame filtering of the ship's main plane point cloud are performed to obtain the ship's target plane point cloud. Calculate the local curvature of the target ship planar point cloud, determine the boundary points based on the local curvature, and determine the candidate ship region point cloud based on the boundary points and the target ship planar point cloud; Calculate the covariance matrix of the point cloud of the candidate ship region, determine the ship's main direction and ship parameters based on the covariance matrix, and monitor the ship's berthing based on the ship's main direction and ship parameters.

[0005] In an optional implementation, the step of converting the point cloud data to polar coordinates and filtering it to obtain the ship's main plane point cloud includes: The initial planar point cloud is obtained by filtering the polar coordinates of the point cloud according to a preset polar coordinate threshold. A point cloud plane is generated based on the initial planar point cloud, and the ship's main planar point cloud is determined based on the residual threshold of the point cloud plane.

[0006] In an optional implementation, the step of generating a point cloud plane based on the initial planar point cloud and determining the ship's main planar point cloud based on a residual threshold of the point cloud plane includes: Based on a randomly sampled point cloud plane in the initial planar point cloud, the standard deviation from the initial point cloud to the sampled point cloud plane is calculated using an iterative algorithm. The error fluctuation corresponding to the sampling point cloud plane is generated based on the standard deviation. If the error fluctuation is greater than the residual threshold, then the residual threshold is adjusted; Repeat the above steps until the target residual threshold is obtained, and determine the ship's main plane point cloud based on the target residual threshold.

[0007] In an optional implementation, the step of performing point cloud density filtering and point cloud continuous frame filtering on the ship's main plane point cloud to obtain the ship's target plane point cloud includes: The ship's main plane point cloud is divided into multiple point cloud regions, and the point cloud density corresponding to each point cloud region is calculated; Point cloud regions with point cloud density lower than a preset density are identified as interference regions and removed, while point cloud regions with point cloud density higher than the preset density are identified as target regions. Obtain continuous frames corresponding to the target area, identify point clouds that appear less than a preset number of times in the continuous frames as interference points and remove them to obtain the ship target planar point cloud.

[0008] In an optional implementation, the step of calculating the local curvature of the ship target planar point cloud and determining the boundary points based on the local curvature includes: Calculate the mean of the point cloud normal vector and the neighborhood normal vector of the ship target planar point cloud; The neighborhood curvature of each point cloud is determined based on the mean of the point cloud normal vector and the neighborhood normal vector. Points in the domain with curvature greater than a curvature threshold are identified as boundary points.

[0009] In an optional implementation, the steps of calculating the covariance matrix of the point cloud of the candidate ship region and determining the ship's principal direction and ship parameters based on the covariance matrix include: Calculate the mean coordinates of the point cloud in the candidate ship region point cloud, and construct the covariance matrix based on the mean coordinates; Extract the largest eigenvector of the covariance matrix, determine the ship's main direction based on the largest eigenvector, and determine the ship's parameters based on the ship's main direction.

[0010] In an optional implementation, the step of determining the ship parameters based on the ship's main direction includes: Construct the directed bounding box of the ship, calculate the parameters of the directed bounding box according to the ship's main direction, and generate the minimum bounding rectangle according to the parameters of the directed bounding box. The parameters of the directed bounding box include the ship's length, ship's width, and ship's height. The relative positions of the ship and the object being moored are determined based on the mapping relationship between the center coordinates of the minimum bounding rectangle and the coordinate system of the object being moored. The ship speed is calculated based on the continuous frame difference of the center coordinates and the point cloud scanning period; Calculate the mooring angle based on the main direction of the minimum bounding rectangle and the mooring direction of the object being moored; The minimum distance between the smallest bounding rectangle and the edge of the object being approached is taken as the minimum approach distance.

[0011] Secondly, the present invention provides a ship berthing monitoring device, comprising: The point cloud acquisition module is used to acquire point cloud data corresponding to the ship, convert the point cloud data into point cloud polar coordinates and filter it to obtain the ship's main plane point cloud. The point cloud filtering module is used to perform point cloud density filtering and point cloud continuous frame filtering on the main plane point cloud of the ship to obtain the target plane point cloud of the ship. The point cloud computing module is used to calculate the local curvature of the target plane point cloud of the ship, determine the boundary points based on the local curvature, and determine the candidate region point cloud of the ship based on the boundary points and the target plane point cloud of the ship. The berthing monitoring module is used to calculate the covariance matrix of the point cloud of the candidate region of the ship, determine the main direction and parameters of the ship based on the covariance matrix, and perform berthing monitoring of the ship based on the main direction and parameters of the ship.

[0012] Thirdly, the present invention provides an electronic device including a processor and a memory, the memory storing machine-executable instructions that can be executed by the processor, the processor executing the machine-executable instructions to implement the ship berthing monitoring method described in the first aspect.

[0013] Fourthly, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the ship berthing monitoring method described in the first aspect.

[0014] This invention provides a method, apparatus, device, and storage medium for monitoring ship berthing. By performing polar coordinate filtering, point cloud density screening, and continuous frame screening on the ship's point cloud data, the main plane of the ship can be accurately extracted from the complex point cloud data, ensuring the continuity of the main plane. The main direction of the ship is determined by the covariance matrix, and then the ship's berthing is monitored in all aspects by using the main direction and ship parameters, thereby improving the accuracy and reliability of the ship berthing monitoring results.

[0015] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0016] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 A block diagram of an electronic device provided by an embodiment of the present invention is shown; Figure 2 A schematic flowchart of a ship berthing monitoring method provided by an embodiment of the present invention is shown; Figure 3 A flowchart illustrating a method for acquiring planar point clouds of ship targets provided by an embodiment of the present invention is shown. Figure 4 The diagram shows a functional block diagram of a ship berthing monitoring device provided in an embodiment of the present invention.

[0018] icon: 100 - Electronic equipment; 110 - Memory; 120 - Processor; 130 - Communication module; 400 - Ship berthing monitoring device; 410 - Point cloud acquisition module; 420 - Point cloud filtering module; 430 - Point cloud computing module; 440 - Berthing monitoring module. Detailed Implementation

[0019] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0020] Therefore, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.

[0021] It should be noted that relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0022] Existing methods for monitoring ship berthing have the following limitations: Point cloud plane segmentation algorithm: Traditional solutions often use general RANSAC or DBSCAN algorithms to fit the main plane of the ship, but they do not design a dynamic residual threshold adjustment mechanism for the harbor scenario, resulting in a high misjudgment rate of plane segmentation caused by interference such as water surface clutter and wave reflection. Target plane screening algorithm: Existing methods mostly rely on clustering algorithms with fixed parameters, without combining polar coordinate thresholds and temporal consistency checks, making it difficult to distinguish between the main body of the ship and transient disturbances (such as splashing waves). Minimum bounding rectangle fitting algorithm: Traditional schemes often use AABB (axis-aligned bounding box) or fixed OBB (directed bounding box) models, which do not consider the asymmetric motion state of the ship during berthing (such as the deformation of the bounding box caused by the outward push of the bow), resulting in deviations between the rectangle parameters (length, width, height, rotation angle) and the actual ship attitude. Multi-parameter synchronous calculation algorithm: Existing technologies mostly focus on static position or obstacle detection, and have not achieved full-dimensional synchronous calculation of velocity, included angle, and overlap area spacing. Furthermore, the dynamic monitoring of the compression of the berthing ball relies on prior models and has not been combined with real-time point cloud feature matching optimization.

[0023] In order to solve the technical problems of the prior art, this application adopts the following technical solution.

[0024] Please refer to Figure 1 , Figure 1This is a block diagram of an electronic device 100 provided in this embodiment. The electronic device 100 includes a memory 110, a processor 120, and a communication module 130. The memory 110, processor 120, and communication module 130 are electrically connected to each other directly or indirectly to realize data transmission or interaction. For example, these components can be electrically connected to each other through one or more communication buses or signal lines.

[0025] The memory 110 is used to store programs or data. The memory 110 may be, but is not limited to, random access memory (RAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), etc.

[0026] The processor 120 is used to read / write data or programs stored in memory and to perform corresponding functions.

[0027] The communication module 130 is used to establish a communication connection between the electronic device 100 and other communication terminals through the network, and to send and receive data through the network.

[0028] It should be understood that, Figure 1 The structure shown is only a schematic diagram of the electronic device 100. The electronic device 100 may also include components that are larger than... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown. Figure 1 The components shown can be implemented using hardware, software, or a combination thereof.

[0029] Please refer to Figure 2 , Figure 2 This is a flowchart illustrating a ship berthing monitoring method provided in this embodiment. The method includes: S210. Obtain the point cloud data corresponding to the ship, convert the point cloud data into point cloud polar coordinates and filter it to obtain the ship's main plane point cloud.

[0030] Point cloud data can be acquired using 3D LiDAR. A 3D LiDAR can be deployed on the side of the object being approached, ensuring that the vertical field of view covers the area from the ship's waterline to the deck. The point cloud data acquired by the 3D LiDAR is then converted to polar coordinates and filtered using a set polar coordinate threshold. For example, invalid points (such as wave reflection points) with elevation angles below a water surface height threshold (e.g., -5° to +15°) are removed, and the remaining point cloud is used as the ship's main plane point cloud.

[0031] S220. Perform point cloud density filtering and point cloud continuous frame filtering on the main plane point cloud of the ship to obtain the target plane point cloud of the ship.

[0032] During the berthing process of a ship, various factors such as waves and sunlight may be detected by lidar, thus forming interference point clouds. Interference point clouds are usually not fixed for a long time. For example, as the ship moves, the positions of interference such as waves and sunlight will also change. Therefore, interference point clouds can be filtered by point cloud density and continuous point cloud frames to obtain the planar point cloud of the ship target.

[0033] S230. Calculate the local curvature of the target plane point cloud of the ship, determine the boundary points based on the local curvature, and determine the candidate region point cloud of the ship based on the boundary points and the target plane point cloud of the ship.

[0034] In order to determine the core area of ​​a ship, such as the ship's hull, from the ship's target planar point cloud and eliminate boundary interference such as deck edges, railings, and crane booms, it is necessary to determine the ship's boundary points from the ship's target planar point cloud, and then determine the candidate ship region point cloud corresponding to the ship's hull through the boundary points.

[0035] S240. Calculate the covariance matrix of the point cloud of the candidate ship region, determine the ship's main direction and ship parameters based on the covariance matrix, and monitor the ship's berthing based on the ship's main direction and ship parameters.

[0036] After determining the point cloud of the candidate region corresponding to the main body of the ship, it is also necessary to extract various parameters of the ship, such as the ship's orientation, bow extrapolation state, speed, etc. Therefore, by calculating the covariance matrix of the point cloud of the candidate region of the ship, the main orientation and parameters of the ship can be determined. Based on the main orientation and parameters of the ship, real-time monitoring and precise guidance of the ship's berthing can be achieved.

[0037] This embodiment uses polar coordinate filtering, point cloud density filtering, and continuous frame filtering to accurately extract the ship's principal plane from complex point cloud data, ensuring the continuity of the ship's principal plane. The ship's principal direction is determined by the covariance matrix, and then the ship's berthing is monitored in all aspects using the ship's principal direction and ship parameters, thereby improving the accuracy and reliability of the ship berthing monitoring results.

[0038] In one embodiment, the step of converting the point cloud data to polar coordinates and filtering it to obtain the ship's main plane point cloud includes: The initial planar point cloud is obtained by filtering the polar coordinates of the point cloud according to a preset polar coordinate threshold. A point cloud plane is generated based on the initial planar point cloud, and the ship's main planar point cloud is determined based on the residual threshold of the point cloud plane.

[0039] First, the point cloud data obtained from the 3D coordinate points of the LiDAR scan is converted into polar coordinates, that is, the point cloud position is described by azimuth, elevation, and distance. Then, a polar coordinate threshold is set according to information such as the water surface reflection angle and the side of the ship. For example, the angle of the water surface reflection point is usually concentrated in the lower elevation angle, such as around -5°, while the side of the ship is usually in the medium to high elevation angle range, such as above +15°. Therefore, only the point cloud between -5° and +15° can be retained as the initial planar point cloud to remove most of the interfering point cloud.

[0040] Then, the ship's main plane point cloud, i.e. the plane corresponding to the ship's side, is determined based on the initial plane point cloud. Since the ship's side is the most stable part of the ship, it can provide a reliable spatial reference for subsequent ship berthing monitoring.

[0041] In one embodiment, the step of generating a point cloud plane based on the initial planar point cloud and determining the ship's main planar point cloud based on a residual threshold of the point cloud plane includes: Based on a randomly sampled point cloud plane in the initial planar point cloud, the standard deviation from the initial point cloud to the sampled point cloud plane is calculated using an iterative algorithm. The error fluctuation corresponding to the sampling point cloud plane is generated based on the standard deviation. If the error fluctuation is greater than the residual threshold, then the residual threshold is adjusted; Repeat the above steps until the target residual threshold is obtained, and determine the ship's main plane point cloud based on the target residual threshold.

[0042] A point cloud plane is randomly sampled from the initial planar point cloud. The equation of the point cloud plane can be obtained using common plane calculation equations. We use this to represent the fluctuation error, and then use the improved RANSAC (Random Sample Consensus) algorithm to calculate the fluctuation error.

[0043] The traditional RANSAC algorithm sets a fixed threshold, then uses points whose distance from the line (i.e., the standard deviation) is less than the threshold as interior points to vote for the line. Finally, the line with the most votes is used as the final model, thus obtaining the final optimized parameters.

[0044] This embodiment uses an improved RANSAC algorithm. Instead of setting a fixed target residual threshold, it adjusts the residual threshold based on the error fluctuations corresponding to the sampled point cloud plane. If the error fluctuations after each iteration of fitting are inconsistent, it indicates that the current residual threshold is too large or too small, requiring adjustment until all error fluctuations converge. This final residual threshold is then used as the target residual threshold. For example, the initial residual threshold can be set to 0.05m. After obtaining the target residual threshold, all points whose distance to the sampled point cloud plane is less than the target residual threshold are used as the ship's main plane point cloud.

[0045] Please refer to Figure 3 , Figure 3 This is a flowchart illustrating a method for acquiring planar point clouds of ship targets provided in this embodiment.

[0046] In one implementation, step S220 includes: step S2210-step S2230.

[0047] S2210. Divide the ship's main plane point cloud into multiple point cloud regions and calculate the point cloud density corresponding to each point cloud region.

[0048] S2220. Point cloud regions with point cloud density lower than a preset density are identified as interference regions and removed, and point cloud regions with point cloud density higher than the preset density are identified as target regions.

[0049] S2230. Obtain the continuous frames corresponding to the target area, identify the point cloud that appears less than a preset number of times in the continuous frames as interference points and remove them to obtain the ship target planar point cloud.

[0050] After dividing the point cloud of the ship's main plane into multiple point cloud regions, the point cloud density of each region is calculated. Then point cloud density With preset density Perform a comparison, if If the value is calculated as points / m², then the point cloud region is marked as an interference region and removed, thus obtaining the target region.

[0051] Then, a temporal consistency check is performed on the target area by establishing a sliding time window, such as 5 frames, and then removing transient interference points that appear consecutively for less than 3 frames, such as wave reflection points, thereby obtaining the final ship target planar point cloud.

[0052] This embodiment filters the point cloud multiple times to ensure that the plane used for subsequent analysis has temporal and spatial continuity, thereby improving the accuracy of the analysis results.

[0053] In one implementation, the step of calculating the local curvature of the ship target planar point cloud and determining boundary points based on the local curvature includes: Calculate the mean of the point cloud normal vector and the neighborhood normal vector of the ship target planar point cloud; The neighborhood curvature of each point cloud is determined based on the mean of the point cloud normal vector and the neighborhood normal vector. Points in the domain with curvature greater than a curvature threshold are identified as boundary points.

[0054] For each point cloud in the planar point cloud of a ship target, it is surrounded by multiple neighboring point clouds. The normal vectors of its neighboring point clouds are calculated, then the average of the normal vectors of all neighboring point clouds is calculated. Finally, the angle difference between the normal vector of each neighboring point cloud and the average is calculated, and its standard deviation is used as the local curvature of that point cloud. , in For point cloud normal vectors, It is the mean of the neighborhood normal vectors.

[0055] If the local curvature exceeds the curvature threshold, the point cloud is marked as a boundary point, such as the boundary between the deck and the water surface. Then, candidate regions are divided according to the distribution of the boundary points and the target planar point cloud, such as the main body region of the ship, in order to eliminate local interference points.

[0056] In one implementation, the steps of calculating the covariance matrix of the point cloud of the candidate ship region and determining the ship's principal direction and ship parameters based on the covariance matrix include: Calculate the mean coordinates of the point cloud in the candidate ship region point cloud, and construct the covariance matrix based on the mean coordinates; Extract the largest eigenvector of the covariance matrix, determine the ship's main direction based on the largest eigenvector, and determine the ship's parameters based on the ship's main direction.

[0057] Cloud covariance matrix of candidate region points:

[0058] in For point cloud coordinates, These are the mean coordinates.

[0059] Extracting the largest eigenvector of the covariance matrix Determine the ship's main direction and rotation angle. .

[0060] The steps for determining the ship parameters based on the ship's main direction include: Construct the directed bounding box of the ship, calculate the parameters of the directed bounding box according to the ship's main direction, and generate the minimum bounding rectangle according to the parameters of the directed bounding box. The parameters of the directed bounding box include the ship's length, ship's width, and ship's height. The relative positions of the ship and the object being moored are determined based on the mapping relationship between the center coordinates of the minimum bounding rectangle and the coordinate system of the object being moored. The ship speed is calculated based on the continuous frame difference of the center coordinates and the point cloud scanning period; Calculate the mooring angle based on the main direction of the minimum bounding rectangle and the mooring direction of the object being moored; The minimum distance between the smallest bounding rectangle and the edge of the object being approached is taken as the minimum approach distance.

[0061] Construct the initial OBB (directed bounding box): Based on the main direction Calculate the length, width, and height of the bounding box (e.g., ship length, ship width, and ship height). If the ship is in a bow-out position (e.g., rotating angle) Deviation from berthing direction exceeds A quadratic curve fitting was performed on the bow region, and the fitting formula is as follows: Adjust the bounding box boundaries to match the ship's dynamic attitude. Keep the midships parallel to the main plane to avoid bounding box deformation due to obstruction.

[0062] Using the center coordinates of the smallest bounding rectangle The mapping relationship between the ship and the coordinate system of the object being docked determines the relative position of the ship and the object being docked (such as lateral offset and longitudinal spacing). Combine point cloud time series data (such as the center displacement of rectangular boxes in consecutive frames) ) and lidar scanning frequency Calculate the ship's speed: ,in .

[0063] Based on the main direction of the minimum bounding rectangle Mooring direction relative to the object being moored The included angle: .

[0064] Calculate the minimum distance between the smallest bounding rectangle and the edge of the object being approached, and use this distance as the closest point distance between the ship's hull and the edge of the dock.

[0065] When the distance between the vessel and the object it is moored to is ≤3.3m or 3.8m, the distance between the minimum circumscribed rectangle and the berthing ball area is determined by... Combined with the initial diameter of the berth ball Calculate the compression amount: .

[0066] To perform the corresponding steps in the above embodiments and various possible methods, an implementation of a ship berthing monitoring device is given below. Please refer to [link / reference]. Figure 4 , Figure 4 This is a functional block diagram of a ship berthing monitoring device provided in an embodiment of the present invention. It should be noted that the basic principle and technical effects of the ship berthing monitoring device provided in this embodiment are the same as those in the above embodiments. For the sake of brevity, any parts not mentioned in this embodiment can be referred to the corresponding content in the above embodiments. The ship berthing monitoring device 400 includes: Point cloud acquisition module 410 is used to acquire point cloud data corresponding to the ship, convert the point cloud data into point cloud polar coordinates and filter it to obtain the ship's main plane point cloud. The point cloud filtering module 420 is used to perform point cloud density filtering and point cloud continuous frame filtering on the ship's main plane point cloud to obtain the ship's target plane point cloud. The point cloud computing module 430 is used to calculate the local curvature of the ship target plane point cloud, determine the boundary points based on the local curvature, and determine the ship candidate region point cloud based on the boundary points and the ship target plane point cloud. The berthing monitoring module 440 is used to calculate the covariance matrix of the point cloud of the candidate region of the ship, determine the main direction and parameters of the ship based on the covariance matrix, and perform berthing monitoring of the ship based on the main direction and parameters of the ship.

[0067] Optionally, the above modules can be stored in the form of software or firmware. Figure 1 The memory shown is either stored in or embedded in the operating system (OS) of the ship berthing monitoring device, and can be accessed by... Figure 1 The processor executes the commands. Meanwhile, the data and program code required to execute these modules can be stored in memory.

[0068] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can also be implemented in other ways. The apparatus embodiments described above are merely illustrative; for example, the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram and / or flowchart, and combinations of blocks in block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0069] In addition, the functional modules in the various embodiments of the present invention can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0070] If the aforementioned functions are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0071] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for monitoring ship berthing, characterized in that, include: Obtain the point cloud data corresponding to the ship, convert the point cloud data into point cloud polar coordinates and filter it to obtain the ship's main plane point cloud; The point cloud density and continuous frame filtering of the ship's main plane point cloud are performed to obtain the ship's target plane point cloud. Calculate the local curvature of the target ship planar point cloud, determine the boundary points based on the local curvature, and determine the candidate ship region point cloud based on the boundary points and the target ship planar point cloud; Calculate the covariance matrix of the point cloud of the candidate ship region, determine the ship's main direction and ship parameters based on the covariance matrix, and monitor the ship's berthing based on the ship's main direction and ship parameters.

2. The ship berthing monitoring method according to claim 1, characterized in that, The step of converting the point cloud data into polar coordinates and filtering it to obtain the ship's main plane point cloud includes: The initial planar point cloud is obtained by filtering the polar coordinates of the point cloud according to a preset polar coordinate threshold. A point cloud plane is generated based on the initial planar point cloud, and the ship's main planar point cloud is determined based on the residual threshold of the point cloud plane.

3. The ship berthing monitoring method according to claim 2, characterized in that, The step of generating a point cloud plane based on the initial planar point cloud and determining the ship's main planar point cloud based on the residual threshold of the point cloud plane includes: Based on a randomly sampled point cloud plane in the initial planar point cloud, the standard deviation from the initial point cloud to the sampled point cloud plane is calculated using an iterative algorithm. The error fluctuation corresponding to the sampling point cloud plane is generated based on the standard deviation. If the error fluctuation is greater than the residual threshold, then the residual threshold is adjusted; Repeat the above steps until the target residual threshold is obtained, and determine the ship's main plane point cloud based on the target residual threshold.

4. The ship berthing monitoring method according to claim 1, characterized in that, The steps of performing point cloud density filtering and point cloud continuous frame filtering on the ship's main plane point cloud to obtain the ship's target plane point cloud include: The ship's main plane point cloud is divided into multiple point cloud regions, and the point cloud density corresponding to each point cloud region is calculated; Point cloud regions with point cloud density lower than a preset density are identified as interference regions and removed, while point cloud regions with point cloud density higher than the preset density are identified as target regions. Obtain continuous frames corresponding to the target area, identify point clouds that appear less than a preset number of times in the continuous frames as interference points and remove them to obtain the ship target planar point cloud.

5. The ship berthing monitoring method according to claim 1, characterized in that, The steps of calculating the local curvature of the ship target planar point cloud and determining the boundary points based on the local curvature include: Calculate the mean of the point cloud normal vector and the neighborhood normal vector of the ship target planar point cloud; The neighborhood curvature of each point cloud is determined based on the mean of the point cloud normal vector and the neighborhood normal vector. Points in the domain with curvature greater than a curvature threshold are identified as boundary points.

6. The ship berthing monitoring method according to claim 1, characterized in that, The steps of calculating the covariance matrix of the point cloud of the candidate ship region and determining the ship's principal direction and parameters based on the covariance matrix include: Calculate the mean coordinates of the point cloud in the candidate ship region point cloud, and construct the covariance matrix based on the mean coordinates; Extract the largest eigenvector of the covariance matrix, determine the ship's main direction based on the largest eigenvector, and determine the ship's parameters based on the ship's main direction.

7. The ship berthing monitoring method according to claim 1, characterized in that, The steps for determining the ship parameters based on the ship's main direction include: Construct the directed bounding box of the ship, calculate the parameters of the directed bounding box according to the ship's main direction, and generate the minimum bounding rectangle according to the parameters of the directed bounding box. The parameters of the directed bounding box include the ship's length, ship's width, and ship's height. The relative positions of the ship and the object being moored are determined based on the mapping relationship between the center coordinates of the minimum bounding rectangle and the coordinate system of the object being moored. The ship speed is calculated based on the continuous frame difference of the center coordinates and the point cloud scanning period; Calculate the mooring angle based on the main direction of the minimum bounding rectangle and the mooring direction of the object being moored; The minimum distance between the smallest bounding rectangle and the edge of the object being approached is taken as the minimum approach distance.

8. A ship berthing monitoring device, characterized in that, include: The point cloud acquisition module is used to acquire point cloud data corresponding to the ship, convert the point cloud data into point cloud polar coordinates and filter it to obtain the ship's main plane point cloud. The point cloud filtering module is used to perform point cloud density filtering and point cloud continuous frame filtering on the main plane point cloud of the ship to obtain the target plane point cloud of the ship. The point cloud computing module is used to calculate the local curvature of the target plane point cloud of the ship, determine the boundary points based on the local curvature, and determine the candidate region point cloud of the ship based on the boundary points and the target plane point cloud of the ship. The berthing monitoring module is used to calculate the covariance matrix of the point cloud of the candidate region of the ship, determine the main direction and parameters of the ship based on the covariance matrix, and perform berthing monitoring of the ship based on the main direction and parameters of the ship.

9. An electronic device, characterized in that, It includes a processor and a memory, the memory storing machine-executable instructions that can be executed by the processor, the processor executing the machine-executable instructions to implement the ship berthing monitoring method according to any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the ship berthing monitoring method as described in any one of claims 1-7.

Images