Bulk carrier main structure segmentation method based on point cloud, storage medium and computer device

Abstract

The application discloses a bulk cargo ship body structure segmentation method based on a point cloud, a storage medium and a computer device, and the method comprises the following steps: inputting a bulk cargo ship point cloud image to be segmented into a PointNet++ network to obtain a segmented mask image; performing denoising on the cabin point cloud; correcting the bow and stern point clouds based on the relative position relationship, updating the bow point cloud and the stern point cloud; finding out the points belonging to the bow and the stern in the hatch point cloud, and merging the points with the bow point cloud and the stern point cloud respectively; removing the noise points in the ship side point cloud, and dividing the ship side point cloud into a first ship side point cloud and a second ship side point cloud; finding out the points belonging to noise and dust in the material point cloud, deleting the noise points, and merging the dust points in the material point cloud with a dust point cloud; finding out the points belonging to the skirt in the chute point cloud, and merging the points with a skirt point cloud. The PointNet++ is used for ship point cloud segmentation, the relative position relationship between the parts of the ship is considered, the point cloud with segmentation errors is corrected, and the precision of ship segmentation is greatly improved.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of image processing, and more particularly, the present application relates to a bulk cargo ship main structure segmentation method based on point cloud, a storage medium and a computer device. BACKGROUND

[0002] The 3D radar point cloud data is not affected by light and can scan objects in all directions, and can show the spatial position relationship between objects.

[0003] With the continuous development of port unmanned degree, the degree of port automation operation is continuously improved. When bulk materials are loaded or unloaded, a laser radar is usually used to scan the port bulk cargo operation site in real time, and each part of the bulk cargo ship in the operation site needs to be identified to facilitate the port automation operation.

[0004] The point cloud deep learning method is a technical trend in the port automation industry. The research and application of point cloud component segmentation method are in the development and exploration stage, and can handle not complex business scenarios, so it cannot realize accurate segmentation of the bulk cargo ship main structure. SUMMARY

[0005] The present application provides a bulk cargo ship main structure segmentation method based on point cloud, aiming at improving the above problems.

[0006] The present application is realized in this way, a bulk cargo ship main structure segmentation method based on point cloud, the method comprises the following steps:

[0007] (1) input the bulk cargo ship image to be segmented into PointNet++ network to obtain the segmented mask image, that is, complete the class label of each point, and label as ship head point cloud, ship tail point cloud, ship cabin point cloud, material point cloud, dust point cloud, chute point cloud, skirt point cloud, cabin cover point cloud and ship side point cloud;

[0008] (2) denoising the ship cabin point cloud; merging the ship head point cloud and the ship tail point cloud, further correcting the point cloud classes of the ship head and the ship tail based on the relative position relationship, updating the ship head and ship tail point clouds; finding out the points belonging to the ship head and the ship tail in the cabin cover point cloud, and merging them with the ship head point cloud and the ship tail point cloud respectively; removing the noise points in the ship side point cloud, and dividing the ship side point cloud into two first ship side point clouds and second ship side point clouds; finding out the points belonging to noise and dust in the material point cloud, deleting the noise points, merging the dust points in the material point cloud with the dust point cloud, finding out the points belonging to the skirt in the chute point cloud, and merging them with the skirt point cloud, to complete the segmentation of the bulk cargo ship main structure.

[0009] Further, the updating process of the ship head point cloud and the ship tail point cloud is as follows:

[0010] (221) Clustering the bow point cloud Bow_original and the stern point cloud Stern_original, removing noise clusters, to obtain the denoised bow point cloud Bow_pure and the denoised stern point cloud Stern_pure;

[0011] (222) Merging the bow point cloud Bow_pure and the stern point cloud Stern_pure, and obtaining a plurality of clustering clusters [Bow_Stern1, Bow_Stern2, …, Bow_Stern n ] by clustering;

[0012] (223) Reading the i-th clustering cluster Bow_Stern i in the coordinate range [Bow_Stern i_min , Bow_Stern i_max ] on the Y axis;

[0013] (224) Comparing the coordinate range of the clustering cluster Bow_Stern i on the Y axis with the coordinate range [Bow_y_min, Bow_y_max] of the bow on the Y axis and the coordinate range [Stern_y_min, Stern_y_max] of the stern on the Y axis, and marking the clustering cluster Bow_Stern i as the bow or the stern;

[0014] (225) Merging all clustering clusters marked as the bow to form a new bow point cloud, and merging all clustering clusters marked as the stern to form a new stern point cloud.

[0015] Further, the marking method of the clustering cluster Bow_Stern i is as follows:

[0016] If the coordinate range of the clustering cluster Bow_Stern i on the Y axis satisfies formula (1), the clustering cluster Bow_Stern i is marked as the bow cloud, and the clustering cluster Bow_Stern i satisfies formula (2) on the Y axis, and the clustering cluster Bow_Stern i is marked as the stern.

[0017]

[0018]

[0019] Further, the identification method of the points belonging to the bow and the stern in the hatch cover point cloud is as follows:

[0020] (231) Clustering the hatch point cloud Hatch_original, obtaining m clustering clusters [Hatch1, Hatch2,..., Hatch m ];

[0021] (232) Counting the ratio ratio1, ratio2, ratio3 of points belonging to the bow, the stern and the hatch in the i-th clustering cluster Hatch i , if ratio1 is greater than the ratio threshold, it is determined that the clustering cluster Hatch i belongs to the hatch; if ratio2 is greater than the ratio threshold, it is determined that the clustering cluster Hatch i belongs to the bow; if ratio3 is greater than the ratio threshold, it is determined that the clustering cluster Hatch i belongs to the stern.

[0022] Further, the point detection process of the clustering cluster Hatch i belonging to the bow, the stern and the hatch is as follows:

[0023] For any point HT_j (X i , Y j , Z j ) in the clustering cluster Hatch j , if Cabin_y_min-0.3 < Y j < Cabin_y_max+0.3, it is determined that the point HT_j belongs to the cabin, if Bow_y_min-0.3 < Y j < Bow_y_max+0.3, it is determined that the point HT_j belongs to the bow, and if Stern_y_min-0.3 < Y j < Stern_y_max+0.3, it is determined that the point HT_j belongs to the stern.

[0024] Further, the formation process of the first and second ship side point clouds is as follows:

[0025] (241) Clustering the ship side point cloud Shipboard_Original, removing noise clusters in the clustering clusters, and arranging the clustering clusters in descending order of the number of points, obtaining n clustering clusters [Shipboard1, Shipboard2,..., Shipboard n ];

[0026] (242) Obtaining the coordinate range of the Shipboard1 clustering cluster with the largest number of points in the X-axis [Shipboard 1_x_min , Shipboard 1_x_max] and the second most populated cluster Shipboard2 in the X-axis coordinate range [Shipboard 2_x_min Shipboard 2_x_max ];

[0027] (243) Merge the point clouds of clusters Shipboard1 and Shipboard2 to form the shipboard point cloud. 1_2 Obtain the point cloud of the shipboard. 1_2 In the Z-axis coordinate range [Shipboard] 1_2_z_min Shipboard 1_2_z_max Simultaneously calculate the shipboard point cloud. 1_2 The average value of the coordinates on the X and Z axes;

[0028] (244) From the cluster [Shipboard3, Shipboard4, ..., Shipboard] n ]Delete the point cloud Shipboard far from the ship's side 1_2 The remaining clusters are then combined with the shipboard point cloud. 1_2 Merge and update the shipboard point cloud. 1_2 ;

[0029] (245) Add cloud markers to the shipboard. 1_2 Divided into the first ship's side point cloud and the second ship's side point cloud;

[0030] (246) Separate points that do not belong to the corresponding ship side from the first ship side point cloud and the second ship side point cloud;

[0031] (247) Cluster the points in the current first and second hull point clouds, remove noise clusters, and obtain the point clouds of the two hulls of the bulk carrier.

[0032] Furthermore, it does not belong to the shipboard dot cloud pattern. 1_2 The specific method for detecting clusters is as follows:

[0033] From clusters [Shipboard3, Shipboard4, ..., Shipboard] n Extract the k-th cluster Shipboard from [ ] k Obtain the clustered Shipboard k In the coordinate range of the X and Z axes [Shipboard] k_x_min Shipboard k_x_max [Shipboard] k_z_minShipboard k_z_max If the cluster is Shipboard k The cluster Shipboard is defined as satisfying either of the following two conditions. k Not part of the shipboard dot cloud 1_2 Conditions (1) and (2) are as follows:

[0034] Condition (1): If the cluster Shipboard k Meets Shipboard k_x_max >Shipboard x_max_mean +1.5 or Shipboard k_x_min <Shipboard x_min_mean -1.5, then the cluster is Shipboard k Not part of the ship's side cloud pattern;

[0035] Condition (2): If the cluster Shipboard k Meets Shipboard k_z_max >Shipboard 1_2_z_max +0.5 or Shipboard k_z_min <Shipboard 1_2_z_min When the value is -0.5, the cluster is Shipboard. k It does not belong to the ship's side cloud pattern.

[0036] Furthermore, the specific methods for forming the first and second hull side point clouds in step (245) are as follows:

[0037] Shipboard (from current shipboard point cloud) 1_2 Extract the s-th point Sh_s(X) s Y s Z s If Shipboard 1_2_z_min -1.0 < Z s <Shipboard 1_2_z_mean At that time, point Sh_s is the first shipboard point cloud, if Shipboard 1_2_z_mean <Z s <Shipboard 1_2_z_max At +1.0, point Sh_s represents the second hull point cloud.

[0038] Furthermore, the specific methods for detecting points in the first and second hull point clouds that do not belong to either hull are as follows:

[0039] Obtain the coordinate range of the first ship hull point cloud on the X-axis as [S1_x_min, S1_x_max], and the coordinate range of the second ship hull point cloud on the X-axis as [S2_x_min, S2_x_max]. Calculate the average coordinates of the first and second ship hull point clouds on the X-axis as S1_x_mean and S2_x_mean.

[0040] For any point S1 on the first ship's side point cloud i Detection point S1 i Does it satisfy S1? i_x >S1_x_mean-0.1, if the detection result is negative, then point S1 i Not belonging to the first ship's side cloud;

[0041] For any point S2 on the second hull point cloud i Detection point S2 i Does it satisfy S2? i_x >S2_x_mean-0.1, if the detection result is negative, then point S2 i It does not belong to the second ship's side point cloud.

[0042] Furthermore, the specific methods for detecting points in the material point cloud that belong to noise or dust are as follows:

[0043] (251) Remove noise from the material point cloud based on the cabin area;

[0044] (252) Noise in the material point cloud and dust point cloud is removed by clustering to form material clusters and dust clusters;

[0045] (253) Obtain the coordinate range of the point cloud of the cluster with the most points, Matter1, on the Y-axis [Matter 1_y_min Matter 1_y_max ], through the coordinate range [Matter 1_y_min Matter 1_y_max The equidistant partitioning divides the cluster Matter1 into n consecutive subclusters [Matter]. 1_1 Matter 1_2 Matter 1_n ];

[0046] (254) Calculate the values ​​of all subclusters [Matter] 1_1 Matter 1_2 Matter 1_n Extract the i-th sub-cluster Matter from the maximum and minimum values ​​of the X-axis coordinates. 1_i Matter, the maximum value on the X-axis. 1_i_xmax ;

[0047] (255)extract the maximum value Matter 1_1_xmax , Matter 1_2_xmax , …, Matter 1_n_xmax ] of the coordinate in X-axis from all sub-clusters 1_xmax , and keep the sub-cluster whose maximum value of the coordinate in X-axis satisfies |Matter 1_i_xmax -Matter 1_xmax |≤0.5;

[0048] (256)calculate the average value Matter 1_xmean of the maximum value of the coordinate in X-axis of the kept sub-clusters;

[0049] (257)calculate the maximum value of the coordinate in X-axis of each of the remaining material cluster point clouds [Matter2, Matter3, …, Matter m ], the maximum value of the coordinate in X-axis of the i-th material cluster Matter i is Matter i_xmax , and detect whether the material cluster Matter i_xmax belongs to noise, flying dust or material based on the relationship between Matter 1_xmean and Matter i .

[0050] Further, if Matter i_max -Matter 1_xmean >0.3, mark the material cluster Matter i as noise in the material; if Matter i_max -Matter 1_xmean <-0.3, mark the material cluster Matter i as flying dust in the material; if |Matter i_max -Matter 1_xmean |<0.3 and the material cluster Matter i is located in the cabin, mark the material cluster Matter i as material.

[0051] Further, the detection method of the points belonging to the apron in the chute point cloud is as follows:

[0052] (261)cluster the chute point cloud and the apron point cloud respectively, remove noise clusters, and obtain the denoised apron point cloud and the denoised chute point cloud;

[0053] (262)For the i-th point Ch_i in the chute point cloud Chute_pure, the coordinate value Ch_ix of the point Ch_i in the X axis is extracted, and when Crony_x_min≤Ch_ix≤Crony_x_max is satisfied, the point Ch_i is marked as a skirt point and merged with the skirt point cloud.

[0054] Wherein, Crony_x_min, Crony_x_max are the minimum and maximum values of the skirt point cloud Crony_pure in the X axis.

[0055] The application is implemented as a storage medium storing a computer program, which, when executed by a processor, implements the steps of the bulk cargo ship body structure segmentation method based on point cloud.

[0056] The application is implemented as a computer device comprising a processor and a storage medium storing program code, which, when executed by the processor, implements the steps of the bulk cargo ship body structure segmentation method based on point cloud.

[0057] The application first performs ship point cloud segmentation based on the PointNet++ method, and then reclassifies the incorrectly segmented point cloud based on the segmentation result, taking into account the relative position relationship between the parts of the ship, thereby greatly improving the accuracy of ship segmentation. BRIEF DESCRIPTION OF DRAWINGS

[0058] Figure 1 The flowchart of the bulk cargo ship body structure segmentation method based on point cloud provided by the embodiment of the application. DETAILED DESCRIPTION

[0059] The specific embodiments of the application will be further described in detail below with reference to the drawings, and by describing the embodiments, to help those skilled in the art have a more complete, accurate and in-depth understanding of the inventive concept and technical solutions of the application.

[0060] Figure 1 The flowchart of the bulk cargo ship body structure segmentation method based on point cloud provided by the embodiment of the application, which comprises the following steps:

[0061] (1) input the bulk cargo ship image to be segmented into the PointNet++ network, and label the point cloud in the bulk cargo ship image as a part category, which is labeled as a bow point cloud, a stern point cloud, a cabin point cloud, a material point cloud, a dust point cloud, a chute point cloud, a skirt point cloud, a cabin cover point cloud and a ship side point cloud.

[0062] (11) point cloud image acquisition and labeling PointNet++ network;

[0063] The ship cloud data of different time periods is collected, and each ship point cloud is divided into 12 categories for labeling. The point cloud categories are divided into 12 categories such as ship head, ship tail, ship cabin, partition, material, dust, roller, skirt, cabin cover, ship side, wharf and others. The coordinates of each point are represented as (X, Y, Z), and each point has a category id. The ship point cloud dataset is constructed, including 900 ship point clouds of different time periods. The invention mainly focuses on 9 categories such as ship head, ship tail, ship cabin, material, dust, roller, skirt, cabin cover and ship side.

[0064] (12) Point cloud segmentation

[0065] A dataset is constructed using 900 point cloud data to train a 12-category ship point cloud segmentation model. The ship point cloud image is segmented using the segmentation model to generate a mask image. Then, according to the mask image, the point cloud blocks of each part of the ship are generated.

[0066] (2) The relative position relationship between the parts of the ship is integrated, and the incorrectly segmented point clouds in the point clouds labeled as cabin, bow, stern, cabin cover, ship side, material dust, roller and skirt are corrected in sequence, and the main structure segmentation of the bulk cargo ship is completed.

[0067] (21) Find the noise points in the cabin point cloud Cabin_original and remove them;

[0068] The DBSCAN clustering algorithm is used for clustering on the cabin point cloud Cabin_original, and the minimum number of points is set to 20 and the radius is set to 1.0. Through clustering, one or more clusters can be obtained. The cluster that is far away from other cluster centers and has fewer point clouds is regarded as a noise cluster, and the noise cluster is removed from the cabin point cloud Cabin_original to obtain the cabin point cloud Cabin_pure after removing the noise.

[0069] When the bulk cargo ship is empty, the target category is the least and the segmentation result is the most accurate, so the bulk cargo ship image is taken when the bulk cargo ship is empty, and after steps (1) and (21) are performed, a 3D box with the smallest volume is constructed to wrap the cabin. The coordinates of the 6 vertices of the 3D box are (Cabin_x_min, Cabin_x_max, Cabin_y_min, Cabin_y_max, Cabin_z_min, Cabin_z_max). Then, using the bow and stern point cloud, the coordinate range of the bow on the Y axis [Bow_y_min, Bow_y_max] and the coordinate range of the stern on the Y axis [Stern_y_min, Stern_y_max] are obtained. The above ten key parameters are accurately obtained for the bulk cargo ship point cloud category optimization after loading.

[0070] The ship coordinate system in this invention takes the center of the bulk carrier as the origin, the direction perpendicular to the bottom of the hull as the X-axis, the direction pointing towards the water surface as the positive direction, the longitudinal direction of the ship (the direction of the hull extension) as the Y-axis, the direction pointing towards the bow as the positive direction, the transverse direction of the ship as the Z-axis, and the right side of the ship traveling in the positive direction as the positive direction.

[0071] (22) Find the noise points in the bow point cloud (Bow_original) and stern point cloud (Stern_original) respectively, and remove them.

[0072] (221) Cluster the bow point cloud (Bow_original) and stern point cloud (Stern_original) respectively. The clustering parameters for the bow and stern are set in the same way. The minimum number of points is set to 50 and the radius is 1.0. If there are multiple point cloud clusters, the noise clusters are removed to obtain the denoised bow point cloud (Bow_pure) and the denoised stern point cloud (Stern_pure).

[0073] (222) Merge the bow point cloud (Bow_pure) and stern point cloud (Stern_pure) together, and obtain multiple clusters [Bow_Stern1, Bow_Stern2, ..., Bow_Stern] through clustering. n ], Bow_Stern i This represents the i-th cluster;

[0074] (223) Read the cluster Bow_Stern i Within the Y-axis coordinate range [Bow_Stern] i_min Bow_Stern i_max ], where Bow_Stern i_max Bow_Stern i_min They are Bow_Stern i The maximum and minimum values ​​of the Y-axis coordinates (it's best to keep these values);

[0075] (224) Cluster the Bow_Stern i Compare the Y-axis coordinate range with the Y-axis coordinate ranges of the bow and stern, and compare Bow_Stern. i Marked as the bow or stern;

[0076] The range of the bow Y coordinate is [Bow_y_min, Bow_y_max], and the range of the stern Y coordinate is [Stern_y_min, Stern_y_max].

[0077] In this embodiment of the invention, if the cluster Bow_Stern iIf the coordinate range of Y-axis satisfies formula (1), the cluster is labeled as Bow_Stern i The cluster is labeled as Bow_Stern i If the coordinate range of Y-axis satisfies formula (2), the cluster is labeled as Bow_Stern i The cluster is labeled as Stern.

[0078]

[0079]

[0080] (225)All the clusters labeled as Bow are merged to form a new Bow point cloud, and all the clusters labeled as Stern are merged to form a new Stern point cloud.

[0081] (23)Find out the points belonging to Bow and Stern in the Hatch point cloud, and merge them with the existing Bow point cloud and Stern point cloud, respectively;

[0082] (231)Cluster the Hatch_original point cloud, set the minimum number of points to 20 and the radius to 1.0, and obtain m clusters [Hatch1, Hatch2,..., Hatch m ];

[0083] (232)Detect whether the i-th cluster Hatch i belongs to Bow, Stern or Hatch, and the cluster Hatch i is detected as follows:

[0084] For any point j in the cluster Hatch i , denoted as HT_j, its coordinates in the ship coordinate system are (X j , Y j , Z j ), when Cabin_y_min-0.3<Y j <Cabin_y_max+0.3, it is determined that the point HT_j belongs to the cabin, when Bow_y_min-0.3<Y j <Bow_y_max+0.3, it is determined that the point HT_j belongs to the bow, when Stern_y_min-0.3<Y j <Stern_y_max+0.3, it is determined that the point HT_j belongs to the stern, and all the points in the cluster Hatch i are traversed to count the number of points in the cabin, the bow and the stern, respectively, denoted as N i1 , N i2 , N i3 ;

[0085] Calculate the number of points N located in the cabin, the bow and the stern respectively i1 i2 i3 The ratio of the total number of points N of the cluster Hatch i to the total number of points N of the cluster Hatch i is calculated respectively, denoted as ratio1, ratio2, ratio3, and the calculation formula is as follows:

[0086]

[0087] When ratio1 is greater than a set ratio threshold (0.8), it is determined that the cluster Hatch i belongs to the cabin, which is the cabin cover; when ratio2 is greater than a set ratio threshold, it is determined that the cluster Hatch i belongs to the bow; when ratio3 is greater than a set ratio threshold, it is determined that the cluster Hatch i belongs to the stern, and if ratio1, ratio2, ratio3 are all less than the set ratio threshold, then the cluster Hatch i is a noise cluster.

[0088] (233) If the cluster Hatch i belongs to the bow or the stern, then the cluster Hatch i is merged with the bow point cloud or the stern point cloud, and finally all clusters belonging to the cabin cover are merged to form a cabin cover cluster.

[0089] (24) Find noise points in the shipboard point cloud, and after removing the noise points, the shipboard point cloud is divided into two shipboard corresponding first shipboard point cloud and second shipboard point cloud.

[0090] The shipboard is the outermost edge part of the cabin, which is usually strip-shaped and located on both sides of the cabin, divided into port and starboard. When the shipboard is divided, the points on the shipboard and the upper edge of the cabin are easily confused, resulting in the cabin points being incorrectly divided into shipboard points. The relative positional relationship between the cabin and the shipboard can be calculated to further divide them.

[0091] (241) Cluster the shipboard point cloud Shipboard_Original, set the radius to 1.0 and the minimum number of points to 25, remove the noise clusters in the cluster, arrange the clusters in descending order of the number of points to obtain n clusters [Shipboard1, Shipboard2, …, Shipboard n ], extract the cluster Shipboard1 with the largest number of points and the cluster Shipboard2 with the second largest number of points, which contain most of the shipboard point cloud; ​​

[0092] (242) Calculate the coordinate range of clustering cluster Shipboard1 in X-axis [Shipboard 1_x_min , Shipboard 1_x_max ] and the coordinate range of clustering cluster Shipboard2 in X-axis [Shipboard 2_x_min , Shipboard 2_x_max ];

[0093] (243) Merge the point cloud of clustering cluster Shipboard1 and the point cloud of clustering cluster Shipboard2, and form a shipboard point cloud Shipboard 1_2 after merging, obtain the coordinate range of shipboard point cloud Shipboard 1_2 in Z-axis [Shipboard 1_2_z_min , Shipboard 1_2_z_max ], and calculate the coordinate average of shipboard point cloud Shipboard 1_2 in X-axis and Z-axis, the calculation formula is as follows:

[0094] Shipboard x_max_mean = (Shipboard 1_x_max + Shipboard 2_x_max ) / 2;

[0095] Shipboard x_min_mean = (Shipboard 1_x_min + Shipboard 2_x_min ) / 2;

[0096] Shipboard 1_2_z_mean = (Shipboard 1_2_z_min + Shipboard 1_2_z_max ) / 2.

[0097] (244) Find the clustering cluster far away from shipboard point cloud Shipboard 1_2 from the remaining n-2 other clustering clusters [Shipboard3, Shipboard4, …, Shipboard n ], and delete it, merge the remaining clustering clusters with shipboard point cloud Shipboard 1_2 , and update shipboard point cloud Shipboard 1_2 ;

[0098] In the embodiment of the application, the kth clustering cluster Shipboard n is extracted from the remaining n-2 other clustering clusters [Shipboard3, Shipboard4, …, Shipboard n ].k , obtaining a clustering cluster Shipboard k In the coordinate range of the X-axis, the Z-axis [Shipboard k_x_min , Shipboard k_x_max ], [Shipboard k_z_min , Shipboard k_z_max ], if the clustering cluster Shipboard k satisfies any one of the following two conditions, then the clustering cluster Shipboard k does not belong to the ship side point cloud Shipboard 1_2 , condition (1) and condition (2) are as follows:

[0099] Condition (1): if the clustering cluster Shipboard k satisfies Shipboard k_x_max > Shipboard x_max_mean +1.5 or Shipboard k_x_min < Shipboard x_min_mean -1.5, then the clustering cluster Shipboard k does not belong to the ship side point cloud;

[0100] Condition (2): if the clustering cluster Shipboard k satisfies Shipboard k_z_max > Shipboard 1_2_z_max +0.5 or Shipboard k_z_min < Shipboard 1_2_z_min -0.5, then the clustering cluster Shipboard k does not belong to the ship side point cloud.

[0101] (245) The ship side point cloud Shipboard 1_2 is divided into a first ship side point cloud and a second ship side point cloud.

[0102] The s-th point is extracted from the current ship side point cloud Shipboard 1_2 , denoted as Sh_s, and the coordinates of the point in the ship coordinate system are (X s , Y s , Z s ), and it is detected whether the point Sh_s belongs to the first ship side point cloud or the second ship side point cloud, and the ship side point cloud Shipboard 1_2All points in the first ship side point cloud are denoted as S1, and all points in the second ship side point cloud are denoted as S2. The coordinate range of the first ship side point cloud on the X axis is [S1_x_min, S1_x_max], and the coordinate range of the second ship side point cloud on the X axis is [S2_x_min, S2_x_max]. The average value of the coordinates of the first ship side point cloud and the second ship side point cloud on the X axis is calculated, and the formula is as follows:

[0103] S1_x_mean = (S1_x_max + S1_x_min) / 2;

[0104] S2_x_mean = (S2_x_max + S2_x_min) / 2;

[0105] In the embodiment of the present application, the detection process of the point Sh_s belonging to the first ship side point cloud or the second ship side point cloud is as follows:

[0106] Shipboard 1_2_z_min -1.0 < Z s < Shipboard 1_2_z_mean When the point Sh_s is the first ship side point cloud;

[0107] Shipboard 1_2_z_mean < Z s < Shipboard 1_2_z_max +1.0, the point Sh_s is the second ship side point cloud.

[0108] (246) Separate the points not belonging to the corresponding ship side from the first ship side point cloud and the second ship side point cloud;

[0109] For any point S1 i on the first ship side point cloud, it is detected whether the point S1 i satisfies S1 i_x > S1_x_mean - 0.1. If the detection result is yes, the point S1 i belongs to the ship side point, and if the detection result is no, the point S1 i is deleted from the first ship side point cloud;

[0110] For any point S2 i on the second ship side point cloud, it is detected whether the point S2 i satisfies S2 i_x > S2_x_mean - 0.1. If the detection result is yes, the point S2 i belongs to the ship side point, and if the detection result is no, the point S2 i is deleted from the second ship side point cloud.

[0111] (247) Clustering the points in the current first ship side point cloud and the second ship side point cloud, setting the minimum point number as 10 and the clustering radius as 0.3, removing noise clusters, and obtaining the point clouds of the two ship sides of the bulk cargo ship.

[0112] (25) Finding points belonging to noise and dust in the material point cloud, removing the noise points, and merging the dust points in the material point cloud with the dust point cloud.

[0113] During the loading process, dust may appear due to too fast unloading speed. In order to avoid environmental pollution, large dust should be avoided during the loading process, so dust is also the focus. During the loading process, when dust is found, the moving range of the chute should be avoided to be too large.

[0114] (251) Purifying the material point cloud, extracting the cth point in the material point cloud, and marking the coordinates as (X c ,Y c ,Z c ), if the point satisfies the following conditions, the point belongs to the material point cloud;

[0115]

[0116] (252) Clustering the material point cloud and the classified dust point cloud respectively, setting the clustering radius as 0.5 and the minimum point number as 25, obtaining the material clustering cluster and the dust clustering cluster, removing the noise clusters in the material clustering cluster and the dust clustering cluster, arranging the material clustering clusters in order from more to less in point number, forming the material clustering cluster [Matter1, Matter2,..., Matter m ], the clustering cluster Matter1 with the largest point number, and the dust clustering cluster [Dust1, Dust2,..., Dust n ];

[0117] (253) Obtaining the coordinate range [Matter 1_y_min ,Matter 1_y_max ] of the clustering cluster Matter1 point cloud on the Y axis, dividing the coordinate range [Matter 1_y_min ,Matter 1_y_max ] into 20 continuous coordinate intervals, forming a sub-cluster in each coordinate interval, and dividing the clustering cluster Matter1 into 20 continuous sub-clusters [Matter 1_1 , Matter 1_2 ,..., Matter 1_20 ];

[0118] In the embodiment of the application, the coordinate range division of the clustering cluster Matter1 on the Y axis is specifically as follows:

[0119] inter = (Matter 1_y_max 1_y_min ) / 20;

[0120] The i-th sub-cluster Matter 1_i The coordinate value range of Y-axis is:

[0121] [Matter 1_y_min +inter×(i-1), Matter 1_y_min +inter×i];

[0122] Wherein, Matter 1_y_max represents the maximum value of the coordinate of the clustering cluster Matter1 point cloud on the Y-axis, Matter 1_y_min represents the minimum value of the coordinate of the clustering cluster Matter1 point cloud on the Y-axis, n is the interval number, which is set to 20 here, and inter represents the difference between the maximum value and the minimum value of the coordinate of the sub-cluster on the Y-axis. The calculation result is kept to one digit.

[0123] (254) Calculate the maximum and minimum values of the coordinates of all sub-clusters [Matter 1_1 , Matter 1_2 ,..., Matter 1_20 ] on the X-axis respectively; extract the maximum value Matter 1_i of the coordinate of the i-th sub-cluster Matter 1_i_xmax on the X-axis.

[0124] (255) Extract the maximum values [Matter 1_1_xmax , Matter 1_2_xmax ,..., Matter 1_20_xmax ] of the coordinates of all sub-clusters on the X-axis, extract the maximum value Matter 1_xmax of the maximum values of the coordinates of the sub-clusters on the X-axis, and keep the maximum values of the coordinates of the sub-clusters on the X-axis that satisfy |Matter 1_i_xmax -Matter 1_xmax |≤0.5.

[0125] (256) Calculate the average value Matter 1_xmean of the maximum values of the coordinates of the remaining sub-clusters on the X-axis.

[0126] (257) Calculate the maximum values of the coordinates of the remaining material cluster point clouds [Matter2, Matter3,..., Matter m ] on the X-axis respectively, and the maximum value of the coordinate of the i-th material cluster Matter i on the X-axis is recorded as Matter i_xmax , and it is judged whether it is equal to Matter​1_xmean The relationship between the two is determined by the following conditions:

[0127] If Matter i_max -Matter 1_xmean > 0.3, the material cluster Matter i is marked as noise in the material;

[0128] If Matter i_max -Matter 1_xmean <-0.3, the material cluster Matter i is marked as dust in the material;

[0129] If |Matter i_max -Matter 1_xmean | < 0.3, and the material cluster Matter i is located in the ship cabin, the material cluster Matter i is marked as material.

[0130] (258) Remove noise points in the material, and merge the dust in the material with the current dust point cloud.

[0131] (26) Find points belonging to the skirt in the chute point cloud, and merge them with the skirt point cloud.

[0132] (261) The skirt point cloud is recorded as Crony_original, and the chute point cloud is recorded as Chute_original. The chute and skirt point clouds are clustered respectively, with a minimum point number of 50 and a radius of 1.0. Noise clusters are removed to obtain the denoised skirt point cloud Crony_pure and the denoised chute point cloud Chute_pure. The coordinate range [Crony_x_min, Crony_x_max] of the skirt point cloud Crony_pure in the X axis is obtained.

[0133] (262) In the chute and skirt point cloud segmentation, the chute target object is usually large, and the skirt points are usually not marked as the chute. Sometimes, the chute points are marked as the skirt. For the i-th point Ch_i in the chute point cloud Chute_pure, the coordinate value Ch_ix of the point Ch_i in the X axis is extracted. When Crony_x_min<=Ch_ix<=Crony_x_max is satisfied, the point Ch_i is marked as a skirt point and merged with the skirt point cloud.

[0134] The present application first performs ship point cloud segmentation based on the PointNet++ method, and then reclassifies the incorrectly segmented point cloud based on the above segmentation results, greatly improving the accuracy of ship segmentation.

[0135] The application also provides a storage medium storing a computer program, which, when executed by a processor, implements the steps of the bulk carrier hull structure segmentation method based on point clouds as above.

[0136] Embodiments of the application provide a computer device, comprising a processor and a storage medium storing program code, which, when executed by the processor, implements the steps of the bulk carrier hull structure segmentation method based on point clouds as above.

[0137] The above-mentioned method, as long as various non-essential improvements are made by using the method concept and technical solutions of the application, or the concept and technical solutions of the application are directly applied to other occasions without improvement, are within the protection scope of the application.

Claims

1. A point cloud based bulk carrier hull structure segmentation method, characterized in that, The method comprises the following steps: (1) input the bulk carrier image to be segmented into a PointNet++ network to obtain a segmented mask image, i.e. complete the class label of each point, which is labeled as a bow point cloud, a stern point cloud, a cabin point cloud, a material point cloud, a dust point cloud, a chute point cloud, a skirt point cloud, a hatch point cloud and a side point cloud; (2) denoise the cabin point cloud; merge the bow point cloud and the stern point cloud, further correct the point cloud classes of the bow and the stern based on the relative position relationship, update the bow point cloud and the stern point cloud; find out the points belonging to the bow and the stern in the hatch point cloud, and merge them with the bow point cloud and the stern point cloud respectively; remove the noise points in the side point cloud, and divide the side point cloud into a first side point cloud and a second side point cloud corresponding to two sides; find out the points belonging to noise and dust in the material point cloud, remove the noise points, and merge the dust points in the material point cloud with the dust point cloud; find out the points belonging to the skirt in the chute point cloud, and merge them with the skirt point cloud to complete the segmentation of the main structure of the bulk carrier; The updating process of the bow point cloud and the stern point cloud is as follows: (221) cluster the bow point cloud Bow_original and the stern point cloud Stern_original, remove noise clusters, and obtain the denoised bow point cloud Bow_pure and the denoised stern point cloud Stern_pure; (222)Merge the bow point cloud Bow_pure and the stern point cloud Stern_pure, and obtain a plurality of clustering clusters [Bow_Stern1, Bow_Stern2, …, Bow_Stern n ] by clustering. (223) read the i-th cluster Bow_Stern i In the coordinate range [Bow_Stern i_min ,Bow_Stern i_max ] of Y axis; (224)Cluster Bow_Stern i Comparing the coordinate range of Y axis with the coordinate range of Y axis of the bow [Bow_y_min, Bow_y_max] and the coordinate range of Y axis of the stern [Stern_y_min, Stern_y_max], cluster Bow_Stern i Marked as bow or stern; (225) merge all the clusters marked as the bow to form a new bow point cloud, and merge all the clusters marked as the stern to form a new stern point cloud; Wherein, the ship coordinate system takes the center of the bulk carrier as the origin, takes the direction perpendicular to the bottom of the cabin as the X-axis direction, takes the aspect pointing to the water surface as the positive direction, takes the longitudinal direction of the ship body as the Y-axis direction, takes the direction pointing to the bow as the positive direction, and takes the lateral direction of the ship body as the Z-axis direction, and takes the right side of the positive driving direction as the positive direction.

2. The bulk carrier hull structure segmentation method based on point cloud of claim 1, wherein, Cluster Bow_Stern i The marking method is as follows: If the cluster Bow_Stern i If the coordinate range of Y-axis satisfies formula (1), the cluster Bow_Stern i Marked as bow cloud, the cluster Bow_Stern i If the coordinate range of Y-axis satisfies formula (2), the cluster Bow_Stern i Marked as stern.

3. The bulk carrier hull structure segmentation method based on point cloud of claim 1, wherein, The identification method of the points belonging to the bow and the stern in the hatch point cloud is as follows: (231)The ship hatch cover point cloud Hatch_original is clustered to obtain m clustering clusters [Hatch1, Hatch2,..., Hatch m ] (232) the i-th cluster Hatch i the ratio ratio1, ratio2, ratio3 of the points belonging to the bow, the stern and the hatch cover, if ratio1 is greater than a ratio threshold value, the cluster Hatch is identified i belongs to the hatch cover; if ratio2 is greater than a ratio threshold value, the cluster Hatch is identified i belongs to the bow; if ratio3 is greater than a ratio threshold value, the cluster Hatch is identified i belongs to the stern.

4. The bulk carrier hull structure segmentation method based on point cloud of claim 3, wherein, Cluster hatch i The point detection process for the bow, stern and hatch is as follows: For any point HT_j(X i , Y j , Z j ) in the cluster Hatch j , if Cabin_y_min - 0.3 < Y j < Cabin_y_max + 0.3, then the point HT_j is identified as belonging to the cabin, if Bow_y_min - 0.3 < Y j < Bow_y_max + 0.3, then the point HT_j is identified as belonging to the bow, and if Stern_y_min - 0.3 < Y j < Stern_y_max + 0.3, then the point HT_j is identified as belonging to the stern.

5. The bulk carrier hull structure segmentation method based on point cloud of claim 1, wherein, The formation process of the first side point cloud and the second side point cloud is as follows: (241) Clustering the shipboard point cloud Shipboard_Original, removing the noise cluster in the clustering cluster, arranging the clustering cluster according to the point number from more to less, and obtaining n clustering clusters [Shipboard1, Shipboard2, …, Shipboard n ] (242) Get the coordinate range of the X-axis of the cluster Shipboardl with the most points [Shipboard 1_x_min , Shipboard 1_x_max ] and the coordinate range of the X-axis of the cluster Shipboard2 with the second most points [Shipboard 2_x_min , Shipboard 2_x_max ]; (243) Merge the point clouds of clusters Shipboard1 and Shipboard2 to form the shipboard point cloud. 1_2 Obtain the point cloud of the shipboard. 1_2 In the Z-axis coordinate range [Shipboard] 1_2_z_min Shipboard 1_2_z_max Simultaneously calculate the shipboard point cloud. 1_2 The average value of the coordinates on the X and Z axes; (244) From the cluster [Shipboard3, Shipboard4, ..., Shipboard] n ]Delete the point cloud Shipboard far from the ship's side 1_2 The remaining clusters are then combined with the shipboard point cloud. 1_2 Merge and update the Shipboard point cloud. 1_2 ; (245)The shipboard point cloud Shipboard′ 1_2 is divided into a first shipboard point cloud and a second shipboard point cloud; (246) separate the points not belonging to the corresponding side from the first side point cloud and the second side point cloud; (247) cluster the points in the current first side point cloud and the second side point cloud, remove noise clusters, and obtain the point clouds of the two sides of the bulk carrier.

6. The bulk carrier hull structure segmentation method based on point cloud of claim 5, wherein, Does not belong to the shipboard point cloud Shipboard 1_2 The detection method of the clustering cluster is as follows: extract the kth clustering cluster Shipboard from the clustering cluster [Shipboard3, Shipboard4, …, Shipboard n ] k , get the clustering cluster Shipboard k In the coordinate range [Shipboard k_x_min , Shipboard k_x_max ], [Shipboard k_z_min , Shipboard k_z_max ] of the X-axis and the Z-axis, if the clustering cluster Shipboard k satisfies any one of the following two conditions, the clustering cluster Shipboard k does not belong to the ship side point cloud Shipboard 1_2 , the condition (1) and the condition (2) are specifically as follows: Condition (1): If the cluster Shipboard k satisfies Shipboard k_x_max > Shipboard x_max_mean +1.5 or Shipboard k_x_min < Shipboard x_min_mean -1.5, then the cluster Shipboard k does not belong to the shipboard point cloud; Condition (2): If the cluster Shipboard k satisfies Shipboard k_z_max > Shipboard 1_2_z_max + 0.5 or Shipboard k_z_min < Shipboard 1_2_z_min - 0.5, then the cluster Shipboard k does not belong to the shipboard point cloud.

7. The bulk carrier hull structure segmentation method based on point cloud of claim 5, wherein, The formation method of the first side point cloud and the second side point cloud in step (245) is as follows: From the current shipboard point cloud Shipboard 1_2 extract the s-th point Sh_s(X s , Y s , Z s ), if Shipboard 1_2_z_min -1.0 < Z s < Shipboard 1_2_z_mean , the point Sh_s is the first shipboard point cloud, if Shipboard 1_2_z_mean < Z s < Shipboard 1_2_z_max +1.0, the point Sh_s is the second shipboard point cloud.

8. The bulk carrier hull structure segmentation method based on point cloud of claim 5, wherein, The detection method of the points not belonging to the two sides in the first side point cloud and the second side point cloud is as follows: Obtain the coordinate range of the first side point cloud on the X-axis [S1_x_min, S1_x_max], the coordinate range of the second side point cloud on the X-axis [S2_x_min, S2_x_max], and the average value of the first side point cloud and the second side point cloud on the X-axis S1_x_mean, S2_x_mean; For any point S1 on the first ship side point cloud i , it is detected whether S1 i satisfies S1 i_x > S1_x_mean - 0.1, if the detection result is no, then the point S1 i does not belong to the first ship side point cloud; For any point S2 on the second ship side point cloud i , it is detected whether S2 i satisfies S2 i_x > S2_x_mean - 0.1, if the detection result is no, then the point S2 i does not belong to the second ship side point cloud.

9. The bulk carrier hull structure segmentation method based on point cloud of claim 1, wherein, The detection method of the points belonging to noise and dust in the material point cloud is as follows: (251) remove noise in the material point cloud according to the cabin range; (252) remove noise in the material point cloud and the dust point cloud by clustering to form material clusters and dust clusters; (253)Obtain the coordinate range of the cluster Matter1 point cloud with the most points in the Y-axis [Matter 1_y_min ,Matter 1_y_max ], divide the cluster Matter1 into n continuous sub-clusters [Matter 1_y_min ,Matter 1_y_max ] by equidistant division of the coordinate range [Matter 1_1 ,Matter 1_2 ,...,Matter 1_n ]; (254)Calculate the maximum and minimum of the coordinate of all sub-clusters [Matter 1_1 , Matter 1_2 ,..., Matter 1_n ] on the X-axis, respectively, and extract the maximum of the coordinate of the i-th sub-cluster Matter 1_i on the X-axis Matter 1_i_xmax ; (255) From the maximum coordinate of all subclusters on the X-axis [Matter] 1_1_xmax Matter 1_2_xmax Matter 1_n_xmax Extract the maximum value Matter from ] 1_xmax , retain the satisfaction | Matter 1_i_xmax -Matter 1_xmax The maximum X-axis coordinate of subclusters with a value ≤ 0.5; (256) The average of the maximum values of the coordinates of the remaining sub-clusters in the X axis, Matter, is sought 1_xmean ; (257)Residual material cluster point clouds [Matter2, Matter3,..., Matter m ] are obtained respectively by subtracting the cluster point cloud of the first material Matter1 from the cluster point cloud of the second material Matter2, the cluster point cloud of the second material Matter2 from the cluster point cloud of the third material Matter3, and so on. i The maximum coordinate value of the i-th material cluster Matteri on the X-axis is obtained. i_xmax The maximum coordinate value of the i-th material cluster Matteri on the X-axis is obtained. i_xmax The relationship between Matteri and Matteri+1 is detected. 1_xmean The relationship between Matteri and Matteri+1 is detected. i It is detected whether the i-th material cluster Matteri belongs to noise, dust or material.

10. The bulk carrier hull structure segmentation method based on point cloud of claim 9, wherein, If Matter i_max - Matter 1_xmean > 0.3, then the cluster of material Matter i is flagged as noise in the material; if Matter i_max - Matter 1_xmean < -0.3, then the cluster of material Matter i is flagged as dust in the material; if | Matter i_max - Matter 1_xmean | < 0.3 and the cluster of material Matter i is located in the hold, then the cluster of material Matter i is flagged as material.

11. The bulk carrier hull structure segmentation method based on point cloud of claim 1, wherein, The detection method of the points belonging to the skirt in the chute point cloud is specifically as follows: (261) respectively clustering the chute point cloud and the skirt point cloud, removing the noise cluster, obtaining the denoised skirt point cloud and the denoised chute point cloud, and calculating the X-axis range [Crony_x_min, Crony_x_max] of the skirt point cloud; (262) for the i-th point Ch_i in the chute point cloud Chute_pure, extracting the coordinate value Ch_ix of the point Ch_i in the X-axis, when satisfying Crony_x_min≤Ch_ix≤Crony_x_max, the point Ch_i is marked as a skirt point, and is merged with the skirt point cloud.

12. A storage medium storing a computer program, characterized in that, The computer program is executed by the processor to realize the steps of the bulk cargo ship body structure segmentation method based on the point cloud in any one of claims 1 to 11.

13. A computer device comprising a processor and a storage medium storing program code, the program code being executed by the processor to realize the steps of the bulk cargo ship body structure segmentation method based on the point cloud in any one of claims 1 to 11.

Images