A hull special-shaped part positioning method and system based on disorder stacking

Abstract

The application discloses a hull special-shaped part positioning method and system based on disordered stacking, which comprises the following steps: obtaining a scene point cloud containing a plurality of hull special-shaped parts, removing background point clouds from the scene point cloud, obtaining part point clouds corresponding to the plurality of hull special-shaped parts, calculating the curvature and the normal vector angle corresponding to each point cloud data point in the part point cloud, then obtaining a top layer part point cloud corresponding to a top layer part from the part point cloud according to the curvature and the normal vector angle, extracting the outer contour of the top layer part point cloud, generating a circumscribed rectangle corresponding to the top layer part point cloud along the outer contour through a preset bounding box method, obtaining the barycentric three-dimensional coordinates of the circumscribed rectangle through a preset barycentric calculation formula, and converting the barycentric three-dimensional coordinates through a three-dimensional calibration method to obtain the positioning coordinates of the top layer part, thereby improving the accuracy and efficiency of part positioning.

Description

Technical Field

[0001] This invention relates to the field of shipbuilding technology, specifically to a method and system for positioning irregularly shaped hull parts based on disordered stacking. Background Technology

[0002] Hull parts are an essential component of shipbuilding. On average, each ship requires approximately 46,000 hull parts during its construction. These hull parts come in a wide variety of shapes and sizes, with the average length ranging from 150mm to 1500mm and the width from 100mm to 700mm. Because different types of hull parts often need to be assembled during shipbuilding, current shipbuilding processes often involve randomly stacking various shapes and sizes of irregularly shaped hull parts in pallets and selecting the necessary hull parts from the ship according to the shipbuilding requirements.

[0003] When sorting parts, manual visual identification or model matching methods are often used. However, manual visual identification relies on the experience and focus of workers, which is not only inefficient but also prone to errors that could lead to low accuracy. The model matching method is used when parts of uniform shape and size are stacked haphazardly in pallets. After the parts are cut, they need to be sorted into pallets according to their shape characteristics, meaning that parts of the same shape are placed in the same pallet. This greatly increases the workload of workers and multiplies the number of pallets, thus increasing the space occupied. In addition, to ensure the integrity of the pallets, the parts need to be sorted again according to the designed flow code after grinding, which further increases the workload of workers and results in low efficiency of parts identification. Summary of the Invention

[0004] To address the aforementioned technical problems, this invention discloses a method and system for locating irregularly shaped ship hull parts based on disordered stacking, thereby improving the efficiency and accuracy of locating irregularly shaped ship hull parts.

[0005] To achieve the above objectives, this invention discloses a method for positioning irregularly shaped ship hull parts based on disordered stacking, comprising:

[0006] A point cloud scan is performed on several irregularly shaped ship hull parts in the tray that do not need to be stacked to obtain a scene point cloud containing the several irregularly shaped ship hull parts. The background point cloud is then removed from the scene point cloud according to a preset background subtraction algorithm to obtain the part point cloud corresponding to the several irregularly shaped ship hull parts.

[0007] The curvature and normal vector angle of each point cloud data point in the part point cloud are calculated according to the preset point cloud data point curvature calculation method, and then the top part point cloud corresponding to the top part is obtained from the part point cloud according to the curvature and normal vector angle.

[0008] Extract the outer contour of the top-level part point cloud, and generate the bounding rectangle corresponding to the top-level part point cloud along the outer contour using a preset bounding box method;

[0009] Based on the circumscribed rectangle, the three-dimensional coordinates of the centroid of the circumscribed rectangle are obtained through a preset centroid calculation formula, and the three-dimensional coordinates of the centroid are transformed using a three-dimensional calibration method to obtain the positioning coordinates of the top-level part.

[0010] This invention discloses a method for locating irregularly shaped ship hull parts based on disordered stacking. When multiple irregularly shaped ship hull parts do not need to be stacked on a tray, in order to achieve ordered extraction of individual parts from these multiple non-stackable irregularly shaped ship hull parts, a point cloud scan is first performed on the placed irregularly shaped parts to allow for individual segmentation based on the point cloud data. Since the segmentation object should only contain the point cloud of the irregularly shaped ship hull parts, a background subtraction algorithm is used to clear the background point cloud in the initially obtained scene to reduce the amount of data processing and the influence of background point cloud on point cloud segmentation, thereby improving the accuracy of part positioning.

[0011] Furthermore, after obtaining the point cloud of parts containing only the irregularly shaped hull components, in order to extract the parts in an orderly manner, this invention adopts a top-down extraction order to identify each part sequentially. Therefore, this invention needs to obtain the point cloud corresponding to the top single part among the multiple irregularly shaped hull components. Then, because the curvature and normal vector angles corresponding to point cloud data points at different heights are different, while the curvature and normal vector angles corresponding to point cloud data points at the same height are the same, this invention uses the point cloud data point curvature calculation method to calculate the curvature and normal vector angle of each point cloud data point, thereby realizing the extraction of the top part point cloud and improving the accuracy of the top part positioning.

[0012] After obtaining the point cloud of the top-level part, the bounding box method is used to extract the outer matrix corresponding to the top-level part, which facilitates the calculation of the centroid coordinates of the top-level part and improves the efficiency of part recognition. Then, the centroid coordinates are converted into the positioning coordinates of the top-level part using a three-dimensional calibration method, so as to achieve accurate positioning of the upper-level part and improve the accuracy of part positioning.

[0013] As a preferred example, the point cloud scanning of several irregularly shaped hull parts within the tray, which do not require stacking, to obtain a scene point cloud containing the several irregularly shaped hull parts includes:

[0014] The 3D camera performs point cloud scanning on several irregularly shaped ship hull parts inside the tray without stacking them, and obtains a scene point cloud including the several irregularly shaped ship hull parts; wherein, the 3D camera is located directly above the tray.

[0015] This invention uses a 3D camera to capture point cloud images of irregularly stacked ship hull parts made of mixed materials. This facilitates the positioning of individual parts based on the block segmentation characteristics of the point cloud, improving the accuracy of part segmentation. At the same time, the 3D camera is positioned directly above the tray to avoid the influence of the scanning angle on the segmentation of the top layer parts, thereby improving the accuracy of subsequent positioning of the top layer parts.

[0016] As a preferred example, the step of removing background point clouds from the scene point cloud according to a preset background subtraction algorithm to obtain part point clouds corresponding to the plurality of irregularly shaped ship hull parts includes:

[0017] Search for data points in the scene point cloud with the same coordinates as the background point cloud using a pre-constructed octree;

[0018] The data points are removed from the scene point cloud to obtain the part point cloud corresponding to the plurality of irregularly shaped ship hull parts.

[0019] This invention removes the background point cloud from the scene point cloud to obtain a part point cloud containing only the irregularly shaped parts of the ship hull. This reduces the amount of data processing required later, improves the efficiency of part positioning, and avoids interference from the pallet and floor on part positioning, thereby improving the accuracy of part positioning.

[0020] As a preferred example, the calculation of the curvature and the angle between the normal vectors corresponding to each point cloud data point in the part's point cloud according to the preset point cloud data point curvature calculation method includes:

[0021] Obtain the neighborhood point set corresponding to each point cloud data point, and calculate the centroid corresponding to the neighborhood point set;

[0022] Based on the neighborhood point set and the centroid, calculate the covariance matrix corresponding to each point cloud data point;

[0023] Eigenvalue decomposition is performed on the covariance matrix corresponding to each point cloud data point to obtain the first eigenvalue and the second eigenvalue corresponding to each point cloud data point.

[0024] Based on the first feature value and the second feature value, the curvature corresponding to each point cloud data point is obtained through a preset curvature calculation formula;

[0025] Calculate the feature vector of the smallest feature value among the first feature value and the second feature value, and obtain the normal of each point cloud data point and each neighboring point in its corresponding neighborhood point set;

[0026] The angle of the normal vector corresponding to each point cloud data point is calculated using a preset formula based on the normal vector angle of each neighboring point in the neighborhood set for each point cloud data point.

[0027] This invention improves the positioning accuracy of top-level parts by calculating the curvature and normal vector angles corresponding to data points at the same height.

[0028] As a preferred example, the step of generating the bounding rectangle corresponding to the point cloud of the top-level part along the outer contour using a preset bounding box method includes:

[0029] The OBB bounding box corresponding to the outer contour and the two-dimensional coordinate axis corresponding to the OBB bounding box are generated according to the preset OBB bounding box method.

[0030] The 0BB bounding box is expanded according to the axis direction of the two-dimensional coordinate axis and the preset expansion distance to obtain the outer rectangle corresponding to the point cloud of the top-level part.

[0031] This invention utilizes OBB bounding boxes to obtain the circumscribed rectangle, reducing spatial redundancy and improving memory space utilization. Furthermore, after generating the minimum bounding box, it expands the minimum bounding box outward by a certain distance to resolve errors in bounding box generation and improve the accuracy of part positioning.

[0032] As a preferred example, obtaining the three-dimensional coordinates of the centroid of the circumscribed rectangle using a preset centroid calculation formula includes:

[0033] The three-dimensional coordinates of the centroid of the circumscribed rectangle are calculated according to a preset software program and a preset centroid calculation formula; wherein, the centroid calculation formula is:

[0034]

[0035] Wherein, n represents the number of point cloud data points in the circumscribed rectangle, and x i y i and z i This represents the three-dimensional coordinates of the centroid corresponding to the i-th point cloud data point.

[0036] This invention calculates and determines the centroid coordinates corresponding to the circumscribed rectangle, thereby improving the positioning accuracy of individual parts.

[0037] As a preferred example, the method of transforming the three-dimensional coordinates of the center of gravity using three-dimensional calibration to obtain the positioning coordinates of the top-level part includes:

[0038] The coordinate system of the 3D camera is unified with the coordinate system of the robotic arm used to grasp the top part by hand-eye calibration, and the coordinate system transformation matrix between the 3D camera and the robotic arm is obtained.

[0039] The three-dimensional coordinates of the center of gravity are transformed according to the coordinate system transformation matrix to obtain the positioning coordinates of the top part in the robotic arm.

[0040] This invention utilizes the coordinate transformation matrix of the coordinate system of the gripping device and the camera to transform the three-dimensional coordinates of the centroid of the circumscribed rectangle into the positioning coordinates of the robotic arm, so as to facilitate the subsequent gripping of the top-level part based on the positioning coordinates and improve the positioning accuracy.

[0041] On the other hand, the present invention discloses a positioning system for irregularly shaped ship hull parts based on disordered stacking, the system including a point cloud scanning module, a point cloud trimming module, a rectangle extraction module and a coordinate positioning module;

[0042] The point cloud scanning module is used to perform point cloud scanning on several irregularly shaped ship hull parts in the tray that do not need to be stacked, to obtain a scene point cloud containing the several irregularly shaped ship hull parts, and to remove the background point cloud from the scene point cloud according to a preset background subtraction algorithm, so as to obtain the part point cloud corresponding to the several irregularly shaped ship hull parts.

[0043] The point cloud clipping module is used to calculate the curvature and normal vector angle corresponding to each point cloud data point in the part point cloud according to a preset point cloud data point curvature calculation method, and then obtain the top part point cloud corresponding to the top part from the part point cloud according to the curvature and normal vector angle.

[0044] The rectangle extraction module is used to extract the outer contour of the top-level part point cloud and generate the bounding rectangle corresponding to the top-level part point cloud along the outer contour using a preset bounding box method.

[0045] The coordinate positioning module is used to obtain the three-dimensional coordinates of the centroid of the circumscribed rectangle according to the circumscribed rectangle through a preset centroid calculation formula, and to transform the three-dimensional coordinates of the centroid using a three-dimensional calibration method to obtain the positioning coordinates of the top part.

[0046] This invention discloses a positioning system for irregularly shaped ship hull parts based on disordered stacking. When multiple irregularly shaped ship hull parts do not need to be stacked on a pallet, in order to achieve ordered extraction of individual parts from these multiple non-stackable irregularly shaped ship hull parts, a point cloud scan is first performed on several non-stackable irregularly shaped parts. This allows for individual segmentation based on the point cloud data. Since the segmentation object should only contain the point cloud of the irregularly shaped ship hull parts, a background subtraction algorithm is used to clear the background point cloud in the initially obtained scene to reduce the amount of data processing and the influence of background point cloud on point cloud segmentation, thereby improving the accuracy of part positioning.

[0047] Furthermore, after obtaining the point cloud of parts containing only the irregularly shaped hull components, in order to extract the parts in an orderly manner, this invention adopts a top-down extraction order to identify each part sequentially. Therefore, this invention needs to obtain the point cloud corresponding to the top single part among the multiple irregularly shaped hull components. Then, because the curvature and normal vector angles corresponding to point cloud data points at different heights are different, while the curvature and normal vector angles corresponding to point cloud data points at the same height are the same, this invention uses the point cloud data point curvature calculation method to calculate the curvature and normal vector angle of each point cloud data point, thereby realizing the extraction of the top part point cloud and improving the accuracy of the top part positioning.

[0048] After obtaining the point cloud of the top-level part, the bounding box method is used to extract the outer matrix corresponding to the top-level part, which facilitates the calculation of the centroid coordinates of the top-level part and improves the efficiency of part recognition. Then, the centroid coordinates are converted into the positioning coordinates of the top-level part using a three-dimensional calibration method, so as to achieve accurate positioning of the upper-level part and improve the accuracy of part positioning.

[0049] As a preferred example, the point cloud scanning module includes a scanning unit and a clearing unit;

[0050] The scanning unit is used to perform point cloud scanning on several non-stacked ship hull-shaped parts inside the tray according to a preset 3D camera, to obtain a scene point cloud including the several ship hull-shaped parts; wherein, the 3D camera is located directly above the tray.

[0051] The clearing unit is used to search for data points with the same coordinates as the background point cloud from the scene point cloud using a pre-constructed octree; remove the data points from the scene point cloud to obtain the part point cloud corresponding to the plurality of ship hull irregular parts.

[0052] This invention uses a 3D camera to capture point cloud images of randomly stacked, mixed, irregularly shaped ship hull parts. This facilitates the localization of individual parts based on the block segmentation characteristics of the point cloud, improving the accuracy of part segmentation. Simultaneously, the 3D camera is positioned directly above the tray to avoid the influence of the scanning angle on the segmentation of the top-layer parts, further improving the accuracy of subsequent top-layer part localization. Furthermore, the background point cloud is cleared from the scene point cloud, resulting in a part point cloud containing only the irregularly shaped ship hull parts. This reduces the amount of data processing required later, improves the efficiency of part localization, and avoids interference from the tray and floor, further enhancing the accuracy of part localization.

[0053] As a preferred example, the point cloud cropping module includes a curvature calculation unit and an angle calculation unit;

[0054] The curvature calculation unit is used to obtain the neighborhood point set corresponding to each point cloud data point and calculate the centroid corresponding to the neighborhood point set; calculate the covariance matrix corresponding to each point cloud data point based on the neighborhood point set and the centroid; perform eigenvalue decomposition on the covariance matrix corresponding to each point cloud data point to obtain the first eigenvalue and the second eigenvalue corresponding to each point cloud data point; and obtain the curvature corresponding to each point cloud data point through a preset curvature calculation formula based on the first eigenvalue and the second eigenvalue.

[0055] The included angle calculation unit is used to calculate the feature vector of the smallest feature value among the first feature value and the second feature value, and obtain the normal of each point cloud data point and each neighboring point in its corresponding neighborhood point set; based on the normal of each point cloud data point and each neighboring point in its corresponding neighborhood point set, the included angle of the normal vector corresponding to each point cloud data point is calculated by a preset formula for calculating the included angle of the normal vector.

[0056] This invention improves the positioning accuracy of top-level parts by calculating the curvature and normal vector angles corresponding to data points at the same height. Attached Figure Description

[0057] Figure 1 : A flowchart illustrating a method for locating irregularly shaped ship hull parts based on disordered stacking, provided by an embodiment of the present invention;

[0058] Figure 2 : A schematic diagram of a positioning system for irregularly stacked ship hull parts based on disordered stacking provided by an embodiment of the present invention;

[0059] Figure 3 : A flowchart illustrating a method for positioning irregularly shaped ship hull parts based on disordered stacking, as provided in another embodiment of the present invention;

[0060] Figure 4: A schematic diagram of an octree structure provided in another embodiment of the present invention;

[0061] Figure 5 This is a schematic diagram illustrating background point cloud removal according to another embodiment of the present invention.

[0062] Figure 6 This is a schematic diagram of point cloud extraction for a top-level component, provided in another embodiment of the present invention.

[0063] Figure 7 : A schematic diagram of an OBB bounding box provided in another embodiment of the present invention;

[0064] Figure 8 This is a schematic diagram illustrating the expansion of the OBB bounding box range according to another embodiment of the present invention.

[0065] Figure 9 : A schematic diagram of a robot structure provided in another embodiment of the present invention;

[0066] Figure 10 This is a schematic diagram of a coordinate transformation relationship provided in another embodiment of the present invention;

[0067] Figure 11 This is a schematic diagram of the pose change of a robot, provided as another embodiment of the present invention. Detailed Implementation

[0068] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0069] Example 1

[0070] This embodiment discloses a method for locating irregularly shaped ship hull parts based on disordered stacking. For the specific implementation process of the positioning method, please refer to [reference needed]. Figure 1 It mainly includes steps 101 to 104, the steps being:

[0071] Step 101: Perform point cloud scanning on several irregularly shaped ship hull parts in the tray that do not need to be stacked to obtain a scene point cloud containing the several irregularly shaped ship hull parts, and remove the background point cloud from the scene point cloud according to a preset background subtraction algorithm to obtain the part point cloud corresponding to the several irregularly shaped ship hull parts.

[0072] In this embodiment, the main steps are: using a preset 3D camera to perform point cloud scanning on several non-stacked ship hull-shaped parts inside the tray to obtain a scene point cloud including the several ship hull-shaped parts; wherein, the 3D camera is located directly above the tray.

[0073] Next, a pre-constructed octree is used to search for data points in the scene point cloud that have the same coordinates as the background point cloud; the data points are then removed from the scene point cloud to obtain the part point cloud corresponding to the several irregularly shaped ship parts.

[0074] In this embodiment, this step uses a 3D camera to capture point cloud images of the disorderly stacked, mixed ship hull-shaped parts. This facilitates the location of individual parts based on the block segmentation characteristics of the point cloud, improving the accuracy of part segmentation. Simultaneously, the 3D camera is positioned directly above the tray to avoid the influence of the scanning angle on the segmentation of the top-layer parts, improving the accuracy of subsequent top-layer part location. Next, the background point cloud of the scene is cleared to obtain a part point cloud containing only the ship hull-shaped parts. This reduces the amount of data processing required later, improves the efficiency of part location, and avoids interference from the tray and floor, further enhancing the accuracy of part location.

[0075] Step 102: Calculate the curvature and normal vector angle corresponding to each point cloud data point in the part point cloud according to the preset point cloud data point curvature calculation method, and then obtain the top part point cloud corresponding to the top part from the part point cloud according to the curvature and normal vector angle.

[0076] In this embodiment, the steps are as follows: Obtain the neighborhood point set corresponding to each point cloud data point, and calculate the centroid corresponding to the neighborhood point set; calculate the covariance matrix corresponding to each point cloud data point based on the neighborhood point set and the centroid; perform eigenvalue decomposition on the covariance matrix corresponding to each point cloud data point to obtain the first eigenvalue and the second eigenvalue corresponding to each point cloud data point; obtain the curvature corresponding to each point cloud data point using a preset curvature calculation formula based on the first eigenvalue and the second eigenvalue; calculate the eigenvector of the smallest eigenvalue among the first eigenvalue and the second eigenvalue to obtain the normal vector of each point cloud data point and each neighboring point in its corresponding neighborhood point set; calculate the angle of the normal vector corresponding to each point cloud data point using a preset normal vector angle calculation formula based on the normal vector of each point cloud data point and each neighboring point in its corresponding neighborhood point set.

[0077] In this embodiment, this step is based on the fact that the curvature and normal vector angles corresponding to data points at the same height are the same, and the accuracy of positioning of the top part is improved by calculating the curvature and normal vector angles.

[0078] Step 103: Extract the outer contour of the top-level part point cloud, and generate the bounding rectangle corresponding to the top-level part point cloud along the outer contour using a preset bounding box method.

[0079] In this embodiment, the main steps are: generating the OBB bounding box corresponding to the outer contour and the two-dimensional coordinate axis corresponding to the OBB bounding box according to the preset OBB bounding box method; expanding the OBB bounding box according to the axial direction of the two-dimensional coordinate axis and the preset expansion distance to obtain the outer rectangle corresponding to the point cloud of the top part.

[0080] In this embodiment, this step uses an OBB bounding box to obtain the circumscribed rectangle, reducing spatial redundancy and improving memory space utilization. At the same time, after generating the minimum bounding box, the minimum bounding box is expanded outward by a certain distance to resolve the error in bounding box generation and improve the accuracy of part positioning.

[0081] Step 104: Based on the circumscribed rectangle, obtain the three-dimensional coordinates of the centroid of the circumscribed rectangle using a preset centroid calculation formula, and transform the three-dimensional coordinates of the centroid using a three-dimensional calibration method to obtain the positioning coordinates of the top-level part.

[0082] In this embodiment, the main step is to calculate the three-dimensional coordinates of the centroid of the circumscribed rectangle according to a preset software program and a preset centroid calculation formula; wherein, the centroid calculation formula is:

[0083]

[0084] Wherein, n represents the number of point cloud data points in the circumscribed rectangle, and x i y i and z i This represents the three-dimensional coordinates of the centroid corresponding to the i-th point cloud data point.

[0085] Next, the coordinate system of the 3D camera and the coordinate system of the robotic arm used to grasp the top part are unified by hand-eye calibration to obtain the coordinate system transformation matrix between the 3D camera and the robotic arm.

[0086] The three-dimensional coordinates of the center of gravity are transformed according to the coordinate system transformation matrix to obtain the positioning coordinates of the top part in the robotic arm.

[0087] In this embodiment, this step calculates and determines the centroid coordinates corresponding to the circumscribed rectangle. The centroid coordinates are used to improve the positioning accuracy of individual parts. Then, the coordinate transformation matrix of the coordinate system of the gripping device and the camera is used to transform the three-dimensional coordinates of the centroid of the circumscribed rectangle into the positioning coordinates of the robotic arm. This facilitates the subsequent gripping of the top-level part based on the positioning coordinates, thereby improving the positioning accuracy.

[0088] On the other hand, this embodiment also discloses a positioning system for irregularly shaped ship hull parts based on disordered stacking. For the specific structural composition of the system, please refer to... Figure 2 The system includes a point cloud scanning module 201, a point cloud cropping module 202, a rectangle extraction module 203, and a coordinate positioning module 204.

[0089] The point cloud scanning module 201 is used to perform point cloud scanning on several irregularly shaped ship hull parts in the tray that do not need to be stacked, to obtain a scene point cloud containing the several irregularly shaped ship hull parts, and to remove the background point cloud from the scene point cloud according to a preset background subtraction algorithm, so as to obtain the part point cloud corresponding to the several irregularly shaped ship hull parts.

[0090] The point cloud clipping module 202 is used to calculate the curvature and normal vector angle corresponding to each point cloud data point in the part point cloud according to a preset point cloud data point curvature calculation method, and then obtain the top part point cloud corresponding to the top part from the part point cloud according to the curvature and normal vector angle.

[0091] The rectangle extraction module 203 is used to extract the outer contour of the top-level part point cloud and generate the bounding rectangle corresponding to the top-level part point cloud along the outer contour using a preset bounding box method.

[0092] The coordinate positioning module 204 is used to obtain the three-dimensional coordinates of the centroid of the circumscribed rectangle according to the circumscribed rectangle through a preset centroid calculation formula, and to convert the three-dimensional coordinates of the centroid using a three-dimensional calibration method to obtain the positioning coordinates of the top part.

[0093] In this embodiment, the point cloud scanning module 201 includes a scanning unit and a clearing unit.

[0094] The scanning unit is used to perform point cloud scanning on several non-stacked ship hull-shaped parts inside the tray according to a preset 3D camera, and obtain a scene point cloud including the several ship hull-shaped parts; wherein, the 3D camera is located directly above the tray.

[0095] The clearing unit is used to search for data points with the same coordinates as the background point cloud from the scene point cloud using a pre-constructed octree; remove the data points from the scene point cloud to obtain the part point cloud corresponding to the plurality of ship hull irregular parts.

[0096] In this embodiment, the point cloud cropping module 202 includes a curvature calculation unit and an angle calculation unit.

[0097] The curvature calculation unit is used to obtain the neighborhood point set corresponding to each point cloud data point and calculate the centroid corresponding to the neighborhood point set; calculate the covariance matrix corresponding to each point cloud data point based on the neighborhood point set and the centroid; perform eigenvalue decomposition on the covariance matrix corresponding to each point cloud data point to obtain the first eigenvalue and the second eigenvalue corresponding to each point cloud data point; and obtain the curvature corresponding to each point cloud data point based on the first eigenvalue and the second eigenvalue using a preset curvature calculation formula.

[0098] The included angle calculation unit is used to calculate the feature vector of the smallest feature value among the first feature value and the second feature value, and obtain the normal of each point cloud data point and each neighboring point in its corresponding neighborhood point set; based on the normal of each point cloud data point and each neighboring point in its corresponding neighborhood point set, the included angle of the normal vector corresponding to each point cloud data point is calculated by a preset formula for calculating the included angle of the normal vector.

[0099] This embodiment discloses a method and system for locating irregularly shaped ship hull parts based on disordered stacking. When multiple irregularly shaped ship hull parts do not need to be stacked on a tray, in order to achieve the ordered extraction of individual parts from the multiple irregularly shaped ship hull parts that do not need to be stacked, a point cloud scan is first performed on the irregularly shaped parts that do not need to be stacked, so that individual segmentation can be performed based on the point cloud data. Since the segmentation object should only be the point cloud containing the irregularly shaped ship hull parts when performing part segmentation, in order to reduce the amount of data processing and the influence of background point cloud on point cloud segmentation, a background subtraction algorithm is used to clear the background point cloud of the initially obtained scene point cloud, thereby improving the accuracy of part positioning.

[0100] Furthermore, after obtaining the point cloud of parts containing only the irregularly shaped hull components, in order to extract the parts in an orderly manner, this invention adopts a top-down extraction order to identify each part sequentially. Therefore, this invention needs to obtain the point cloud corresponding to the top single part among the multiple irregularly shaped hull components. Then, because the curvature and normal vector angles corresponding to point cloud data points at different heights are different, while the curvature and normal vector angles corresponding to point cloud data points at the same height are the same, this invention uses the point cloud data point curvature calculation method to calculate the curvature and normal vector angle of each point cloud data point, thereby realizing the extraction of the top part point cloud and improving the accuracy of the top part positioning.

[0101] After obtaining the point cloud of the top-level part, the bounding box method is used to extract the outer matrix corresponding to the top-level part, which facilitates the calculation of the centroid coordinates of the top-level part and improves the efficiency of part recognition. Then, the centroid coordinates are converted into the positioning coordinates of the top-level part using a three-dimensional calibration method, so as to achieve accurate positioning of the upper-level part and improve the accuracy of part positioning.

[0102] Example 2

[0103] This embodiment also discloses a method for positioning irregularly shaped ship hull parts based on disordered stacking. For the specific implementation process of this method, please refer to... Figure 3 The method mainly includes steps 301 to 304, which are mainly as follows:

[0104] Step 301: Use a 3D camera to scan the tray containing several irregularly stacked ship hull parts, obtain the scene point cloud including the several irregularly stacked ship hull parts, and clear the background point cloud of the scene point cloud to obtain the part point cloud corresponding to the several irregularly stacked ship hull parts.

[0105] In this embodiment, the step is as follows: a 3D camera placed directly above the tray is used to take point cloud images of the disorderly stacked ship hull-shaped parts mixed in the tray, so as to obtain a scene point cloud including the tray, the ground and the ship hull-shaped parts.

[0106] After obtaining the scene point cloud, a pre-constructed octree is used to search for data points in the scene point cloud that have the same coordinates as the background point cloud. These data points are then removed from the scene point cloud to obtain the part point clouds corresponding to the various irregularly shaped ship hull parts. Specifically, the structure of the octree can be referred to... Figure 4 The steps for constructing the octree are as follows:

[0107] (1) Set the maximum recursion depth and find the maximum size of the scene, and build the first cube with this size;

[0108] (2) sequentially drop the unit element into the cube that can be contained and has no child nodes. If the maximum recursion depth is not reached, subdivide it into eight equal parts, and then distribute all the unit elements in the cube to the eight child cubes. If it is found that the number of unit elements allocated to the child cube is not zero and is the same as that of the parent cube, then the child cube stops subdividing. This is because, according to the theory of space partitioning, the space obtained by subdivision must be less. If the number is the same, then no matter how many times it is cut, the number will still be the same, which will result in an infinite number of cuts.

[0109] (3) Repeat step 2 until the maximum recursion depth is reached.

[0110] The background point cloud existing in the scene point cloud is obtained based on the pre-constructed octree, and then the background point cloud is removed from the scene point cloud to finally obtain the part point cloud corresponding to the irregular hull part. Specifically, refer to... Figure 5 This is a schematic diagram of background point cloud removal provided in this embodiment.

[0111] Step 302: Calculate the curvature and normal vector angle of each data point in the part point cloud, and perform point cloud clipping on the part point cloud according to the curvature and normal vector angle to obtain the top part point cloud corresponding to the top part among the several irregular hull parts.

[0112] In this embodiment, the main step is as follows: since the several irregularly shaped hull parts are mixed and stacked in a disordered manner, the plane scanned by the 3D vision camera will show a part of multiple lower layer parts, so it is necessary to cut out the outline of the uppermost layer part for grasping.

[0113] In this embodiment, a point cloud data point curvature calculation method is used to calculate the curvature between each point cloud data point, i.e., the angle between the curvature and the normal vector, as a similarity criterion between point clouds. The similarity between each data point is calculated based on the curvature and the angle between the normal vectors. Then, the top-level part point cloud corresponding to the top-level part is cropped from the part point cloud. Specifically, the specific steps of the point cloud data point curvature calculation method are as follows:

[0114] Step 1: For each data point p, select its surrounding neighborhood point set g§.

[0115] Step 2: For the neighborhood point set g§, calculate its centroid C.

[0116] Step 3: Construct the covariance matrix C, where the formula for calculating the covariance matrix is:

[0117]

[0118] Wherein, p_i is a point in the neighborhood point set g§.

[0119] Step 4: Perform eigenvalue decomposition on the covariance matrix C to obtain eigenvalues ​​λ1 and λ2.

[0120] Step 5: Calculate the curvature, the formula for calculating the curvature is:

[0121]

[0122] Step 6: The eigenvector with the smallest eigenvalue, i.e., the normal vector of point q1, is denoted as n. The normal vectors of the g-neighbor points of point q1 are {ni1, ni2, ..., nig}. Calculate the angle between the normal vectors of point q1 and the normal vectors of its r-neighbors:

[0123]

[0124] The above steps can be repeated for each point in the point cloud to calculate the curvature and the angle between the normal vectors of the point cloud.

[0125] Specifically, refer to Figure 6 This is a schematic diagram of extracting the top-level part point cloud from the part point cloud provided in this embodiment.

[0126] Step 303: Extract the outer contour of the point cloud of the top-level part, and extract the bounding rectangle corresponding to the outer contour using the preset OBB bounding box method.

[0127] In this embodiment, the step is as follows: Generate the OBB bounding box corresponding to the outer contour and the corresponding two-dimensional coordinate axes of the OBB bounding box according to a preset OBB bounding box method. Specifically, the key to determining the OBB bounding box is to first determine the principal axis direction of the OBB. By obtaining the x and y components of a series of point coordinates of the graphic object, the covariance matrix is ​​obtained according to the covariance calculation formula, wherein the covariance calculation formula is:

[0128]

[0129] Based on the covariance matrix, the symmetry matrix corresponding to the outer contour is calculated using the symmetry matrix calculation formula, wherein the symmetry matrix calculation formula is:

[0130]

[0131] Where C is a symmetric matrix, its diagonal represents the variance of the x and y components, and the off-diagonal region represents the covariance of the two components. The eigenvalues ​​and eigenvectors are solved by the covariance matrix, and the direction of the eigenvector corresponding to the largest eigenvalue is the principal axis direction of the OBB bounding box. Let the determined direction vector be denoted as P = [P0 P1].

[0132] Next, project the coordinate points onto the direction vector, find the maximum and minimum values ​​of the x and y components in each direction, and denote them as follows: The center of the bounding box is calculated according to a preset center calculation formula, wherein the center calculation formula is:

[0133]

[0134] Then, the half-length of the bounding box is obtained according to the center and the preset half-length calculation formula, wherein the half-length calculation formula is:

[0135]

[0136] The above method yields the smallest OBB bounding box for the graphic object. Specifically, this embodiment provides a schematic diagram of the OBB bounding box corresponding to an irregularly shaped ship hull part. Please refer to the schematic diagram. Figure 7 .

[0137] Then, the OBB bounding box is expanded according to the axial direction of the two-dimensional coordinate axis and the preset expansion distance to obtain the outer rectangle corresponding to the top part point cloud. Since a certain error is allowed between the detected graphic and the detected graphic, this project sets the error parameter to m (unit: m). The generated minimum OBB bounding box is expanded outward along the axial direction by a distance of m, thereby obtaining the bounding box range of the graphic detection. A schematic diagram of the bounding box range can be found in [reference needed]. Figure 8 ,in Figure 8 The dashed line represents the minimum bounding box range generated, while the solid line represents the bounding box range after expanding by a distance of m. The expanded range is the search range for image detection.

[0138] Step 304: Calculate the three-dimensional coordinates of the centroid of the circumscribed rectangle, and use a three-dimensional calibration method to transform the three-dimensional coordinates of the centroid to obtain the positioning coordinates of the top part.

[0139] In this embodiment, the main step is to calculate the three-dimensional coordinates of the centroid of the circumscribed rectangle according to a preset software program and a preset centroid calculation formula; wherein, the centroid calculation formula is:

[0140]

[0141] Wherein, n represents the number of point cloud data points in the circumscribed rectangle, and x i y i and z i This represents the three-dimensional coordinates of the centroid corresponding to the i-th point cloud data point.

[0142] Specifically, in this embodiment, the formula for calculating the center of gravity of the part is:

[0143]

[0144] In the formula: r i =(x i ,y i ,z i ), i = 1, 2, 3…, are the coordinates of each particle; m i Let be the corresponding mass of the particle.

[0145] When designing a workpiece center of gravity extraction algorithm using C++, let m in the above formula i If = 1, then the formula for the centroid extraction algorithm becomes:

[0146]

[0147] The centroid coordinates of the workpiece point cloud can be calculated using a C++ program and the above formula. Once the centroid is extracted, the centroid coordinates are saved.

[0148] Next, the coordinate system of the 3D camera is unified with the coordinate system of the robotic arm used to grasp the top part by hand-eye calibration, and the coordinate system transformation matrix between the 3D camera and the robotic arm is obtained. Then, the three-dimensional coordinates of the center of gravity are transformed according to the coordinate system transformation matrix to obtain the positioning coordinates of the top part in the robotic arm.

[0149] Specifically, in this embodiment, a robot is used to grasp the parts. The specific structural composition of the robot's thermal interface material is described in [reference needed]. Figure 9 ,like Figure 9 As shown, the robot includes a robot base, a magnetic suction tool, and a grasping robot. The three-dimensional coordinates of the center are converted into the coordinates of the robot using a three-dimensional calibration method. Specifically, the coordinate system of the robot's robotic arm is unified with that of the 3D camera through a hand-eye calibration method.

[0150] The above method utilizes the position transformation relationship between the camera and robot coordinate systems to convert the object coordinates in the camera into the target point coordinates of the robot's grinding tool movement, thereby achieving a one-to-one correspondence between the coordinates of the 3D model trajectory points and the coordinates of the robot's magnetic tool. In this embodiment, the hand-eye calibration system involves coordinate transformations of four coordinate systems: the robot's base coordinate system {B}, the magnetic tool coordinate system {E}, the camera coordinate system {C}, and the calibration object coordinate system {O}. Their coordinate transformation relationships are referenced... Figure 10 .

[0151] Reference Figure 10 The coordinate system and matrix transformation relationships are shown in Table 1:

[0152] Table 1. Coordinate Systems and Matrix Transformations

[0153] {B} Robotic arm base coordinate system {B} suction cup coordinate system {B} Camera coordinate system A Pose transformation relationship between suction cup and robotic arm coordinate system B Pose transformation relationship between suction cup and camera coordinate system C Transformation relationship between camera and suction cup coordinate system D Pose transformation relationship between the robotic arm base and the camera coordinate system

[0154] In the eye-out calibration method, if the robot is controlled to move between any two poses within the camera's field of view, the robot's pose change can be referred to... Figure 11 , refer to Figure 11 Then the pose transformation matrix has the following relationship:

[0155] A1·B·C1=A2·B·C2

[0156] After shifting the terms left and right, we get:

[0157]

[0158] The formula can be transformed into the following more general expression:

[0159] aX=Xb

[0160] Where X is the transformation matrix D, a is the product of the pose transformation matrices between the grinding tool and the base during the two pose movements, and b is the product of the pose transformation matrices between the camera and the calibration object during the two pose movements. Using the formula D = A·B·C, the position transformation matrix between the camera coordinate system {C} and the robot base coordinate system {B} can be obtained.

[0161] The purpose of hand-eye calibration is to solve for the transformation matrix D, which can be used to transform between camera coordinates and robotic arm coordinates.

[0162] This embodiment provides a method for positioning irregularly stacked ship hull parts based on disordered stacking. The method identifies and determines the center of gravity of each irregularly stacked ship hull parts mixed in a tray. This technology uses 3D vision to guide the robotic arm, achieving accurate, stable, and fast automatic loading and unloading, thereby improving loading and unloading efficiency and the degree of automation of the process.

[0163] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific embodiments of the present invention and are not intended to limit the scope of protection of the present invention. In particular, it should be noted that any modifications, equivalent substitutions, improvements, etc., made by those skilled in the art within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for positioning irregularly shaped ship hull parts based on disordered stacking, characterized in that, include: A point cloud scan is performed on several irregularly shaped ship hull parts in the tray that do not need to be stacked to obtain a scene point cloud containing the several irregularly shaped ship hull parts. The background point cloud is then removed from the scene point cloud according to a preset background subtraction algorithm to obtain the part point cloud corresponding to the several irregularly shaped ship hull parts. The curvature and normal vector angle of each point cloud data point in the part point cloud are calculated according to the preset point cloud data point curvature calculation method, and then the top part point cloud corresponding to the top part is obtained from the part point cloud according to the curvature and normal vector angle. Extract the outer contour of the top-level part point cloud, and generate the bounding rectangle corresponding to the top-level part point cloud along the outer contour using a preset bounding box method; Based on the circumscribed rectangle, the three-dimensional coordinates of the centroid of the circumscribed rectangle are obtained through a preset centroid calculation formula, and the three-dimensional coordinates of the centroid are transformed using a three-dimensional calibration method to obtain the positioning coordinates of the top-level part.

2. The method for positioning irregularly shaped ship hull parts based on disordered stacking as described in claim 1, characterized in that, The step of performing point cloud scanning on several irregularly stacked ship hull parts in a tray to obtain a scene point cloud containing the several irregularly stacked ship hull parts includes: The 3D camera performs point cloud scanning on several irregularly shaped ship hull parts inside the tray without stacking them, and obtains a scene point cloud including the several irregularly shaped ship hull parts; wherein, the 3D camera is located directly above the tray.

3. The method for positioning irregularly shaped ship hull parts based on disordered stacking as described in claim 1, characterized in that, The step of removing background point clouds from the scene point cloud according to a preset background subtraction algorithm to obtain part point clouds corresponding to the plurality of irregularly shaped ship hull parts includes: Search for data points in the scene point cloud with the same coordinates as the background point cloud using a pre-constructed octree; The data points are removed from the scene point cloud to obtain the part point cloud corresponding to the plurality of irregularly shaped ship hull parts.

4. The method for positioning irregularly shaped ship hull parts based on disordered stacking as described in claim 1, characterized in that, The step of calculating the curvature and the angle between the normal vectors corresponding to each point cloud data point in the part's point cloud according to a preset point cloud data point curvature calculation method includes: Obtain the neighborhood point set corresponding to each point cloud data point, and calculate the centroid corresponding to the neighborhood point set; Based on the neighborhood point set and the centroid, calculate the covariance matrix corresponding to each point cloud data point; Eigenvalue decomposition is performed on the covariance matrix corresponding to each point cloud data point to obtain the first eigenvalue and the second eigenvalue corresponding to each point cloud data point. Based on the first feature value and the second feature value, the curvature corresponding to each point cloud data point is obtained through a preset curvature calculation formula; Calculate the feature vector of the smallest feature value among the first feature value and the second feature value, and obtain the normal of each point cloud data point and each neighboring point in its corresponding neighborhood point set; The angle of the normal vector corresponding to each point cloud data point is calculated using a preset formula based on the normal vector angle of each neighboring point in the neighborhood set for each point cloud data point.

5. The method for positioning irregularly shaped ship hull parts based on disordered stacking as described in claim 1, characterized in that, The step of generating the bounding rectangle corresponding to the point cloud of the top-level part along the outer contour using a preset bounding box method includes: The OBB bounding box corresponding to the outer contour and the two-dimensional coordinate axis corresponding to the OBB bounding box are generated according to the preset OBB bounding box method. The 0BB bounding box is expanded according to the axis direction of the two-dimensional coordinate axis and the preset expansion distance to obtain the outer rectangle corresponding to the point cloud of the top-level part.

6. The method for positioning irregularly shaped ship hull parts based on disordered stacking as described in claim 1, characterized in that, The step of obtaining the three-dimensional coordinates of the centroid of the circumscribed rectangle using a preset centroid calculation formula includes: The three-dimensional coordinates of the centroid of the circumscribed rectangle are calculated according to a preset software program and a preset centroid calculation formula; wherein, the centroid calculation formula is: Wherein, n represents the number of point cloud data points in the circumscribed rectangle, and x i y i and z i This represents the three-dimensional coordinates of the centroid corresponding to the i-th point cloud data point.

7. The method for positioning irregularly shaped ship hull parts based on disordered stacking as described in claim 2, characterized in that, The method of transforming the three-dimensional coordinates of the center of gravity to obtain the positioning coordinates of the top-level part using three-dimensional calibration includes: The coordinate system of the 3D camera is unified with the coordinate system of the robotic arm used to grasp the top part by hand-eye calibration, and the coordinate system transformation matrix between the 3D camera and the robotic arm is obtained. The three-dimensional coordinates of the center of gravity are transformed according to the coordinate system transformation matrix to obtain the positioning coordinates of the top part in the robotic arm.

8. A positioning system for irregularly shaped ship hull parts based on disordered stacking, characterized in that, The system includes a point cloud scanning module, a point cloud cropping module, a rectangle extraction module, and a coordinate positioning module; The point cloud scanning module is used to perform point cloud scanning on several irregularly shaped ship hull parts in the tray that do not need to be stacked, to obtain a scene point cloud containing the several irregularly shaped ship hull parts, and to remove the background point cloud from the scene point cloud according to a preset background subtraction algorithm, so as to obtain the part point cloud corresponding to the several irregularly shaped ship hull parts. The point cloud clipping module is used to calculate the curvature and normal vector angle corresponding to each point cloud data point in the part point cloud according to a preset point cloud data point curvature calculation method, and then obtain the top part point cloud corresponding to the top part from the part point cloud according to the curvature and normal vector angle. The rectangle extraction module is used to extract the outer contour of the top-level part point cloud and generate the bounding rectangle corresponding to the top-level part point cloud along the outer contour using a preset bounding box method. The coordinate positioning module is used to obtain the three-dimensional coordinates of the centroid of the circumscribed rectangle according to the circumscribed rectangle through a preset centroid calculation formula, and to transform the three-dimensional coordinates of the centroid using a three-dimensional calibration method to obtain the positioning coordinates of the top part.

9. A positioning system for irregularly shaped ship hull parts based on disordered stacking as described in claim 8, characterized in that, The point cloud scanning module includes a scanning unit and a clearing unit; The scanning unit is used to perform point cloud scanning on several non-stacked ship hull-shaped parts inside the tray according to a preset 3D camera, to obtain a scene point cloud including the several ship hull-shaped parts; wherein, the 3D camera is located directly above the tray. The clearing unit is used to search for data points with the same coordinates as the background point cloud from the scene point cloud using a pre-constructed octree; remove the data points from the scene point cloud to obtain the part point cloud corresponding to the plurality of ship hull irregular parts.

10. A positioning system for irregularly shaped ship hull parts based on disordered stacking as described in claim 8, characterized in that, The point cloud cropping module includes a curvature calculation unit and an angle calculation unit; The curvature calculation unit is used to obtain the neighborhood point set corresponding to each point cloud data point and calculate the centroid corresponding to the neighborhood point set. Based on the neighborhood point set and the centroid, calculate the covariance matrix corresponding to each point cloud data point; Eigenvalue decomposition is performed on the covariance matrix corresponding to each point cloud data point to obtain the first eigenvalue and the second eigenvalue corresponding to each point cloud data point. Based on the first feature value and the second feature value, the curvature corresponding to each point cloud data point is obtained through a preset curvature calculation formula; The included angle calculation unit is used to calculate the feature vector of the smallest feature value among the first feature value and the second feature value, and obtain the normal of each point cloud data point and each neighboring point in its corresponding neighborhood point set; based on the normal of each point cloud data point and each neighboring point in its corresponding neighborhood point set, the included angle of the normal vector corresponding to each point cloud data point is calculated by a preset formula for calculating the included angle of the normal vector.

Images