Crane suspended load anti-collision detection method based on real-time three-dimensional point cloud driving

Abstract

The embodiment of the invention discloses a crane suspended load anti-collision detection method based on real-time three-dimensional point cloud driving, and the method comprises the steps: obtaining a suspended load three-dimensional detection region through a three-dimensional position coordinate of a lifting hook, carrying out the clustering of a third region three-dimensional point cloud in the suspended load three-dimensional detection region, obtaining a plurality of point cloud clusters, and carrying out the scoring, and obtaining an optimal point cloud cluster with the lowest score to obtain an accurate hoisting object axis alignment bounding box based on the optimal point cloud cluster, and determining an anti-collision detection area bounding box based on the size of the accurate hoisting object axis alignment bounding box, so as to obtain an anti-collision detection area according to the three-dimensional point cloud of the first area. And obtaining a fourth area three-dimensional point cloud arranged between the accurate hoisting and loading shaft alignment bounding box and the anti-collision detection area bounding box, and carrying out anti-collision detection according to the fourth area three-dimensional point cloud. The method can be applied to three-dimensional size identification of the lifting object and anti-collision detection among a shipbuilding portal crane body, the lifting object and surrounding potential obstacles in the construction process.

Description

Technical Field

[0001] This invention relates to the field of crane safety technology, and in particular to a crane load collision detection method based on real-time three-dimensional point cloud driving. Background Technology

[0002] In shipbuilding and the assembly of large structural components, shipbuilding gantry cranes undertake a large number of heavy component hoisting tasks. Due to the segmented hull, the diverse types and shapes of components, and the complex hoisting environment, there is a risk of collision between the gantry crane itself, the hoisted load, and surrounding facilities, personnel, or other components. A collision could not only damage equipment but also severely impact production progress and create safety hazards. Therefore, achieving accurate three-dimensional dimensional perception of the hoisted load and comprehensive collision avoidance detection of the work space during hoisting has become a crucial issue for improving the safety and efficiency of lifting operations.

[0003] Currently, several technical solutions exist in the industry for collision avoidance or object recognition of lifting equipment. However, these solutions suffer from the following drawbacks: Methods that use industrial cameras to capture images and pre-trained deep learning models to identify and locate specific types of hoisted objects perform well when recognizing known, limited-type objects. However, shipyards and similar environments involve thousands of different components with vastly different shapes and sizes, making it difficult for a single model to cover all targets. Furthermore, two-dimensional vision lacks depth information, making it impossible to directly obtain the three-dimensional dimensions of the hoisted object or accurately determine its spatial relationship with surrounding obstacles. Therefore, existing solutions are insufficient in addressing the complex and diverse hoisting objects in shipyards, the full-space monitoring of large hoisted objects, and the adaptability to the special structures of gantry cranes. They fail to meet the requirements for high-precision, full-coverage, and real-time collision avoidance and size recognition, thus failing to meet the collision avoidance needs under complex working conditions. Summary of the Invention

[0004] This invention discloses a crane load collision avoidance detection method based on real-time three-dimensional point cloud driving, in order to overcome the above-mentioned technical problems.

[0005] To achieve the above objectives, the technical solution of the present invention is as follows: A crane load collision avoidance detection method based on real-time 3D point cloud driving includes the following steps: S1: Obtain the 3D point cloud of the first area within the working area of ​​the shipbuilding gantry crane and the 3D position coordinates of the hook of the shipbuilding gantry crane; wherein, the shipbuilding gantry crane is a gantry crane or a portal crane. S2: Based on the three-dimensional position coordinates of the hook, determine the three-dimensional detection area of ​​the load, and obtain the three-dimensional point cloud of the third region within the three-dimensional detection area of ​​the load; S3: Based on the three-dimensional point cloud of the third region within the three-dimensional detection area of ​​the suspended object, multiple point cloud clusters are obtained based on a clustering algorithm, and scores are obtained for the multiple point cloud clusters. Then, the point cloud cluster with the lowest score is obtained; and the optimal point cloud cluster is obtained. S4: Based on the optimal cloud cluster, obtain the precise load shaft alignment bounding box to determine the precise size of the load shaft alignment bounding box; S5: Based on the precise dimensions of the sling shaft alignment bounding box, determine the collision avoidance detection area bounding box, and obtain the third-dimensional point cloud of the fourth area set between the precise sling shaft alignment bounding box and the collision avoidance detection area bounding box according to the first area three-dimensional point cloud, and perform collision avoidance detection according to the third-dimensional point cloud of the fourth area.

[0006] Furthermore, when the shipbuilding gantry crane is a gantry crane, the method for obtaining the three-dimensional position coordinates of the shipbuilding gantry crane's hook is as follows: S11: The theoretical three-dimensional position coordinates of the hook are obtained as follows:

[0007]

[0008]

[0009] In the formula: This represents the theoretical position coordinates of the hook in the X direction; This represents the theoretical position coordinates of the hook in the Y direction; This represents the theoretical position coordinates of the hook in the Z direction; This represents the X-axis coordinate of the origin of the trolley's coordinate system; This represents the Y-axis coordinate of the origin of the trolley's coordinate system; Indicates the real-time height of the hook bottom; This indicates the real-time position of the trolley in the X direction, that is, the real-time position of the trolley along the trolley track. This indicates the offset of the hook in the X direction; This indicates the offset of the hook in the Y direction; S12: Based on the XY plane where the maximum coordinate of the hook top in the Z direction is located, the three-dimensional point cloud of the first region is segmented to obtain the three-dimensional point cloud of the second region above the XY plane where the maximum coordinate of the hook top in the Z direction is located. S13: Cluster the 3D point cloud of the second region to obtain M rope-related point cloud clusters, and obtain the cluster center coordinates of the m-th rope-related point cloud cluster. Where m is the index of the suspending tether point cloud cluster; M is the total number of suspending tether point cloud clusters; These are the coordinates of the cluster center of the m-th suspending rope point cloud cluster in the X, Y, and Z directions, respectively. S14: When the number of cloud clusters on the lifting rope M is less than the number of hooks N: S141: Obtain the distance between the coordinates of the cluster center of the first sling point cloud cluster in the XY plane and the coordinates of the theoretical 3D positions of all hooks in the XY plane. ; S142: When When the minimum value is reached, the final three-dimensional position coordinates of the nth hook are obtained as follows:

[0010] In the formula: These are the three-dimensional position coordinates of the nth hook in the X, Y, and Z directions, respectively. These are the coordinates of the first suspending rope point cloud cluster in the X and Y directions, respectively; This represents the theoretical position coordinates of the nth hook in the Z direction; S143: Based on the 2nd and 3rd sequentially For M suspending rope point cloud clusters, repeat S141-S142; S144: At this point, the final three-dimensional position coordinates of the remaining NM hooks are as follows: ; S15: When the number of cloud clusters on the lifting rope M is greater than or equal to the number of hooks N: Obtain the distance between the theoretical 3D position of the nth hook in the XY plane and the coordinates of the cluster center of all the rope point cloud clusters in the XY plane. ; when When the minimum value is reached, the final three-dimensional position coordinates of the nth hook are as follows: .

[0011] Furthermore, when the shipbuilding gantry crane is a portal crane, the method for obtaining the three-dimensional position coordinates of the hook is as follows; S111: Obtain the position coordinates of the crane boom after rotation relative to the cab:

[0012]

[0013] In the formula: These are the X-coordinates and Y-coordinates of the position of the crane boom relative to the cab after rotation. Indicates transpose; These are the X-coordinates and Y-coordinates of the initial position of the crane boom relative to the cab, respectively. This is the two-dimensional rotation matrix of the crane about the Z-axis; The two-dimensional rotation angle of the cockpit around the Z-axis; S112: Obtain the increase in end displacement of the hook in the XY plane along the extension direction of the crane boom;

[0014] In the formula: This indicates the increase in end displacement of the hook in the XY plane along the extension direction of the crane boom; This represents the two-dimensional rotation angle of the cockpit around the Z-axis; Indicates the pitch angle of the crane boom; Indicates the length of the crane boom; express The component in the X direction; express The component in the Y direction; S113: Obtain the three-dimensional position coordinates of the hook as follows:

[0015] In the formula: The X-axis position coordinates of the cockpit relative to the origin of the gantry crane coordinate system; The position coordinates of the cockpit relative to the origin of the gantry crane coordinate system are in the Y direction. Expanded into an explicit formula, it is expressed as follows:

[0016]

[0017]

[0018] In the formula: This indicates the position coordinates of the hook in the X direction; This indicates the position coordinates of the hook in the Y direction; This indicates the position coordinates of the hook in the Z direction.

[0019] Furthermore, when the shipbuilding gantry crane is a portal crane, the three-dimensional detection area of ​​the hoisted object is obtained as follows:

[0020] In the formula: This is the three-dimensional detection area for the suspended load. The coordinates of any point in the three-dimensional detection area of ​​the suspended object in the global coordinate system are defined in the X direction. The coordinates of any point in the 3D detection area of ​​the suspended object in the global coordinate system are in the Y direction. The coordinates of any point in the three-dimensional detection area of ​​the suspended object in the global coordinate system are in the Z direction. It is a three-dimensional space; This indicates the position coordinates of the hook in the X direction; This indicates the position coordinates of the hook in the Y direction; This indicates the position coordinates of the hook in the Z direction; This represents the ground height value along the Z-axis in the global coordinate system. The ground clearance height value; Indicates transpose; in,

[0021]

[0022] In the formula: This represents the actual half-expansion in the X direction; This represents the actual half-expansion in the Y direction; The scaling function varies piecewise with weight; The unscaled baseline half-expansion in the X direction; The unscaled baseline half-expansion in the Y direction;

[0023] In the formula: This represents the current weight of the load being lifted. This is the scaling factor; This is the scaling limit value.

[0024] Furthermore, when the shipbuilding gantry crane is a gantry crane, the three-dimensional detection area of ​​the hoisted object is: When the number of hooks is 1, the three-dimensional detection area of ​​the load is:

[0025]

[0026]

[0027] In the formula: The unfolding distance of the three-dimensional detection area of ​​the suspended object set in the X direction; The unfolding distance for the three-dimensional detection area of ​​the suspended object set in the Y direction; The unfolding distance of the three-dimensional detection area of ​​the suspended object set in the Z direction; When the number of hooks is 2, the three-dimensional detection area of ​​the load is:

[0028]

[0029]

[0030]

[0031] In the formula: Here are the position coordinates of the first hook in the X direction; Here are the position coordinates of the second hook in the X direction; Here are the coordinates of the first hook in the Y direction; Here are the position coordinates of the second hook in the Y direction; Here are the coordinates of the first hook in the Z direction; Here are the coordinates of the second hook in the Z direction; This represents the maximum value of the Z-axis coordinates of the first and second hooks; When the number of hooks is greater than 2, the method for obtaining the three-dimensional detection area of ​​the load is as follows: First, obtain the smallest rectangular region of the hook in the XY plane of the global coordinate system:

[0032] in:

[0033]

[0034]

[0035]

[0036] In the formula: For the hook index; The total number of hooks in the load; For the first The X-axis coordinate of a hook in the global coordinate system; For the first The Y-coordinate of a hook in the global coordinate system; The smallest rectangular frame in the XY plane; The minimum X-axis coordinate of the hook in all loads in the global coordinate system; The maximum X-axis coordinate of the hook in the global coordinate system for all loads; The minimum Y-axis coordinate of the hook in all loads in the global coordinate system; This represents the maximum Y-axis coordinate of the hook in the global coordinate system for all loads. Indicates transpose; It is a two-dimensional space; Then, the three-dimensional detection area of ​​the suspended object at this time is obtained as follows:

[0037]

[0038] in,

[0039] In the formula: This represents the maximum Z-axis coordinate of the hook in the global coordinate system for all loads.

[0040] Furthermore, the method used to obtain the score of the point cloud cluster is as follows: S31: Based on the three-dimensional point cloud of the third region within the three-dimensional detection area of ​​the suspended object, multiple point cloud clusters are obtained using a clustering algorithm to obtain the position of the center point of the point cloud cluster and the size of the point cloud cluster; The position of the center point of the point cloud cluster is obtained as follows:

[0041] In the formula: The coordinates of the center point of the point cloud cluster; These are the coordinates of the current point cloud cluster's center point in the X, Y, and Z directions, respectively. These are the minimum and maximum values ​​of the X-direction coordinates of the axis-aligned bounding box of the point cloud cluster, respectively; These are the minimum and maximum values ​​of the Y-direction coordinates of the axis-aligned bounding box of the point cloud cluster, respectively; These are the minimum and maximum values ​​of the Z-direction coordinates enclosed by the axis alignment of the point cloud cluster, respectively; The dimensions of the point cloud cluster are obtained as follows:

[0042] In the formula: The dimension of the axis-aligned bounding box of the point cloud cluster at the current moment in the X direction, which is the width of the point cloud cluster at the current moment; The dimension of the axis-aligned bounding box of the point cloud cluster at the current moment in the Y direction, i.e., the length of the point cloud cluster at the current moment; The dimension of the axis-aligned bounding box of the point cloud cluster at the current moment in the Z direction is the height of the point cloud cluster at the current moment; S32: Obtain the Euclidean distance between the current position of the point cloud cluster center point and the previous position of the hoisted object center point; obtain the normalized upper bound of the Euclidean distance between the axis-aligned bounding box of the point cloud cluster and the center point of the axis-aligned bounding box of the previously identified target hoisted object. ; The formula used to obtain the Euclidean distance between the current position of the cloud cluster center point and the previous position of the hoisted object is as follows:

[0043] In the formula: distance is the Euclidean distance between the axis-aligned bounding box of the point cloud cluster and the center point of the axis-aligned bounding box of the previously identified target load, that is, the Euclidean distance between the current center point of the point cloud cluster and the center point of the previously identified load. Represents the square root; , , These are the X, Y, and Z coordinates of the center point of the current point cloud cluster, respectively. , , These are the X, Y, and Z coordinates of the center point of the target hoisting object's bounding box at the previous moment; The formula used to obtain the normalized upper bound of the Euclidean distance between the axis-aligned bounding box of the point cloud cluster and the center point of the axis-aligned bounding box of the previously identified target sling is as follows:

[0044] In the formula: The normalized upper bound of the Euclidean distance between the axis-aligned bounding box of the point cloud cluster and the center point of the axis-aligned bounding box of the previously identified target sling; For the first The Euclidean distance between the axis-aligned bounding box of a point cloud cluster and the center point of the axis-aligned bounding box of the previously identified target load; For indexes of point cloud clusters; The total number of point cloud clusters; S33: Obtain the 3D size difference between the axis-aligned bounding box of the current point cloud cluster and the axis-aligned bounding box of the target hoist at the previous time step: Obtain the normalized upper bound of the 3D size difference between the current point cloud cluster and the axis-aligned bounding box of the target hoist at the previous time step. ; The formula used to obtain the 3D size difference between the axis-aligned bounding box of the current point cloud cluster and the axis-aligned bounding box of the target hoisting object at the previous moment is as follows:

[0045] In the formula: This is the difference between the current 3D size of the point cloud cluster and the 3D size of the axis-aligned bounding box of the target hoisting object at the previous moment; These are the width, length, and height of the axis-aligned bounding box of the target hoisting object at the previous moment, respectively. The formula used to obtain the normalized upper bound of the difference between the 3D size of the current point cloud cluster and the 3D size of the axis-aligned bounding box of the target hoisting object at the previous moment is as follows:

[0046] In the formula: This is the normalized upper bound of the difference between the 3D size of the current point cloud cluster and the 3D size of the axis-aligned bounding box of the target load at the previous moment; S34: Obtain the Euclidean distance from the center point of the axis-aligned bounding box of the point cloud cluster to the center point of the upper surface of the 3D detection area of ​​the suspended object: Obtain the normalized upper bound of the Euclidean distance from the center point of the axis-aligned bounding box of the point cloud cluster to the center point of the upper surface of the 3D detection area of ​​the suspended object. ; The formula used to obtain the Euclidean distance from the center point of the axis-aligned bounding box of the point cloud cluster to the center point of the upper surface of the 3D detection area of ​​the suspended object is as follows:

[0047] In the formula: The Euclidean distance from the center point of the axis-aligned bounding box of the point cloud cluster to the center point of the upper surface of the 3D detection area of ​​the suspended object; , , These are the X, Y, and Z coordinates of the center point of the upper surface of the three-dimensional detection area of ​​the hoisted object at the current moment; The formula used to obtain the normalized upper bound of the Euclidean distance from the center point of the axis-aligned bounding box of the point cloud cluster to the center point of the upper surface of the 3D detection area of ​​the suspended object is as follows:

[0048] In the formula: The normalized upper bound of the Euclidean distance from the center point of the axis-aligned bounding box of the point cloud cluster to the center point of the upper surface of the 3D detection area of ​​the suspended object; S35: Obtain the Euclidean distance between the axis-aligned bounding box of the normalized point cloud cluster and the center point of the axis-aligned bounding box of the previously identified target load, the 3D size difference between the axis-aligned bounding box of the normalized point cloud cluster and the axis-aligned bounding box of the target load at the previous moment, and the Euclidean distance from the center point of the axis-aligned bounding box of the normalized point cloud cluster to the center point of the upper surface of the 3D detection area of ​​the load. The formula used is as follows:

[0049]

[0050]

[0051] In the formula: The Euclidean distance between the axis-aligned bounding box of the normalized point cloud cluster and the center point of the axis-aligned bounding box of the previously identified target sling; The difference in three-dimensional dimensions between the axis-aligned bounding box of the normalized point cloud cluster and the axis-aligned bounding box of the target hoist at the previous moment. The Euclidean distance from the center point of the axis-aligned bounding box of the normalized point cloud cluster to the center point of the upper surface of the 3D detection area of ​​the suspended object; This is a parameter used to ensure that the denominator is not zero; S36: When the size of a point cloud cluster is greater than a size threshold, a score for the point cloud cluster is obtained using the following formula:

[0052] In the formula: They are respectively The weights.

[0053] Furthermore, the center of the collision avoidance detection area enclosure coincides with the center and centerline of the precise load shaft alignment enclosure, and the size of the collision avoidance detection area enclosure is larger than the size of the precise load shaft alignment enclosure.

[0054] Furthermore, S5 includes: S51: Based on the precise load axis alignment bounding box and the anti-collision detection area bounding box, determine the detection area set between the precise load axis alignment bounding box and the anti-collision detection area bounding box, so as to determine the first detection area and the second detection area. The first detection area includes: Each of the b-th side of the enclosure is precisely aligned with the axis of the suspended object as the bottom surface, b=1,2,3,4, and the top surface is set on the four cuboids on the side of the enclosure corresponding to the anti-collision detection area. The bottom surface of the enclosure is precisely aligned with the axis of the suspended object, and the top surface is set on the fifth cuboid on the bottom surface of the enclosure in the collision detection area. The second detection area is the area outside the first detection area, which is between the precise load shaft alignment enclosure and the anti-collision detection area enclosure. S52: The method used to determine whether there is a collision risk in the first detection area is as follows: If a point in the fourth region of the 3D point cloud within the b-th cuboid, where b=1,2,3,4, has a distance less than a set distance threshold between itself and the b-th side of the bounding box precisely aligned with the axis of the suspended object, then there is a risk of collision. If a point in the 3D point cloud of the fourth region within the 5th cuboid has a distance less than the set distance threshold between itself and the bottom surface of the bounding box that is precisely aligned with the axis of the suspended object, then there is a risk of collision. S53: The method used to determine whether there is a collision risk in the second detection area is as follows: Obtain a line segment with endpoints of a point in the 3D point cloud of the fourth region within the second detection area and the center point of the precise sling shaft alignment bounding box. Obtain the distance between the intersection of the line segment and the precise sling shaft alignment bounding box and the point in the 3D point cloud of the fourth region within the second detection area. If the distance is less than a set distance threshold, there is a risk. S54: If there is a collision risk in either the first detection area or the second detection area, then the crane load is at risk of collision.

[0055] Beneficial Effects: The present invention provides a crane load anti-collision detection method based on real-time 3D point cloud driving. It obtains the 3D detection area of ​​the load using the 3D position coordinates of the hook, clusters the 3D point cloud of the third region within the 3D detection area to obtain multiple point cloud clusters, scores them, and obtains the lowest-scoring optimal point cloud cluster. A precise load axis alignment bounding box is then obtained based on the optimal point cloud cluster. Based on the dimensions of the precise load axis alignment bounding box, an anti-collision detection area bounding box is determined. A fourth region 3D point cloud is then obtained based on the first region 3D point cloud, positioned between the precise load axis alignment bounding box and the anti-collision detection area bounding box. Anti-collision detection is then performed based on the fourth region 3D point cloud. This invention enables collision avoidance detection for two different configurations of shipbuilding gantry cranes during operation. It can accurately perceive the three-dimensional dimensions of the hoisted object during lifting, and perform all-around collision avoidance detection of the work area. In the complex and diverse hoisting scenarios of shipyards, it can cover the detection of objects of different shapes and sizes, accurately determine the relationship between the hoisted object and surrounding obstacles, and achieve high detection accuracy, meeting the collision avoidance requirements under complex working conditions. It can be applied to the three-dimensional size recognition of hoisted objects during construction and to collision avoidance detection between the shipbuilding gantry crane body, the hoisted object, and surrounding potential obstacles. Attached Figure Description

[0056] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0057] Figure 1 This is a flowchart of the crane load anti-collision detection method based on real-time 3D point cloud driving according to the present invention; Figure 2 This is a lateral schematic diagram of the laser radar scanning range of the gantry crane in an embodiment of the present invention; Figure 3 This is a forward schematic diagram of the laser radar scanning range of the gantry crane in an embodiment of the present invention; Figure 4 This is a forward schematic diagram of the laser radar scanning range of the gantry crane in an embodiment of the present invention; Figure 5 This is a top view schematic diagram of the laser radar scanning range of the gantry crane in an embodiment of the present invention; Figure 6 This is a schematic diagram of the anti-collision detection area in an embodiment of the present invention; Figure 7 This is a schematic diagram of line segments in the spherical collision avoidance detection of this invention.

[0058] Among them, 1. boom; 2. load; 31. first lidar; 32. second lidar; 4. cab; A. spatial point; B. center point of the load enclosure box. Detailed Implementation

[0059] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0060] This embodiment introduces a crane load collision avoidance detection method based on real-time 3D point cloud driving, including the following steps, such as... Figure 1 As shown: S1: Obtain the 3D point cloud of the first area within the working area of ​​the shipbuilding gantry crane and the 3D position coordinates of the hook of the shipbuilding gantry crane; wherein, the shipbuilding gantry crane is a gantry crane or a portal crane. In this embodiment, all three-dimensional point cloud data within the working area of ​​the crane and the three-dimensional position coordinates of all hooks on the body of the shipbuilding gantry crane (gantry crane or portal crane) are acquired; wherein, gantry cranes (hereinafter referred to as gantry cranes) and portal cranes in the field of lifting equipment such as ports or shipbuilding are collectively referred to as shipbuilding gantry cranes.

[0061] In this embodiment, the first region's 3D point cloud is obtained by merging point cloud data of the working area obtained from two LiDAR scans, downsampling, and segmenting the data within the gantry crane coordinate system. Specifically, the designated space includes the working area and the 3D space within a certain distance on both sides. For a gantry crane, this refers to the 3D space in the left-right and up-down directions facing the cab 4. The specific working area range depends on the scanning range and installation angle of the LiDAR, such as... Figure 2 and Figure 3As shown. Based on two lidar scans, all objects such as parts, ships, transport vehicles, and people within the working area of ​​the crane will be reflected in real time, resulting in a three-dimensional point cloud within the designated space of the crane.

[0062] This embodiment uses a PLC and encoder installed on the crane to collect real-time crane data: for gantry cranes, the real-time crane data includes the positions of all trolleys on the main body, the heights of all hooks, and the real-time load capacity of all hooks; for portal cranes, the real-time crane data includes the rotation angle of the rotating platform on the main body, the pitch angle of the boom, the heights of all hooks, and the real-time load capacity of all hooks. Based on the real-time crane data and the kinematic model of the shipbuilding gantry crane, the original three-dimensional coordinates of all hooks in the gantry crane coordinate system are calculated.

[0063] Preferably, when the shipbuilding gantry crane is a gantry crane, the method for obtaining the three-dimensional position coordinates of the hook of the shipbuilding gantry crane is as follows: Specifically, in this embodiment, the kinematic model of the shipbuilding gantry crane is entirely based on the gantry crane's trolley coordinate system, i.e., the global coordinate system. In this embodiment, the global coordinate system (i.e., the trolley coordinate system) has the vertically upward direction as the positive Z-direction, and the plane containing the trolley track as the XY plane. The direction along the trolley track is the X-direction, and the direction perpendicular to the trolley track within the XY plane is the Y-direction. When the shipbuilding gantry crane is a gantry crane, the three-dimensional position of the hook is obtained based on the gantry crane's kinematic model as follows: S11: The theoretical three-dimensional position coordinates of the hook are obtained as follows: First, obtain the position of the origin of the car's coordinate system relative to the origin of the large vehicle's coordinate system:

[0064] In the formula: This indicates the position of the origin of the trolley coordinate system relative to the base, that is, the position relative to the origin of the mainframe coordinate system; This represents the X-axis coordinate of the origin of the trolley's coordinate system; This represents the Y-axis coordinate of the origin of the trolley's coordinate system; This represents the Z-axis coordinate of the origin of the trolley's coordinate system; Secondly, the theoretical three-dimensional position coordinates of the hook are obtained as follows: 1. The theoretical position of the hook in the X direction is obtained as follows:

[0065] In the formula: This represents the theoretical position coordinates of the hook in the X direction; This represents the X-axis coordinate of the origin of the trolley's coordinate system; This indicates the real-time position of the trolley in the X direction, that is, the real-time position of the trolley along the trolley track. This indicates the offset of the hook in the X direction; In this embodiment, the direction in which the trolley travels along the track is defined as the X-axis direction.

[0066] 2. The theoretical position of the hook in the Y direction;

[0067] In the formula: This represents the theoretical position coordinates of the hook in the Y direction; This represents the Y-axis coordinate of the origin of the trolley's coordinate system; This indicates the offset of the hook in the Y direction; 3. The theoretical position of the hook in the Z direction;

[0068] In the formula: This represents the theoretical position coordinates of the hook in the Z direction; Indicates the real-time height of the hook bottom; Finally, the theoretical three-dimensional position of the hook (hook bottom) in the global coordinate system is obtained:

[0069] In the formula: This represents the theoretical three-dimensional position of the hook in the global coordinate system.

[0070] Specifically, when calculating the three-dimensional position coordinates of the hook, the offset of the hook in the X direction... The offset of the hook in the Y direction Real-time height of hook bottom Real-time position of the trolley along the track direction It is known.

[0071] S12: Based on the XY plane where the maximum coordinate of the hook top in the Z direction is located, the three-dimensional point cloud of the first region is segmented to obtain the three-dimensional point cloud of the second region above the XY plane where the maximum coordinate of the hook top in the Z direction is located. S13: Cluster the 3D point cloud of the second region to obtain M rope-related point cloud clusters, and obtain the cluster center coordinates of the m-th rope-related point cloud cluster. Where m is the index of the suspending tether point cloud cluster; M is the total number of suspending tether point cloud clusters; These are the coordinates of the cluster center of the m-th suspending rope point cloud cluster in the X, Y, and Z directions, respectively. S14: When the number of cloud clusters on the lifting rope M is less than the number of hooks N: S141: Obtain the distance between the coordinates of the cluster center of the first sling point cloud cluster in the XY plane and the coordinates of the theoretical 3D positions of all hooks in the XY plane. The formula used is as follows:

[0072] In the formula: Let be the distance between the cluster center coordinates of the m-th hoisting rope point cloud cluster and the theoretical three-dimensional position coordinates of the n-th hook in the XY plane; Let X be the theoretical position coordinates of the nth hook in the X direction; This represents the theoretical position coordinates of the nth hook in the Y direction; S142: When When the minimum value is reached, the final three-dimensional position coordinates of the nth hook are obtained as follows:

[0073] In the formula: These are the three-dimensional position coordinates of the nth hook in the X, Y, and Z directions, respectively, which are the three-dimensional position coordinates of the hook after the second calibration. These are the coordinates of the first suspending rope point cloud cluster in the X and Y directions, respectively; This represents the theoretical position coordinates of the nth hook in the Z direction; S143: Based on the 2nd and 3rd sequentially For M suspending rope point cloud clusters, repeat S141-S142; S144: At this point, the final three-dimensional position coordinates of the remaining NM hooks are as follows:

[0074] S15: When the number of cloud clusters on the lifting rope M is greater than or equal to the number of hooks N: Obtain the distance between the theoretical 3D position of the nth hook in the XY plane and the coordinates of the cluster center of all the rope point cloud clusters in the XY plane. ; when When the minimum value is reached, the final three-dimensional position coordinates of the nth hook are as follows: .

[0075] Specifically, the nearest-distance pairing method is applicable to obtaining the final three-dimensional position coordinates of the lifting hooks when the number of lifting hooks is greater than 2 (especially when the number of lifting hooks is 2 or 4); where when the number of lifting hooks is 1, there is no need to perform secondary calibration on the position of the lifting hook. First, segment the three-dimensional point cloud in the first region, obtain the three-dimensional point cloud in the second region and perform clustering, which can obtain multiple lifting rope point cloud clusters, and calculate the centroid coordinates of each cluster. To avoid the influence of the disturbance of the lifting rope height on the matching, only the X and Y direction coordinate information of the centroid is used during the matching. The specific pairing method is as follows: for each lifting hook, find the centroid in the "unassigned" centroids that is the closest to it in the XY plane, and assign this centroid to this lifting hook. To ensure a one-to-one relationship, once a centroid is assigned to a certain lifting hook, it is removed from the candidate set and no longer participates in the subsequent matching of the lifting hooks. When the number M of the lifting rope point cloud clusters is greater than or equal to the number N of the lifting hooks (M≥N), if the distance between the nth lifting hook and the mth point cloud cluster is the smallest, then the X and Y direction coordinates of the centroid of the mth point cloud cluster replace the X and Y direction coordinates of the nth lifting hook, so as to achieve the secondary correction of the horizontal position of the lifting hook, and the remaining point cloud clusters that are not matched with the lifting hooks are regarded as interference or non-target point clouds and ignored; the Z direction coordinate of the lifting hook is not modified. When the number of centroids is less than the number of lifting hooks (M<N), then each point cloud cluster is matched with the closest lifting hook to update the final three-dimensional position coordinates of the lifting hook after secondary calibration, and the lifting hooks that do not obtain the centroid assignment remain in their original positions unchanged. This method is simple to implement and has low prior dependence on the on-site structure, and is applicable to most working conditions.

[0076] In this embodiment, based on the theoretical three-dimensional position coordinates of the lifting hook, the position of the lifting hook is secondarily calibrated: by obtaining the point cloud clusters of the lifting ropes above all the lifting hooks through a point cloud clustering algorithm. Specifically, build a Kd-Tree on the three-dimensional point cloud in the second region and call the PCL Euclidean clustering to complete the clustering; where the clustering tolerance and the minimum / maximum cluster size are configurable parameters, and can be calibrated and adjusted according to on-site conditions (such as the crossbeam spacing, the diameter of the lifting rope, and the point cloud density). In this embodiment, the tolerance is about 1.0m, the minimum is 6 points, and the maximum is 1000 points (both are recommended values, not limited). Extract and calculate the centroid of each cluster according to the cluster index to obtain the set of lifting rope point cloud clusters. Then match the point cloud clusters of the lifting ropes above all the lifting hooks with the theoretical three-dimensional position coordinates of the lifting hooks, and update to obtain the secondarily calibrated three-dimensional coordinates of all the lifting hooks in the portal crane coordinate system.

[0077] Preferably, when the shipbuilding portal crane is a pedestal crane, the method for obtaining the three-dimensional position coordinates of the lifting hook is as follows; Specifically, based on the kinematic model of the lifting hook of the pedestal crane in this embodiment, the three-dimensional position of the lifting hook based on the portal crane coordinate system is obtained: where, the position of the cab (cabprime) relative to the origin (pedestal) of the portal crane coordinate system: and the position of the jib relative to the cab: (Only the X and Y coordinates participate in the planar rotation) is known, where, The X-axis position coordinates of the cockpit relative to the origin of the gantry crane coordinate system; The position coordinates of the cockpit relative to the origin of the gantry crane coordinate system are in the Y direction. The coordinates of the cockpit relative to the origin of the gantry crane coordinate system are in the Z direction.

[0078] S111: Obtain the position coordinates of the crane boom after rotation relative to the cab: Specifically, the position of the crane boom (jib) relative to the cab. Rotate the crane arm relative to the cab within the plane to obtain its position after rotation.

[0079]

[0080] In the formula: These are the X-coordinates and Y-coordinates of the position of the crane boom relative to the cab after rotation. Indicates transpose; These are the X-coordinates and Y-coordinates of the initial position of the crane boom relative to the cab, respectively. The two-dimensional rotation matrix of the crane about the Z-axis (acting on) flat); S112: Calculate the two-dimensional rotation angle of the cab around the Z-axis. (S112: Obtain the increase in end displacement of the hook in the XY plane along the extension direction of the crane boom.) Specifically, increase the end displacement (of the boom length) in the plane along the forward extension direction of the crane boom. Subject to the pitch angle of the crane boom relative to its own axis Horizontal projection modulation):

[0081] In the formula: This indicates the increase in end displacement of the hook in the XY plane along the extension direction of the crane boom; This represents the two-dimensional rotation angle of the cockpit around the Z-axis; Indicates the pitch angle of the crane boom; Indicates the length of the crane boom; express The component in the X direction; express The component in the Y direction; S113: Obtain the three-dimensional position coordinates of the hook as follows: that is, the position of the hook (hook bottom) in the global coordinate system:

[0082] In the formula: The X-axis position coordinates of the cockpit relative to the origin of the gantry crane coordinate system; The position coordinates of the cockpit relative to the origin of the gantry crane coordinate system are in the Y direction. Expanded into an explicit formula, it is expressed as follows:

[0083]

[0084] ; In the formula: This indicates the position coordinates of the hook in the X direction; This indicates the position coordinates of the hook in the Y direction; This indicates the position coordinates of the hook in the Z direction.

[0085] S2: Determine the three-dimensional detection area of ​​the load based on the three-dimensional position coordinates of the hook; obtain the three-dimensional point cloud within the three-dimensional detection area of ​​the load, i.e., the three-dimensional point cloud of the third region; Specifically, based on the three-dimensional position coordinates of the hook and the characteristics of the gantry crane body (i.e., the number of hooks involved in the lifting), different three-dimensional detection areas for the load are determined. The position of the three-dimensional detection area of ​​the load is jointly determined by the hook position and the characteristics of the gantry crane body. The hook position, for a gantry crane, is the position after the secondary calibration. For a portal crane, since it does not involve secondary calibration, it is the position of the hook (hook bottom) in the global coordinate system calculated based on the kinematic model of the gantry crane hook.

[0086] In this embodiment, the gantry crane body feature refers to the number of hooks that the gantry crane itself may simultaneously participate in lifting. For a portal crane, it is generally one hook; for a gantry crane, there are various situations where one to four hooks participate in lifting simultaneously.

[0087] Preferably, when the shipbuilding gantry crane is a portal crane, the three-dimensional detection area of ​​the hoisted object is obtained as follows:

[0088] In the formula: This is the three-dimensional detection area for the suspended load. The coordinates of any point in the three-dimensional detection area of ​​the suspended object in the global coordinate system are defined in the X direction. The coordinates of any point in the 3D detection area of ​​the suspended object in the global coordinate system are in the Y direction. The coordinates of any point in the three-dimensional detection area of ​​the suspended object in the global coordinate system are in the Z direction. It is a three-dimensional space; This indicates the position coordinates of the hook in the X direction; This indicates the position coordinates of the hook in the Y direction; This indicates the position coordinates of the hook in the Z direction; This represents the ground height value along the Z-axis in the global coordinate system. The ground clearance height value; in,

[0089]

[0090] In the formula: This represents the actual half-expansion in the X direction; This represents the actual half-expansion in the Y direction; The scaling function varies piecewise with weight; The unscaled baseline half-expansion in the X direction; The unscaled baseline half-expansion in the Y direction;

[0091] In the formula: This represents the current weight of the load being lifted. This is the scaling factor; This is the scaling limit value.

[0092] Specifically, the determination of the three-dimensional detection area of ​​the load of the gantry crane is only related to the position of a single hook in the current load, and its shape is a cuboid.

[0093] Specifically, for ease of description, a global coordinate system is used. Where X and Y are the horizontal (XY plane) coordinate axes, and Z is the vertical direction. The position of the hook (hook bottom) in the global coordinate system is denoted as... .

[0094] Assume the ground is a plane of equal height. The ground clearing height is Vertical lower boundary With upper boundary Meet engineering constraints It also introduces the minimum vertical thickness of the three-dimensional detection area of ​​the hoisted object, defined according to actual working conditions. The ground clearance height represents the minimum coordinate of the three-dimensional detection area of ​​the suspended object in the Z direction in the global coordinate system.

[0095] like ,but ;like ≤ ,but .

[0096] To give the horizontal (XY plane) range, define an unscaled reference half-extension: its physical meaning corresponds to the reference half-length and half-width along the X and Y directions, respectively (obtained by a fixed distance configuration).

[0097] ,and 0, 0 In the formula: This represents the ground height value along the Z-axis in the global coordinate system. The ground clearance height value; The Z-axis height of the lower boundary of the three-dimensional detection area of ​​the suspended object in the global coordinate system; The height of the upper boundary of the three-dimensional detection area of ​​the suspended object on the Z-axis in the global coordinate system; The minimum vertical thickness of the three-dimensional detection area of ​​the suspended object in the global coordinate system; The unscaled reference half-expansion in the XY plane; The unscaled baseline half-expansion in the X direction; The unscaled baseline half-expansion in the Y direction; For transpose; Record the weight of the currently hoisted object as: Scaling limit value Scaling factor (0,1). Define a scaling function that varies piecewise with weight:

[0098] In the formula: This represents the current weight of the load being lifted. This is the scaling factor; This is the scaling limit value; The scaling function varies piecewise with weight; The actual half-expansion of the XY plane is:

[0099] Right now

[0100]

[0101] Therefore, in the XY plane, we obtain ( A rectangle aligned to the center axis:

[0102] In the formula: A rectangular area aligned to the axis; This represents the actual half-expansion in the XY plane; This represents the actual half-expansion in the X direction; This represents the actual half-expansion in the Y direction; In this embodiment, the "3D detection area of ​​the suspended load" (a cuboid aligned with the global coordinate axis, AABB) is defined as:

[0103] In the formula: To and The region related to the three defined quantities; Right now

[0104] In the formula: This is the three-dimensional detection area for the suspended load. The coordinates of any point in the three-dimensional detection area of ​​the suspended object in the global coordinate system are defined in the X direction. The coordinates of any point in the 3D detection area of ​​the suspended object in the global coordinate system are in the Y direction. The coordinates of any point in the three-dimensional detection area of ​​the suspended object in the global coordinate system are in the Z direction. It is a three-dimensional space; The above definition ensures that the upper boundary is always greater than the lower boundary in actual engineering. Based on this, the three-dimensional detection area R of the suspended object is obtained in the global coordinate system.

[0105] in, As configurable parameters, they can be calibrated and adjusted according to site conditions (such as beam spacing, rope diameter, point cloud density, etc.), while the algorithm remains unchanged; when At that time, the XY plane range is scaled by a scaling factor. Scaling should be done proportionally; otherwise, the baseline size should be maintained without scaling.

[0106] Preferably, when the shipbuilding gantry crane is a gantry crane, the three-dimensional detection area of ​​the hoisted object is: When the number of hooks is 1, the three-dimensional detection area of ​​the load is:

[0107]

[0108]

[0109] In the formula: The unfolding distance of the three-dimensional detection area of ​​the suspended object set in the X direction; The unfolding distance for the three-dimensional detection area of ​​the suspended object set in the Y direction; The unfolding distance of the three-dimensional detection area of ​​the suspended object set in the Z direction; When the number of hooks is 2, the three-dimensional detection area of ​​the load is:

[0110]

[0111]

[0112]

[0113] In the formula: Here are the position coordinates of the first hook in the X direction; Here are the position coordinates of the second hook in the X direction; Here are the coordinates of the first hook in the Y direction; Here are the position coordinates of the second hook in the Y direction; Here are the coordinates of the first hook in the Z direction; Here are the coordinates of the second hook in the Z direction; This represents the maximum value of the Z-axis coordinates of the first and second hooks; When the number of hooks is greater than 2, the method for obtaining the three-dimensional detection area of ​​the load is as follows: First, obtain the smallest rectangular region of the hook in the XY plane (i.e., the horizontal plane) of the global coordinate system:

[0114] in:

[0115]

[0116]

[0117]

[0118] In the formula: For the hook index; The total number of hooks in the load; For the first The X-axis coordinate of a hook in the global coordinate system; For the first The Y-coordinate of a hook in the global coordinate system; The smallest rectangular frame in the XY plane; The minimum X-axis coordinate of the hook in all loads in the global coordinate system; The maximum X-axis coordinate of the hook in the global coordinate system for all loads; The minimum Y-axis coordinate of the hook in all loads in the global coordinate system; This represents the maximum Y-axis coordinate of the hook in the global coordinate system for all loads. It is a two-dimensional space; Then, the three-dimensional detection area of ​​the suspended object at this time is obtained as follows:

[0119] in, .

[0120] In the formula: This represents the maximum Z-axis coordinate of the hook in the global coordinate system for all loads.

[0121] Specifically, when the shipbuilding gantry crane is a gantry crane, the three-dimensional detection area of ​​the load is related to the positions of multiple hooks in the current load, and its shape is cuboid. Since gantry cranes may contain two or four hooks depending on their tonnage, the case of a gantry crane with two hook types will be included in the scheme of a gantry crane with four hook types.

[0122] The height of the three-dimensional detection area for the hoisted object is defined as the height from the current hook height downwards to a specified ground clearance height above the ground. When there is only one hook in the hoisted object, the three-dimensional detection area expands downwards from the original three-dimensional coordinates of the hook in the gantry crane coordinate system, with the horizontal length and width determined by a fixed distance. When there are two hooks in the hoisted object, the algorithm calculates an equivalent hook (hook bottom) position in the global coordinate system. The point is denoted as the equivalent hook 3D space point. The calculation method for this equivalent hook 3D space point is as follows: calculate the average X and Y coordinates of the two hooks in the horizontal plane, and expand downwards with the maximum Z coordinate value among the two hooks as the center. The length and width in the XY plane are determined by a fixed distance set according to the project's load detection range. When the number of hooks in the load is three or four, the algorithm will use the smallest axis-aligned rectangle formed by all the hooks in the load on the horizontal plane as a basis, adding the fixed distance in the X and Y directions to form the final horizontal length and width values.

[0123] Based on the 3D detection area of ​​the suspended object, a 3D point cloud within the first region is obtained by segmenting the 3D point cloud of the first region, which is the 3D point cloud of the third region. The 3D point cloud of the third region is a subset of the 3D point cloud of the first region.

[0124] S3: Based on the three-dimensional point cloud of the third region within the three-dimensional detection area of ​​the suspended object, multiple point cloud clusters are obtained based on a clustering algorithm to obtain the scores of the multiple point cloud clusters, and then the point cloud cluster with the lowest score is obtained; finally, the optimal point cloud cluster is obtained.

[0125] S31: Based on the three-dimensional point cloud of the third region within the three-dimensional detection area of ​​the suspended object, multiple point cloud clusters are obtained using a clustering algorithm to obtain the position of the center point of the point cloud cluster and the size of the point cloud cluster; Preferably, the position of the center point of the point cloud cluster is obtained as follows:

[0126] In the formula: The coordinates of the center point of the point cloud cluster; These are the coordinates of the current point cloud cluster's center point in the X, Y, and Z directions, respectively. These are the minimum and maximum values ​​of the X-direction coordinates of the axis-aligned bounding box of the point cloud cluster, respectively; These are the minimum and maximum values ​​of the Y-direction coordinates of the axis-aligned bounding box of the point cloud cluster, respectively; These are the minimum and maximum values ​​of the Z-direction coordinates enclosed by the axis alignment of the point cloud cluster, respectively; The dimensions of the point cloud cluster are obtained as follows:

[0127] In the formula: The dimension of the axis-aligned bounding box of the point cloud cluster at the current moment in the X direction, which is the width of the point cloud cluster at the current moment; The dimension of the axis-aligned bounding box of the point cloud cluster at the current moment in the Y direction, i.e., the length of the point cloud cluster at the current moment; The dimension of the axis-aligned bounding box of the point cloud cluster at the current moment in the Z direction is the height of the point cloud cluster at the current moment; Specifically, the Axis-Aligned Bounding Box (AABB) is one of the most commonly used geometric concepts in computer graphics, collision detection, and spatial partitioning. In this embodiment, the Axis-Aligned Bounding Box (AABB) is the smallest hexahedron parallel to the coordinate axes that encloses the target object. The face parallel to the XY plane within the AABB is used as the bottom face, and the face perpendicular to the XY plane is used as the side face.

[0128] S32: Obtain the Euclidean distance between the current position of the point cloud cluster center point and the previous position of the hoisted object center point; obtain the normalized upper bound of the Euclidean distance between the axis-aligned bounding box of the point cloud cluster and the center point of the axis-aligned bounding box of the previously identified target hoisted object. ; The formula used to obtain the Euclidean distance between the current position of the cloud cluster center point and the previous position of the hoisted object is as follows:

[0129] In the formula: distance is the Euclidean distance between the axis-aligned bounding box of the point cloud cluster and the center point of the axis-aligned bounding box of the previously identified target load, that is, the Euclidean distance between the current center point of the point cloud cluster and the center point of the previously identified load. Represents the square root; , , These are the X, Y, and Z coordinates of the center point of the current point cloud cluster, respectively. , , These are the X, Y, and Z coordinates of the center point of the target hoisting object's bounding box at the previous moment; The formula used to obtain the normalized upper bound of the Euclidean distance between the axis-aligned bounding box of the point cloud cluster and the center point of the axis-aligned bounding box of the previously identified target sling is as follows:

[0130] In the formula: The normalized upper bound of the Euclidean distance between the axis-aligned bounding box of the point cloud cluster and the center point of the axis-aligned bounding box of the previously identified target sling; For the first The Euclidean distance between the axis-aligned bounding box of a point cloud cluster and the center point of the axis-aligned bounding box of the previously identified target load; For indexes of point cloud clusters; The total number of point cloud clusters; Obtain the 3D size difference between the axis-aligned bounding box of the current point cloud cluster and the axis-aligned bounding box of the target hoist at the previous time step: Obtain the normalized upper bound of the 3D size difference between the current point cloud cluster and the axis-aligned bounding box of the target hoist at the previous time step. ; The formula used to obtain the 3D size difference between the axis-aligned bounding box of the current point cloud cluster and the axis-aligned bounding box of the target hoisting object at the previous moment is as follows:

[0131] In the formula: This is the difference between the current 3D size of the point cloud cluster and the 3D size of the axis-aligned bounding box of the target hoisting object at the previous moment; These are the width, length, and height of the axis-aligned bounding box of the target hoisting object at the previous moment, respectively. The formula used to obtain the normalized upper bound of the difference between the 3D size of the current point cloud cluster and the 3D size of the axis-aligned bounding box of the target hoisting object at the previous moment is as follows:

[0132] In the formula: This is the normalized upper bound of the difference between the 3D size of the current point cloud cluster and the 3D size of the axis-aligned bounding box of the target load at the previous moment; S34: Obtain the Euclidean distance from the center point of the axis-aligned bounding box of the point cloud cluster to the center point of the upper surface of the 3D detection area of ​​the suspended object (i.e., the third region): Obtain the normalized upper bound of the Euclidean distance from the center point of the axis-aligned bounding box of the point cloud cluster to the center point of the upper surface of the 3D detection area of ​​the suspended object. The formula used to obtain the Euclidean distance from the center point of the axis-aligned bounding box of the point cloud cluster to the center point of the upper surface of the 3D detection area of ​​the suspended object (i.e., the third area) is as follows:

[0133] In the formula: The Euclidean distance is the distance from the center point of the axis-aligned bounding box of the point cloud cluster to the center point of the upper surface of the three-dimensional detection area (i.e., the third area) of the suspended object. , , These are the X, Y, and Z coordinates of the center point on the upper surface of the three-dimensional detection area (i.e., the third area) of the suspended object at the current moment. The formula used to obtain the normalized upper bound of the Euclidean distance from the center point of the axis-aligned bounding box of the point cloud cluster to the center point of the upper surface of the 3D detection area of ​​the suspended object is as follows:

[0134] In the formula: The normalized upper bound of the Euclidean distance from the center point of the axis-aligned bounding box of the point cloud cluster to the center point of the upper surface of the 3D detection area of ​​the suspended object; S35: Obtain the Euclidean distance between the axis-aligned bounding box of the normalized point cloud cluster and the center point of the axis-aligned bounding box of the previously identified target load, the 3D size difference between the axis-aligned bounding box of the normalized point cloud cluster and the axis-aligned bounding box of the target load at the previous moment, and the Euclidean distance from the center point of the axis-aligned bounding box of the normalized point cloud cluster to the center point of the upper surface of the 3D detection area of ​​the load. The formula used is as follows:

[0135]

[0136]

[0137] In the formula: The Euclidean distance between the axis-aligned bounding box of the normalized point cloud cluster and the center point of the axis-aligned bounding box of the previously identified target sling; The difference in three-dimensional dimensions between the axis-aligned bounding box of the normalized point cloud cluster and the axis-aligned bounding box of the target hoist at the previous moment. The Euclidean distance from the center point of the axis-aligned bounding box of the normalized point cloud cluster to the center point of the upper surface of the three-dimensional detection area of ​​the suspended object (i.e., the third region); In this embodiment, as a parameter used to ensure that the denominator is not zero, .

[0138] S36: When the size of a point cloud cluster is greater than the size threshold, that is, when the size of the axis-aligned bounding box of the point cloud cluster in the X square, the Y direction, and the Z direction are all greater than the size threshold in their respective directions, the score of the point cloud cluster is obtained. The formula used is as follows:

[0139] In the formula: They are respectively The weights.

[0140] Specifically, a score is calculated only when the three-dimensional dimensions of the point cloud cluster exceed the set length, width, and height thresholds, respectively, thus filtering out smaller objects. The length, width, and height thresholds are determined based on the specific project requirements. Ultimately, the point cloud cluster with the lowest score is considered the optimal cluster.

[0141] Specifically, the three-dimensional point cloud of the third region within the three-dimensional detection area of ​​the suspended object is clustered and segmented to obtain all point cloud clusters. The point cloud clusters are scored using a scoring function, and the unique point cloud cluster with the lowest score is selected. Specifically, the clustering segmentation algorithm in the PCL library is used to obtain the 3D point cloud of the third region. This is achieved by constructing a Kd-Tree on the 3D point cloud of the third region and then calling PCL Euclidean clustering. The clustering tolerance and minimum / maximum cluster size are configurable parameters, which are calibrated and adjusted according to site conditions (such as beam spacing, load size range, and point cloud density). In this embodiment, the tolerance is approximately 1.3m, the minimum is 40 points, and the maximum is 10,000 points.

[0142] A scoring algorithm is then applied to the clustered point cloud clusters. The algorithm first eliminates point cloud clusters whose minimum 3D size of the axis-aligned bounding box is smaller than a specified size. This easily removes point clouds of the lifting ropes identified around the actual suspended object, preventing them from affecting the recognition results. The scoring function then calculates the Euclidean distance from the center point of the axis-aligned bounding box of all remaining point cloud clusters to the center point of the upper surface of the 3D detection area of ​​the suspended object (i.e., the third region), the Euclidean distance between the axis-aligned bounding boxes of all point cloud clusters and the center point of the axis-aligned bounding box of the previously identified target suspended object, and the 3D size difference of the axis-aligned bounding boxes. Based on pre-set weights, a score is calculated for each point cloud cluster, and the point cloud cluster with the lowest score is finally selected as the unique target point cloud cluster.

[0143] In this embodiment, the system can store the three-dimensional coordinates and three-dimensional dimensions of the center point of the axis-aligned bounding box of each identified target load in the global coordinate system, and the information of the previously identified target load comes from this stored data.

[0144] S4: Based on the optimal cloud cluster, obtain the precise hoisting axis alignment bounding box and determine the precise dimensions of the hoisting axis alignment bounding box; Specifically, in this embodiment, the point cloud above the optimal point cloud cluster with multiple suspension ropes is removed by a fixed percentage cutting method to obtain a precise load axis alignment bounding box, and the precise size of the load axis alignment bounding box is calculated. Specifically, since the optimal point cloud cluster usually contains the lifting ropes directly connected to the load, this embodiment will remove the point cloud with the highest height from the unique target point cloud cluster according to a pre-set percentage rejection value. This percentage value is determined based on the actual project situation, generally ranging from 5% to 20%. This removes some meaningless lifting rope point clouds from the optimal point cloud cluster, and the remaining point clouds constitute the current target load point cloud cluster, i.e., the precise load point cloud cluster. Then, the precise load axis alignment bounding box, the three-dimensional coordinates of the bounding box's center point, and the three-dimensional dimensions of the bounding box are calculated.

[0145] The calculation of the axis-aligned bounding box (AABB), center, and 3D dimensions of the precise point cloud cluster D of the suspended load based on PCL is as follows: Let the point cloud cluster be:

[0146] In the formula: For the precise location of the load in the point cloud cluster The three-dimensional coordinates of a point in the global coordinate system; For precise indexing of points within a point cloud cluster of suspended loads; The total number of points in the precise point cloud cluster of the suspended load; For the precise location of the load in the point cloud cluster The X-coordinate of a point in the global coordinate system; For the precise location of the load in the point cloud cluster The Y-coordinate of a point in the global coordinate system; For the precise location of the load in the point cloud cluster The Z-axis coordinates of each point in the global coordinate system; D represents the precise cloud of suspended object points; Extreme value scanning (using the PCL function pcl::getMinMax3D to find the extreme values ​​of the following formula):

[0147]

[0148]

[0149] In the formula: To accurately align the load axis bounding box (AABB) to the minimum X-axis boundary in the global coordinate system; To accurately align the load axis bounding box (AABB) to the maximum X-axis boundary in the global coordinate system; To accurately align the load axis bounding box (AABB) to the minimum Y-axis boundary in the global coordinate system; To accurately align the load axis bounding box (AABB) to the maximum Y-axis boundary in the global coordinate system; To accurately align the load axis bounding box (AABB) to the minimum Z-axis boundary in the global coordinate system; To accurately align the load axis bounding box (AABB) to the maximum Z-axis boundary in the global coordinate system; 2) Precise definition of the load axis alignment bounding box (AABB):

[0150] Among the symbols It represents the superposition of the ranges of different axial regions in the global coordinate system. This indicates that the precisely aligned load axis is within the enclosure.

[0151] 3) Precise alignment of the load axis with the center of the enclosure box (3D coordinates):

[0152] in, To ensure precise alignment of the load shaft with the center of the enclosure; 4) Precise alignment of the load axis with the three-dimensional dimensions (length, width, height) of the enclosure box:

[0153] S5: Based on the precise sling axis alignment bounding box, determine the collision avoidance detection area bounding box, and obtain the three-dimensional point cloud set between the precise sling axis alignment bounding box and the collision avoidance detection area bounding box according to the three-dimensional point cloud of the first area, namely the four-dimensional point cloud of the fourth area, and perform collision avoidance detection according to the three-dimensional point cloud of the fourth area.

[0154] S51: Based on the precise load axis alignment bounding box and the anti-collision detection area bounding box, determine the detection area set between the precise load axis alignment bounding box and the anti-collision detection area bounding box, so as to determine the first detection area and the second detection area; wherein, the first detection area and the second detection area are both sub-areas of the detection area.

[0155] The first detection area includes: Each of the b-th side of the enclosure is precisely aligned with the axis of the suspended object as the bottom surface, b=1,2,3,4, and the top surface is set on the four cuboids on the side of the enclosure corresponding to the anti-collision detection area. The bottom surface of the enclosure is precisely aligned with the axis of the suspended object, and the top surface is set on the fifth cuboid on the bottom surface of the enclosure in the collision detection area. The second detection area is the area outside the first detection area within the fourth area between the precise load shaft alignment enclosure and the anti-collision detection area enclosure; S52: The method used to determine whether there is a collision risk in the first detection area is as follows: If a point in the fourth region of the 3D point cloud within the b-th cuboid, where b=1,2,3,4, has a distance less than a set distance threshold between itself and the b-th side of the bounding box precisely aligned with the axis of the suspended object, then there is a risk of collision. If a point in the 3D point cloud of the fourth region within the 5th cuboid has a distance less than the set distance threshold between itself and the bottom surface of the bounding box that is precisely aligned with the axis of the suspended object, then there is a risk of collision. S53: The method used to determine whether there is a collision risk in the second detection area is as follows: Obtain a line segment with endpoints of a point in the 3D point cloud of the fourth region within the second detection area and the center point of the precise sling shaft alignment bounding box. Obtain the distance between the intersection of the line segment and the precise sling shaft alignment bounding box and the point in the 3D point cloud of the fourth region within the second detection area. If the distance is less than a set distance threshold, there is a risk. S54: If there is a collision risk in either the first detection area or the second detection area, then the crane load is at risk of collision.

[0156] Specifically, based on the precise 3D coordinates and dimensions of the center point of the hoisted object's axis-aligned bounding box, the size of the current hoisted object's axis-aligned bounding box is expanded outwards in the five directions (-X, +X, -Y, +Y, -Z) of the gantry crane coordinate system according to the set anti-collision detection area length value, thus obtaining the anti-collision detection area bounding box. This anti-collision detection area length value is manually assigned and related to the actual project requirements. The first region's 3D point cloud is then cut using the anti-collision detection area bounding box and the current hoisted object's axis-aligned bounding box to obtain all 3D point clouds located between the two axis-aligned bounding boxes, i.e., the fourth region's 3D point cloud.

[0157] Finally, based on Figure 6 and Figure 7 The collision avoidance detection method in the model calculates the minimum distance from the axis-aligned bounding box of all 3D point clouds in the fourth region to the current load for safety identification.

[0158] Specifically, the collision avoidance detection method is divided into two cases: axial detection and spherical supplementary detection. The collision avoidance detection method first performs axial detection, and then performs spherical supplementary detection.

[0159] 1) Axial inspection results as follows Figure 6 As shown, Figure 6 The areas marked in black are the first detection areas, and the areas not marked in black are the second detection areas.

[0160] Define the nearest surface of the load's axis-aligned bounding box in the gantry crane coordinate system in the five directions of -X, +X, -Y, +Y, -Z as the nearest surface of the load's bounding box perpendicular to that direction. Figure 6 As shown in the diagram, the corresponding side surface in the -X direction is the nearest surface in the -X direction. Based on the corresponding surface in each direction as the base and with a pre-given anti-collision detection area length value as the height, a cuboid space can be constructed. The set of five cuboid spaces in the five directions (-X, +X, -Y, +Y, -Z) constitutes the first detection area. This area is a subset of the anti-collision detection area.

[0161] For all point clouds within the first detection area, the collision avoidance detection method will perform nearest-distance detection. For example, in the -X direction, it will detect the distance of all point clouds within the corresponding cuboid space in the -X direction to the corresponding surface in the X direction. ,in This represents the X-axis coordinate of the point cloud within any corresponding cuboid space. This represents the X-coordinate of the corresponding plane (this coordinate changes with different planes, for example, in the -Y direction). This represents the Y-axis coordinate of the corresponding plane, and so on. The distance to each point cloud is obtained... Then, by calculating the minimum value in this set, the nearest distance between all point clouds within the first detection area and the bounding box of the suspended object can be obtained. If the nearest distance is less than a predefined threshold, a collision risk is considered to exist.

[0162] 2) Spherical supplementary testing status as follows Figure 7 As shown.

[0163] Define the difference between the collision avoidance detection region and the first detection region as the second detection region. Within this region, the nearest distance for all point clouds cannot be directly calculated using the method described in the axial detection case above; instead, spatial distance needs to be calculated, such as... Figure 7 As shown. Taking any spatial point A as an example, the straight line segment from it to the center point B of the suspended object enclosure is... The straight line segment Two line segments obtained by cutting through the enclosure of the suspended load. These are straight line segments inside the enclosure. and external straight line segments This supplementary spherical test will take straight line segments. The length is the spatial distance. By calculating the spatial distances of all point clouds within the second detection area, the minimum value in this set is taken as the nearest spatial distance. If the nearest spatial distance is less than a predefined threshold, a collision risk is considered present.

[0164] If either the axial detection scenario or the spherical supplementary detection scenario presents a collision risk, the collision avoidance detection method will consider it an overall collision risk. In this case, an audible and visual alarm will be issued to the driver.

[0165] Specifically, for the loads lifted by shipbuilding gantry cranes, the anti-collision distance in this embodiment is a directional anti-collision measure for precise alignment of the load's axis with the surrounding box. The direction includes the sides of the surrounding box in the +X, -X, +Y, -Y, and -Z axial directions, calculated based on the nearest axial distance from the obstacle point cloud within the anti-collision range to each of the sides. For example... Figure 6 As shown. Furthermore, for all spaces below the top surface of the hoisting enclosure (excluding the five areas mentioned above), the collision avoidance distance is calculated as the straight-line distance from each point cloud (spatial point 1) to the center point of the hoisting enclosure, obtained by cutting off the intersection point from a side of the enclosure, and then to that spatial point. For example... Figure 7 As shown.

[0166] Specifically, in this embodiment, an industrial control computer with sufficient computing power is deployed on the vehicle side to collect real-time PLC data and LiDAR point cloud data. The PLC data is directly read from the PLC data processed by the CPE via the S7 protocol. The LiDAR data is used to acquire 3D point clouds based on the driver of different LiDAR models. The PLC data on the gantry crane includes essential components: the rotation angle of the rotating platform, the boom luffing, the main hook lifting height, the main hook load capacity, the auxiliary hook lifting height, and the auxiliary hook load capacity. The PLC data on the gantry crane also includes essential components: the upper trolley position, the lower trolley position, the main hook lifting height, the main hook load capacity, the auxiliary hook lifting height, the auxiliary hook load capacity, the hook lifting height, and the hook load capacity. Optional components include: trolley gear position, upper trolley gear position, lower trolley gear position, main hook gear position, auxiliary hook gear position, and hook gear position. The S7 protocol is a proprietary communication protocol developed by Siemens for its SIMATIC S7 series programmable logic controllers (PLCs). It is mainly used for data exchange and control between PLCs and HMIs (human-machine interfaces), programming software (such as TIA Portal and STEP 7), SCADA systems, and other intelligent devices.

[0167] Specifically, the point cloud data of the lidar is consistent for both gantry and portal-mounted systems. It consists of real-time 3D point cloud data obtained by two lidars scanning from different perspectives, with only the installation location and lidar model being different.

[0168] Specifically, this embodiment accesses data via an industrial control computer: First, based on the Robot Operation System (ROS), the data is processed and converted into real-time data in the distributed system across multiple nodes. This includes the three-dimensional coordinates of all hooks under different kinematic models of the shipbuilding gantry crane (gantry type and portal type) within the shipbuilding gantry crane body, Boolean values ​​indicating whether each hook is loaded or not, and merged three-dimensional point cloud data in the shipbuilding gantry crane coordinate system. Based on the three-dimensional coordinates of all hooks, the Boolean values ​​of hook load, and the merged three-dimensional point cloud data, the real-time three-dimensional point cloud of the shipbuilding gantry crane's load, the precise load axis alignment bounding box, and the point cloud of obstacles surrounding the load are finally segmented. The obstacle point cloud can be further divided into two categories based on requirements: obstacles within the ship hull and obstacles outside the gantry crane.

[0169] This embodiment can transmit information such as the center coordinates of the hoisted object's axis aligned with the bounding box and whether the hoisted object collides with obstacles back to the digital twin platform based on the TCP protocol, as needed.

[0170] The lidar installation scheme in this embodiment is as follows: 1) LiDAR installation scheme for gantry cranes: Because gantry cranes have two-dimensional rotational motion, in addition to collision protection for the load 2, the boom 1 of the crane itself also requires a large collision protection range. Therefore, gantry cranes need to be equipped with two wide-angle lidars, namely the first lidar 31 and the second lidar 32. Figure 3 As shown, the two lidar sensors are vertically mounted, each scanning a 75° area horizontally. During installation, a certain degree of overlap is necessary to facilitate mutual position calibration. The lateral viewing angle is as follows. Figure 2 As shown, each lidar scans a 120° area in the vertical direction. During installation, it is necessary to ensure that the lower edge of the scanning range of each lidar exactly covers the area below the gantry crane. 2) Gantry Crane LiDAR Installation Scheme: Gantry cranes only have translational movement and only need to prevent collisions between the load and surrounding obstacles. To accurately obtain the three-dimensional dimensions of the load and information about surrounding obstacles, two lidars installed vertically on opposite sides are necessary. Figure 4 The diagram shows the installation in the forward view. Two lidars are located below the crossbeams on the rigid leg side and the flexible leg side, respectively. Each lidar scans a 120° area in the vertical direction. Figure 5 This is a top-down view, showing that each lidar scans a region within a 25.4° range in the horizontal direction.

[0171] The radar deployment scheme in this embodiment solves the problems of traditional infrared or ultrasonic-based schemes, which have limited sensor detection range, fixed installation positions, and can only monitor specific directions and local areas, failing to achieve full coverage of the gantry crane's operating space. This embodiment is not affected by factors such as ambient light, temperature, and surface material, has high detection accuracy, and can adapt to the changing operating environment of shipyards.

[0172] Meanwhile, this embodiment addresses the problems of traditional tower crane safety monitoring schemes, which install lidar directly above the trolley. These schemes primarily target the vertical lifting and slewing motions of the tower crane. However, for large gantry cranes such as shipbuilding gantry cranes, the large height and lateral dimensions of the loads mean that sensors installed only on the top of the trolley cannot cover the sides of the load and the area near the ground, resulting in blind spots. Furthermore, the spatial perception models and control logic are typically designed for tower crane structures and cannot be directly adapted to the operating modes of gantry cranes. In this embodiment, the lidar installation scheme addresses these issues: Wide adaptability to different types of gantry cranes: Based on the dual LiDAR solution, different types of gantry cranes only need to be modified in terms of installation location to ensure that the identification area of ​​the suspended object is not obstructed, and the same algorithm can be used to identify the suspended object and perform collision avoidance.

[0173] The solution is highly adaptable to different types of suspended objects: as long as the suspended object has a certain reflectivity that can be scanned by the lidar, regardless of its structural complexity or type, its size can be accurately identified and collision avoidance can be implemented.

[0174] The solution reduces the reliance on the accuracy of PLC and encoder sensors: In this embodiment of the gantry crane solution, the results obtained by high-precision laser radar scanning are used to perform secondary calibration of the hook position, which effectively solves the problems of encoder numerical drift and inaccuracy, and achieves centimeter-level positioning accuracy in the horizontal direction.

[0175] The solution does not require dataset support: This embodiment performs collision avoidance detection, and for different types of scenarios, there is no need to train a model (such as hook position recognition or specific load type recognition), and it can be deployed directly.

[0176] In summary, the layout scheme of this embodiment can meet the requirements of high precision, full coverage, and real-time collision avoidance and size recognition in dealing with the complex and diverse lifting objects in shipyards, full-space monitoring of large lifting loads, and the special structural adaptability of gantry cranes.

[0177] Application Example 1: Collision prevention between loads lifted from below a gantry crane and surrounding low-lying objects during shipbuilding: Scenario and Objective: When using a gantry crane for lifting operations in a shipyard, the load needs to be moved from the ground to above or around the hull. Due to the low decks or segmented structures of the hull during construction, the load is prone to collisions during translation or descent. The objective of this application example is to monitor the spatial distance between the load and surrounding low hull structures and other obstacles in real time to ensure collisions are avoided during lifting.

[0178] Deployment and Parameters: Two additional wide-angle LiDARs are installed above the gantry crane cab 4. An industrial control computer is set up in the cab to receive, process, and transmit data. The solution will collect point cloud data, mechanism status in the PLC and encoder, lifting height, and load weight in real time. Collision avoidance threshold is 5 meters.

[0179] Operation Process: During the hoisting process, the system first combines PLC data and point cloud data to generate a real-time 3D bounding box for the hoisted object, and continuously tracks its position and calculates the distance to the nearest obstacle point cloud in each direction. When the hoisted object is near the ship hull, if the distance in any collision avoidance direction is less than the collision avoidance threshold, an alarm will be triggered to remind the driver to drive carefully. Finally, when the hoisted object reaches above the target position and is slowly lowered, if the minimum distance directly below it is less than the collision avoidance threshold, an alarm will also be triggered to remind the driver to drive carefully. This continues until the hoisted object lands safely.

[0180] Specifically, through this application example, the system can effectively prevent collisions between the hoisted object and the ship's structure and surrounding potential obstacles during the hoisting process, avoiding damage to the hoisted object or the ship's hull. This method achieves centimeter-level real-time monitoring, improves the safety and reliability of hoisting operations, and provides a reusable solution for safety management in complex shipbuilding scenarios.

[0181] Application Example 2: Collision prevention between loads lifted from below a gantry crane and surrounding objects of greater height during shipbuilding: Scenario and Objective: When using gantry cranes for lifting operations in shipyards, if the hull is taller than the bridge, the load needs to cross or approach the higher hull area. In this situation, not only is the load itself at risk of collision, but the crane boom can also easily interfere with the hull structure during swinging or luffing. The objective of this application example is to simultaneously detect the distances of the load and the crane boom in three-dimensional space to the surrounding taller hull structures, ensuring all-around collision avoidance in complex lifting paths.

[0182] Deployment and Parameters: Two wide-angle LiDARs are installed above the gantry crane's cab to cover the load and the side space of the hull. The industrial control computer receives and processes point cloud and PLC data in real time, including hook position, boom angle, luffing status, and load information. In addition to calculating the load bounding box, the system also establishes the boom's directional bounding box and updates its position in real time. The collision avoidance threshold is set to 5 meters.

[0183] Operation Process: During the lifting process, the system generates a bounding box for the lifted object and a bounding box for the crane boom orientation based on PLC parameters and point cloud data, and continuously tracks the spatial relationship between these two bounding boxes and the ship's point cloud. When the lifted object moves close to the higher part of the ship, the system calculates the closest distance between the bounding box and the ship's point cloud; if the minimum distance between the lifted object or any part of the crane boom and the ship's hull is less than a set threshold, an alarm is immediately triggered to remind the operator to adjust the operation. Throughout the lifting process, the system synchronously monitors the spatial position of the crane boom and the lifted object until the lifted object safely passes through or reaches the target area.

[0184] Specifically, through this application example, the system can not only prevent the load from colliding with the elevated hull during movement, but also monitor the safety space of the boom in real time during large-angle luffing or rotation, avoiding accidents caused by interference between the boom and the hull. This method achieves joint monitoring of the boom and the load, further improving the safety and robustness of lifting operations and providing reliable assurance for adapting to various ship heights and complex environments.

[0185] Application Example 3: Collision prevention between the load being lifted by the gantry crane and the ship hull below during the lifting process: Scenario and Objective: When using gantry cranes for lifting operations in shipyards, the loads are typically larger and heavier, and need to be placed directly in designated locations on the ship's hull. In such scenarios, the safe distance between the load and the ship under construction below and around it is crucial; a collision could result in serious damage to the load or the ship. The objective of this application example is to detect the minimum distance between the load and the ship under construction in real time to ensure a smooth and safe placement during lifting.

[0186] Deployment and Parameters: One LiDAR is installed under each of the rigid and flexible legs of the gantry crane's crossbeam, providing full coverage of the load. The industrial control computer collects point cloud data and PLC parameters in real time, including the positions of the upper and lower trolleys and the hook lifting height. Because the LiDARs are positioned on opposite sides, they effectively reduce obstruction of the scan by the load and can even acquire point cloud information below the load. The collision avoidance threshold is also set to 5 meters.

[0187] Operation Process: During the lifting process, the system generates a real-time 3D bounding box for the lifted object based on PLC data and point cloud data, and continuously calculates the minimum distance from the bounding box to the point cloud of surrounding obstacles in each collision avoidance direction. As the lifted object gradually moves near the ship hull, the system can utilize the perspective advantage of dual LiDAR to acquire point cloud information below the lifted object, thereby ensuring safety in all directions, especially in the downward direction. When the minimum distance in any direction is less than the collision avoidance threshold, the system will trigger an alarm, reminding the operator to make operational adjustments, ultimately ensuring the smooth and safe placement of the lifted object.

[0188] Specifically, through this application example, the system can effectively prevent collisions between gantry cranes lifting large, heavy loads and the ship hull below. The installation scheme significantly reduces point cloud occlusion and can scan the bottom area of ​​the load, thus achieving higher-precision real-time monitoring. Ultimately, this method ensures the safety and reliability of large-scale lifting operations, providing a replicable solution for collision avoidance in high-risk shipyard scenarios. It is particularly suitable for port and shipbuilding applications, capable of recognizing the 3D size of loads during construction and detecting collisions between the shipbuilding gantry crane body, the load, and surrounding potential obstacles. It can achieve collision avoidance safety functions for two different configurations of shipbuilding gantry cranes during operation. Furthermore, the recognition and collision avoidance results can be combined with a digital twin platform to achieve higher-dimensional safety assurance for shipbuilding construction operations.

[0189] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A crane load collision avoidance detection method based on real-time 3D point cloud driving, characterized in that, Includes the following steps: S1: Obtain the 3D point cloud of the first area within the working area of ​​the shipbuilding gantry crane and the 3D position coordinates of the hook of the shipbuilding gantry crane; wherein, the shipbuilding gantry crane is a gantry crane or a portal crane. S2: Based on the three-dimensional position coordinates of the hook, determine the three-dimensional detection area of ​​the load to obtain the three-dimensional point cloud within the three-dimensional detection area of ​​the load, i.e., the three-dimensional point cloud of the third region. S3: Based on the three-dimensional point cloud of the third region within the three-dimensional detection area of ​​the suspended object, multiple point cloud clusters are obtained based on a clustering algorithm. The scores of the multiple point cloud clusters are obtained, and then the point cloud cluster with the lowest score is obtained. Finally, the optimal point cloud cluster is obtained. S4: Based on the optimal cloud cluster, obtain the precise load shaft alignment bounding box to determine the precise size of the load shaft alignment bounding box; S5: Based on the precise dimensions of the sling shaft alignment bounding box, determine the collision avoidance detection area bounding box, and obtain the three-dimensional point cloud set between the precise sling shaft alignment bounding box and the collision avoidance detection area bounding box according to the three-dimensional point cloud of the first area, i.e., the three-dimensional point cloud of the fourth area, and perform collision avoidance detection according to the three-dimensional point cloud of the fourth area.

2. The crane load collision avoidance detection method based on real-time 3D point cloud driving according to claim 1, characterized in that, When the shipbuilding gantry crane is a gantry crane, the method for obtaining the three-dimensional position coordinates of the hook of the shipbuilding gantry crane is as follows: S11: The theoretical three-dimensional position coordinates of the hook are obtained as follows: In the formula: This represents the theoretical position coordinates of the hook in the X direction; This represents the theoretical position coordinates of the hook in the Y direction; This represents the theoretical position coordinates of the hook in the Z direction; This represents the X-axis coordinate of the origin of the trolley's coordinate system; This represents the Y-axis coordinate of the origin of the trolley's coordinate system; Indicates the real-time height of the hook bottom; This indicates the real-time position of the trolley in the X direction, that is, the real-time position of the trolley along the trolley track. This indicates the offset of the hook in the X direction; This indicates the offset of the hook in the Y direction; S12: Based on the XY plane where the maximum coordinate of the hook top in the Z direction is located, the three-dimensional point cloud of the first region is segmented to obtain the three-dimensional point cloud of the second region above the XY plane where the maximum coordinate of the hook top in the Z direction is located. S13: Cluster the 3D point cloud of the second region to obtain M rope-related point cloud clusters, and obtain the cluster center coordinates of the m-th rope-related point cloud cluster. Where m is the index of the suspending tether point cloud cluster; M is the total number of suspending tether point cloud clusters; These are the coordinates of the cluster center of the m-th suspending rope point cloud cluster in the X, Y, and Z directions, respectively. S14: When the number of cloud clusters on the lifting rope M is less than the number of hooks N: S141: Obtain the distance between the coordinates of the cluster center of the first sling point cloud cluster in the XY plane and the coordinates of the theoretical 3D positions of all hooks in the XY plane. ; S142: When When the minimum value is reached, the final three-dimensional position coordinates of the nth hook are obtained as follows: In the formula: These are the three-dimensional position coordinates of the nth hook in the X, Y, and Z directions, respectively. These are the coordinates of the first suspending rope point cloud cluster in the X and Y directions, respectively; This represents the theoretical position coordinates of the nth hook in the Z direction; S143: Based on the 2nd and 3rd sequentially For M suspending rope point cloud clusters, repeat S141-S142; S144: At this point, the final three-dimensional position coordinates of the remaining NM hooks are as follows: ； S15: When the number of cloud clusters on the lifting rope M is greater than or equal to the number of hooks N: Obtain the distance between the theoretical 3D position of the nth hook in the XY plane and the coordinates of the cluster center of all the rope point cloud clusters in the XY plane. ; when When the minimum value is reached, the final three-dimensional position coordinates of the nth hook are as follows: 。 3. The crane load collision avoidance detection method based on real-time 3D point cloud driving according to claim 1, characterized in that, When the shipbuilding gantry crane is a portal crane, the method for obtaining the three-dimensional position coordinates of the hook is as follows; S111: Obtain the position coordinates of the crane boom after rotation relative to the cab: In the formula: These are the X-coordinates and Y-coordinates of the position of the crane boom relative to the cab after rotation. Indicates transpose; These are the X-coordinates and Y-coordinates of the initial position of the crane boom relative to the cab, respectively. This is the two-dimensional rotation matrix of the crane about the Z-axis; The two-dimensional rotation angle of the cockpit around the Z-axis; S112: Obtain the increase in end displacement of the hook in the XY plane along the extension direction of the crane boom; In the formula: This indicates the increase in end displacement of the hook in the XY plane along the extension direction of the crane boom; This represents the two-dimensional rotation angle of the cockpit around the Z-axis; Indicates the pitch angle of the crane boom; Indicates the length of the crane boom; express The component in the X direction; express The component in the Y direction; S113: Obtain the three-dimensional position coordinates of the hook as follows: In the formula: The X-axis position coordinates of the cockpit relative to the origin of the gantry crane coordinate system; The Y-axis position coordinates of the cockpit relative to the origin of the gantry crane coordinate system; Expanded into an explicit formula, it is expressed as follows: In the formula: This indicates the position coordinates of the hook in the X direction; This indicates the position coordinates of the hook in the Y direction; This indicates the position coordinates of the hook in the Z direction.

4. The crane load collision avoidance detection method based on real-time 3D point cloud driving according to claim 1, characterized in that, When the shipbuilding gantry crane is a portal crane, the three-dimensional detection area of ​​the hoisted object is obtained as follows: In the formula: This is the three-dimensional detection area for the suspended load. The coordinates of any point in the three-dimensional detection area of ​​the suspended object in the global coordinate system are defined in the X direction. The coordinates of any point in the 3D detection area of ​​the suspended object in the global coordinate system are in the Y direction. The coordinates of any point in the three-dimensional detection area of ​​the suspended object in the global coordinate system are in the Z direction. It is a three-dimensional space; This indicates the position coordinates of the hook in the X direction; This indicates the position coordinates of the hook in the Y direction; This indicates the position coordinates of the hook in the Z direction; This represents the ground height value along the Z-axis in the global coordinate system. The ground clearance height value; Indicates transpose; in, In the formula: This represents the actual half-expansion in the X direction; This represents the actual half-expansion in the Y direction; The scaling function varies piecewise with weight; The unscaled baseline half-expansion in the X direction; The unscaled baseline half-expansion in the Y direction; In the formula: This represents the current weight of the load being lifted. This is the scaling factor; This is the scaling limit value.

5. The crane load collision avoidance detection method based on real-time three-dimensional point cloud driving according to claim 4, characterized in that, When the shipbuilding gantry crane is a gantry crane, the three-dimensional detection area of ​​the hoisted object is: When the number of hooks is 1, the three-dimensional detection area of ​​the load is: In the formula: The unfolding distance of the three-dimensional detection area of ​​the suspended object set in the X direction; The unfolding distance for the three-dimensional detection area of ​​the suspended object set in the Y direction; The unfolding distance of the three-dimensional detection area of ​​the suspended object set in the Z direction; When the number of hooks is 2, the three-dimensional detection area of ​​the load is: In the formula: Here are the position coordinates of the first hook in the X direction; Here are the position coordinates of the second hook in the X direction; Here are the coordinates of the first hook in the Y direction; Here are the position coordinates of the second hook in the Y direction; Here are the coordinates of the first hook in the Z direction; Here are the coordinates of the second hook in the Z direction; This represents the maximum value of the Z-axis coordinates of the first and second hooks; When the number of hooks is greater than 2, the method for obtaining the three-dimensional detection area of ​​the load is as follows: First, obtain the smallest rectangular region of the hook in the XY plane of the global coordinate system: in: In the formula: For the hook index; The total number of hooks in the load; For the first The X-axis coordinate of a hook in the global coordinate system; For the first The Y-coordinate of a hook in the global coordinate system; The smallest rectangular frame in the XY plane; The minimum X-axis coordinate of the hook in all loads in the global coordinate system; The maximum X-axis coordinate of the hook in the global coordinate system for all loads; The minimum Y-axis coordinate of the hook in all loads in the global coordinate system; This represents the maximum Y-axis coordinate of the hook in the global coordinate system for all loads. Indicates transpose; It is a two-dimensional space; Then, the three-dimensional detection area of ​​the suspended object at this time is obtained as follows: in, In the formula: This represents the maximum Z-axis coordinate of the hook in the global coordinate system for all loads.

6. The crane load collision avoidance detection method based on real-time 3D point cloud driving according to claim 1, characterized in that, The method used to obtain the score of the point cloud cluster is as follows: S31: Based on the three-dimensional point cloud of the third region within the three-dimensional detection area of ​​the suspended object, multiple point cloud clusters are obtained using a clustering algorithm to obtain the position of the center point of the point cloud cluster and the size of the point cloud cluster; The position of the center point of the point cloud cluster is obtained as follows: In the formula: The coordinates of the center point of the point cloud cluster; These are the coordinates of the current point cloud cluster's center point in the X, Y, and Z directions, respectively. These are the minimum and maximum values ​​of the X-direction coordinates of the axis-aligned bounding box of the point cloud cluster, respectively; These are the minimum and maximum values ​​of the Y-direction coordinates of the axis-aligned bounding box of the point cloud cluster, respectively; These are the minimum and maximum values ​​of the Z-direction coordinates enclosed by the axis alignment of the point cloud cluster, respectively; The dimensions of the point cloud cluster are obtained as follows: In the formula: The dimension of the axis-aligned bounding box of the point cloud cluster at the current moment in the X direction, which is the width of the point cloud cluster at the current moment; The dimension of the axis-aligned bounding box of the point cloud cluster at the current moment in the Y direction, i.e., the length of the point cloud cluster at the current moment; The dimension of the axis-aligned bounding box of the point cloud cluster at the current moment in the Z direction is the height of the point cloud cluster at the current moment; S32: Obtain the Euclidean distance between the current position of the point cloud cluster center point and the previous position of the hoisted object center point; obtain the normalized upper bound of the Euclidean distance between the axis-aligned bounding box of the point cloud cluster and the center point of the axis-aligned bounding box of the previously identified target hoisted object. ; The formula used to obtain the Euclidean distance between the current position of the cloud cluster center point and the previous position of the hoisted object is as follows: In the formula: distance is the Euclidean distance between the axis-aligned bounding box of the point cloud cluster and the center point of the axis-aligned bounding box of the previously identified target load, that is, the Euclidean distance between the current center point of the point cloud cluster and the center point of the previously identified load. Represents the square root; , , These are the X, Y, and Z coordinates of the center point of the current point cloud cluster, respectively. , , These are the X, Y, and Z coordinates of the center point of the target hoisting object's bounding box at the previous moment; The formula used to obtain the normalized upper bound of the Euclidean distance between the axis-aligned bounding box of the point cloud cluster and the center point of the axis-aligned bounding box of the previously identified target sling is as follows: In the formula: The normalized upper bound of the Euclidean distance between the axis-aligned bounding box of the point cloud cluster and the center point of the axis-aligned bounding box of the previously identified target sling; For the first The Euclidean distance between the axis-aligned bounding box of a point cloud cluster and the center point of the axis-aligned bounding box of the previously identified target load; For indexes of point cloud clusters; The total number of point cloud clusters; S33: Obtain the 3D size difference between the axis-aligned bounding box of the current point cloud cluster and the axis-aligned bounding box of the target hoist at the previous time step: Obtain the normalized upper bound of the 3D size difference between the current point cloud cluster and the axis-aligned bounding box of the target hoist at the previous time step. ; The formula used to obtain the 3D size difference between the axis-aligned bounding box of the current point cloud cluster and the axis-aligned bounding box of the target hoisting object at the previous moment is as follows: In the formula: This is the difference between the current 3D size of the point cloud cluster and the 3D size of the axis-aligned bounding box of the target hoisting object at the previous moment; These are the width, length, and height of the axis-aligned bounding box of the target hoisting object at the previous moment, respectively. The formula used to obtain the normalized upper bound of the difference between the 3D size of the current point cloud cluster and the 3D size of the axis-aligned bounding box of the target hoisting object at the previous moment is as follows: In the formula: This is the normalized upper bound of the difference between the 3D size of the current point cloud cluster and the 3D size of the axis-aligned bounding box of the target load at the previous moment; S34: Obtain the Euclidean distance from the center point of the axis-aligned bounding box of the point cloud cluster to the center point of the upper surface of the 3D detection area of ​​the suspended object: Obtain the normalized upper bound of the Euclidean distance from the center point of the axis-aligned bounding box of the point cloud cluster to the center point of the upper surface of the 3D detection area of ​​the suspended object. ; The formula used to obtain the Euclidean distance from the center point of the axis-aligned bounding box of the point cloud cluster to the center point of the upper surface of the 3D detection area of ​​the suspended object is as follows: In the formula: The Euclidean distance from the center point of the axis-aligned bounding box of the point cloud cluster to the center point of the upper surface of the 3D detection area of ​​the suspended object; , , These are the X, Y, and Z coordinates of the center point of the upper surface of the three-dimensional detection area of ​​the hoisted object at the current moment; The formula used to obtain the normalized upper bound of the Euclidean distance from the center point of the axis-aligned bounding box of the point cloud cluster to the center point of the upper surface of the 3D detection area of ​​the suspended object is as follows: In the formula: The normalized upper bound of the Euclidean distance from the center point of the axis-aligned bounding box of the point cloud cluster to the center point of the upper surface of the 3D detection area of ​​the suspended object; S35: Obtain the Euclidean distance between the axis-aligned bounding box of the normalized point cloud cluster and the center point of the axis-aligned bounding box of the previously identified target load, the 3D size difference between the axis-aligned bounding box of the normalized point cloud cluster and the axis-aligned bounding box of the target load at the previous moment, and the Euclidean distance from the center point of the axis-aligned bounding box of the normalized point cloud cluster to the center point of the upper surface of the 3D detection area of ​​the load. The formula used is as follows: In the formula: The Euclidean distance between the axis-aligned bounding box of the normalized point cloud cluster and the center point of the axis-aligned bounding box of the previously identified target sling; The difference in three-dimensional dimensions between the axis-aligned bounding box of the normalized point cloud cluster and the axis-aligned bounding box of the target hoist at the previous moment. The Euclidean distance from the center point of the axis-aligned bounding box of the normalized point cloud cluster to the center point of the upper surface of the 3D detection area of ​​the suspended object; This is a parameter used to ensure that the denominator is not zero; S36: When the size of a point cloud cluster is greater than a size threshold, a score for the point cloud cluster is obtained using the following formula: In the formula: They are respectively The weights.

7. The crane load collision avoidance detection method based on real-time 3D point cloud driving according to claim 1, characterized in that, The center of the collision avoidance detection area enclosure coincides with the center and centerline of the precise load shaft alignment enclosure, and the size of the collision avoidance detection area enclosure is larger than the size of the precise load shaft alignment enclosure.

8. The crane load collision avoidance detection method based on real-time three-dimensional point cloud driving according to claim 7, characterized in that, S5 includes: S51: Based on the precise load axis alignment bounding box and the anti-collision detection area bounding box, determine the detection area set between the precise load axis alignment bounding box and the anti-collision detection area bounding box, so as to determine the first detection area and the second detection area. The first detection area includes: Each of the b-th side of the enclosure is precisely aligned with the axis of the suspended object as the bottom surface, b=1,2,3,4, and the top surface is set on the four cuboids on the side of the enclosure corresponding to the anti-collision detection area. The bottom surface of the enclosure is precisely aligned with the axis of the suspended object, and the top surface is set on the fifth cuboid on the bottom surface of the enclosure in the collision detection area. The second detection area is the area outside the first detection area, which is between the precise load shaft alignment enclosure and the anti-collision detection area enclosure. S52: The method used to determine whether there is a collision risk in the first detection area is as follows: If a point in the fourth region of the 3D point cloud within the b-th cuboid, where b=1,2,3,4, has a distance less than a set distance threshold between itself and the b-th side of the bounding box precisely aligned with the axis of the suspended object, then there is a risk of collision. If a point in the 3D point cloud of the fourth region within the fifth cuboid has a distance less than a set distance threshold between itself and the bottom surface of the bounding box that is precisely aligned with the axis of the suspended object, then there is a risk of collision. S53: The method used to determine whether there is a collision risk in the second detection area is as follows: Obtain a line segment with endpoints of a point in the 3D point cloud of the fourth region within the second detection area and the center point of the precise sling shaft alignment bounding box. Obtain the distance between the intersection of the line segment and the precise sling shaft alignment bounding box and the point in the 3D point cloud of the fourth region within the second detection area. If the distance is less than a set distance threshold, there is a risk. S54: If there is a collision risk in either the first detection area or the second detection area, then the crane load is at risk of collision.

Images