A crane load anti-collision detection method based on real-time three-dimensional point cloud driving

Abstract

The embodiment discloses a crane load anti-collision detection method based on real-time three-dimensional point cloud driving, obtains a three-dimensional detection area of a hoisted load through three-dimensional position coordinates of a hook, carries out clustering on three-dimensional point clouds in a third area in the three-dimensional detection area of the hoisted load, obtains a plurality of point cloud clusters and scores, obtains an optimal point cloud cluster with the lowest score, obtains an accurate hoisted load axis alignment bounding box based on the optimal point cloud cluster, determines a collision avoidance detection area bounding box based on the size of the accurate hoisted load axis alignment bounding box, obtains three-dimensional point clouds in a fourth area arranged between the accurate hoisted load axis alignment bounding box and the collision avoidance detection area bounding box according to the three-dimensional point clouds in the first area, and carries out collision avoidance detection according to the three-dimensional point clouds in the fourth area. The application can be applied to three-dimensional size identification of a hoisted load in a construction process and anti-collision detection between a shipbuilding gantry crane body, a hoisted load and surrounding potential obstacles.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of hoisting equipment safety, and in particular to a crane load anti-collision detection method based on real-time three-dimensional point cloud driving. BACKGROUND

[0002] In the process of shipbuilding and large structure assembly, shipbuilding gantry crane undertakes a large number of heavy component hoisting tasks. Due to the variety of ship sections and components with different shapes, the hoisting operation environment is complex, and there is a risk of collision between the gantry crane body, the hoisted load and the surrounding facilities, personnel or other components. Once a collision occurs, not only the equipment may be damaged, but also the production progress may be seriously affected and safety hazards may be caused. Therefore, in the hoisting process, accurate three-dimensional size perception of the hoisted load and all-around anti-collision detection of the working space become important issues to improve the safety and efficiency of hoisting operations.

[0003] At present, there are some technical solutions for anti-collision or object recognition of hoisting equipment in the industry, which mainly have the following defects: in the method of using an industrial camera to collect images and identifying and positioning specific types of hoisted loads through a pre-trained deep learning model, the effect is good when identifying known objects with limited types, but the types of components involved in the field environment of a shipyard are as many as thousands, and the shapes and sizes differ greatly, so a single model cannot cover all targets; at the same time, two-dimensional vision lacks depth information, and cannot directly obtain the three-dimensional size of the hoisted load, nor accurately determine the spatial relationship between the hoisted load and the surrounding obstacles, therefore, the existing solutions have deficiencies in dealing with complex and diverse hoisting objects in a shipyard, large hoisted load full-space monitoring and adaptability of gantry cranes with special structures, and cannot meet the requirements of high-precision, full-coverage and real-time anti-collision and size recognition. Thus, it cannot meet the anti-collision requirements under complex working conditions. SUMMARY

[0004] The present application discloses a crane load anti-collision detection method based on real-time three-dimensional point cloud driving to overcome the above technical problems.

[0005] In order to achieve the above purpose, the technical solution of the present application is as follows:

[0006] A crane load anti-collision detection method based on real-time three-dimensional point cloud driving, comprising the following steps:

[0007] S1: obtaining a first region three-dimensional point cloud in a shipbuilding gantry crane working area and a three-dimensional position coordinate of a hook of the shipbuilding gantry crane; wherein the shipbuilding gantry crane is a gantry crane or a portal crane;

[0008] S2: determining a hoisted load three-dimensional detection region according to the three-dimensional position coordinate of the hook to obtain a third region three-dimensional point cloud in the hoisted load three-dimensional detection region;

[0009] S3: obtaining a plurality of point cloud clusters based on a clustering algorithm according to the third region three-dimensional point cloud in the hoisted load three-dimensional detection region, obtaining scores of the plurality of point cloud clusters, and further obtaining a point cloud cluster with the lowest score; and obtaining an optimal point cloud cluster;

[0010] S4: obtaining an accurate hoisted load axis-aligned bounding box according to the optimal point cloud cluster, and determining a size of the accurate hoisted load axis-aligned bounding box;

[0011] S5: determining a collision avoidance detection region bounding box based on the size of the accurate hoisted load axis-aligned bounding box, obtaining a fourth region three-dimensional point cloud between the accurate hoisted load axis-aligned bounding box and the collision avoidance detection region bounding box according to the first region three-dimensional point cloud, and performing collision avoidance detection according to the fourth region three-dimensional point cloud.

[0012] Further, when the shipbuilding gantry crane is a gantry crane, a method for obtaining three-dimensional position coordinates of a hook of the shipbuilding gantry crane is as follows:

[0013] S11: obtaining a theoretical three-dimensional position coordinate of the hook as follows:

[0014]

[0015]

[0016]

[0017] In the formula: represents a theoretical position coordinate of the hook in the X direction; represents a theoretical position coordinate of the hook in the Y direction; represents a theoretical position coordinate of the hook in the Z direction; represents an X direction coordinate of the origin of the trolley coordinate system; represents a Y direction coordinate of the origin of the trolley coordinate system; represents a real-time height of the bottom of the hook; represents a real-time position of the trolley in the X direction, i.e., a real-time position of the trolley in the direction of the trolley track; represents an offset of the hook in the X direction; represents an offset of the hook in the Y direction;

[0018] S12: segmenting the first region three-dimensional point cloud according to an XY plane where the maximum coordinate of the top of the hook in the Z direction is located, and obtaining a second region three-dimensional point cloud above the XY plane where the maximum coordinate of the top of the hook in the Z direction is located;

[0019] S13: clustering the second region three-dimensional point cloud to obtain M hook rope point cloud clusters, and obtaining a cluster center coordinate of the mth hook rope point cloud cluster ; wherein, m is the index of the sling point cloud cluster; M is the total number of the sling point cloud clusters; respectively, the coordinates of the cluster center of the mth sling point cloud cluster in the X direction, Y direction, and Z direction;

[0020] S14: When the number M of the sling point cloud clusters is less than the number N of the hooks:

[0021] S141: Obtain the distance between the coordinates of the cluster center of the 1st sling point cloud cluster in the XY plane and the coordinates of the theoretical three-dimensional position of all hooks in the XY plane ;

[0022] S142: When is the minimum, the final three-dimensional position coordinates of the nth hook at this time are as follows:

[0023]

[0024] In the formula: respectively, the three-dimensional position coordinates of the X direction, Y direction, and Z direction of the nth hook; respectively, the coordinates of the 1st sling point cloud cluster in the X direction and Y direction; denotes the theoretical position coordinates of the Z direction of the nth hook;

[0025] S143: Based on the 2nd, 3rd M sling point cloud clusters, repeat S141-S142;

[0026] S144: At this time, the final three-dimensional position coordinates of the remaining N-M hooks are as follows:

[0027] ;

[0028] S15: When the number M of the sling point cloud clusters is greater than or equal to the number N of the hooks:

[0029] Obtain the distance between the coordinates of the theoretical three-dimensional position of the nth hook in the XY plane and the coordinates of the cluster center of all sling point cloud clusters in the XY plane ;

[0030] When is the minimum, the final three-dimensional position coordinates of the nth hook are as follows:

[0031] .

[0032] Further, when the shipbuilding gantry crane is a portal crane, the method for obtaining the three-dimensional position coordinates of the hook is as follows:

[0033] S111: Obtain the position coordinates of the crane arm after rotation relative to the cabin:

[0034]

[0035]

[0036] In the formula: These are the X-coordinates and Y-coordinates of the position of the crane boom relative to the cab after rotation. Indicates transpose; These are the X-coordinates and Y-coordinates of the initial position of the crane boom relative to the cab, respectively. This is the two-dimensional rotation matrix of the crane about the Z-axis; The two-dimensional rotation angle of the cockpit around the Z-axis;

[0037] S112: Obtain the increase in end displacement of the hook in the XY plane along the extension direction of the crane boom;

[0038]

[0039] In the formula: This indicates the increase in end displacement of the hook in the XY plane along the extension direction of the crane boom; This represents the two-dimensional rotation angle of the cockpit around the Z-axis; Indicates the pitch angle of the crane boom; Indicates the length of the crane boom; express The component in the X direction; express The component in the Y direction;

[0040] S113: Obtain the three-dimensional position coordinates of the hook as follows:

[0041] In the formula: The X-axis position coordinates of the cockpit relative to the origin of the gantry crane coordinate system; The Y-axis position coordinates of the cockpit relative to the origin of the gantry crane coordinate system;

[0042] Expanded into an explicit formula, it can be expressed as follows:

[0043]

[0044]

[0045]

[0046] In the formula: This indicates the position coordinates of the hook in the X direction; This indicates the position coordinates of the hook in the Y direction; This indicates the position coordinates of the hook in the Z direction.

[0047] Further, when the shipbuilding gantry crane is a portal crane, the hoisted object three-dimensional detection area is obtained as follows:

[0048]

[0049] In the formula: is the hoisted object three-dimensional detection area; is the X direction coordinate of any point in the hoisted object three-dimensional detection area in the global coordinate system; is the Y direction coordinate of any point in the hoisted object three-dimensional detection area in the global coordinate system; is the Z direction coordinate of any point in the hoisted object three-dimensional detection area in the global coordinate system; is the three-dimensional space; represents the position coordinate of the hook in the X direction; represents the position coordinate of the hook in the Y direction; represents the position coordinate of the hook in the Z direction; is the ground height value in the Z axis direction in the global coordinate system; is the ground clearance height value; represents the transpose;

[0050] wherein,

[0051]

[0052]

[0053] In the formula: is the actual half-expansion in the X direction; is the actual half-expansion in the Y direction; is a scaling function that varies with the weight section; is the reference half-expansion in the X direction without scaling; is the reference half-expansion in the Y direction without scaling;

[0054]

[0055] In the formula: is the current hoisted object weight; is the scaling coefficient; is the scaling limit value.

[0056] Further, when the shipbuilding gantry crane is a portal crane, the hoisted object three-dimensional detection area is:

[0057] When the number of hooks is 1, the hoisted object three-dimensional detection area is:

[0058]

[0059]

[0060]

[0061] wherein: is the spread distance of the hoist load three-dimensional detection area set in the X direction; is the spread distance of the hoist load three-dimensional detection area set in the Y direction; is the spread distance of the hoist load three-dimensional detection area set in the Z direction;

[0062] When the number of hooks is 2, the hoist load three-dimensional detection area is:

[0063]

[0064]

[0065]

[0066]

[0067] wherein: is the position coordinate of the first hook in the X direction; is the position coordinate of the second hook in the X direction; is the position coordinate of the first hook in the Y direction; is the position coordinate of the second hook in the Y direction; is the position coordinate of the first hook in the Z direction; is the position coordinate of the second hook in the Z direction; is the maximum value of the coordinates of the first hook and the second hook in the Z direction;

[0068] When the number of hooks is greater than 2, the hoist load three-dimensional detection area acquisition method is as follows:

[0069] First, the minimum rectangular area of the hooks in the XY plane under the global coordinate system is obtained:

[0070]

[0071] wherein:

[0072]

[0073]

[0074]

[0075]

[0076] wherein: Index of the hook; Total number of hooks in the hoist load; X-direction coordinate of the th hook in the global coordinate system; Y-direction coordinate of the th hook in the global coordinate system; Minimum rectangular frame in the XY plane; Minimum X-direction coordinate of all hooks in the global coordinate system in the hoist load; Maximum X-direction coordinate of all hooks in the global coordinate system in the hoist load; Minimum Y-direction coordinate of all hooks in the global coordinate system in the hoist load; Maximum Y-direction coordinate of all hooks in the global coordinate system in the hoist load; Indicates transposition; Two-dimensional space;

[0077] Then, the three-dimensional detection area of the hoist load at this time is obtained as follows:

[0078]

[0079]

[0080] wherein,

[0081]

[0082] In the formula: Maximum Z-direction coordinate of all hooks in the global coordinate system in the hoist load.

[0083] Further, the method for obtaining the score of the point cloud cluster is as follows:

[0084] S31: Based on the third area three-dimensional point cloud in the three-dimensional detection area of the hoist load, a plurality of point cloud clusters are obtained based on a clustering algorithm to obtain a point cloud cluster center point position and a point cloud cluster size;

[0085] The position of the point cloud cluster center point is obtained as follows:

[0086]

[0087] In the formula: Position coordinate of the point cloud cluster center point; X-direction, Y-direction, and Z-direction coordinates of the current point cloud cluster center point position; Minimum and maximum values of the X-direction coordinate of the axis-aligned bounding box of the point cloud cluster; Minimum and maximum values of the Y-direction coordinate of the axis-aligned bounding box of the point cloud cluster; These are the minimum and maximum values ​​of the Z-direction coordinates enclosed by the axis alignment of the point cloud cluster, respectively;

[0088] The dimensions of the point cloud cluster are obtained as follows:

[0089]

[0090] In the formula: The dimension of the axis-aligned bounding box of the point cloud cluster at the current moment in the X direction, which is the width of the point cloud cluster at the current moment; The dimension of the axis-aligned bounding box of the point cloud cluster at the current moment in the Y direction, i.e., the length of the point cloud cluster at the current moment; The dimension of the axis-aligned bounding box of the point cloud cluster at the current moment in the Z direction is the height of the point cloud cluster at the current moment;

[0091] S32: Obtain the Euclidean distance between the current position of the point cloud cluster center point and the previous position of the hoisted object center point; obtain the normalized upper bound of the Euclidean distance between the axis-aligned bounding box of the point cloud cluster and the center point of the axis-aligned bounding box of the previously identified target hoisted object. ;

[0092] The formula used to obtain the Euclidean distance between the current position of the cloud cluster center point and the previous position of the hoisted object is as follows:

[0093]

[0094] In the formula: distance is the Euclidean distance between the axis-aligned bounding box of the point cloud cluster and the center point of the axis-aligned bounding box of the previously identified target load, that is, the Euclidean distance between the current center point of the point cloud cluster and the center point of the previously identified load. Represents the square root; , , These are the X, Y, and Z coordinates of the center point of the current point cloud cluster, respectively. , , These are the X, Y, and Z coordinates of the center point of the target hoisting object's bounding box at the previous moment;

[0095] The formula used to obtain the normalized upper bound of the Euclidean distance between the axis-aligned bounding box of the point cloud cluster and the center point of the axis-aligned bounding box of the previously identified target sling is as follows:

[0096]

[0097] In the formula: a normalized upper bound of the Euclidean distance between the axis-aligned bounding box of the point cloud cluster and the axis-aligned bounding box center point of the target hoisted object identified last time; a normalized upper bound of the Euclidean distance between the axis-aligned bounding box of the point cloud cluster and the axis-aligned bounding box center point of the target hoisted object identified last time; a normalized upper bound of the Euclidean distance between the axis-aligned bounding box of the point cloud cluster and the axis-aligned bounding box center point of the target hoisted object identified last time; an index of the point cloud cluster; a total number of point cloud clusters;

[0098] S33: obtaining a three-dimensional size difference value of the axis-aligned bounding box of the point cloud cluster at the current moment and the axis-aligned bounding box of the target hoisted object at the last moment, so as to obtain a normalized upper bound of the three-dimensional size difference value of the current point cloud cluster and the three-dimensional size of the axis-aligned bounding box of the target hoisted object at the last moment

[0099] The formula used to obtain the three-dimensional size difference value of the axis-aligned bounding box of the point cloud cluster at the current moment and the axis-aligned bounding box of the target hoisted object at the last moment is as follows:

[0100]

[0101] In the formula, is the three-dimensional size difference value of the current point cloud cluster and the three-dimensional size of the axis-aligned bounding box of the target hoisted object at the last moment; are the width, length and height of the axis-aligned bounding box of the target hoisted object at the last moment, i.e., the width, length and height of the axis-aligned bounding box of the target hoisted object at the last moment;

[0102] The formula used to obtain the normalized upper bound of the three-dimensional size difference value of the current point cloud cluster and the three-dimensional size of the axis-aligned bounding box of the target hoisted object at the last moment is as follows:

[0103]

[0104] In the formula, is the normalized upper bound of the three-dimensional size difference value of the current point cloud cluster and the three-dimensional size of the axis-aligned bounding box of the target hoisted object at the last moment;

[0105] S34: obtaining the Euclidean distance between the axis-aligned bounding box center point of the point cloud cluster and the center point of the upper surface of the three-dimensional detection region of the hoisted object, so as to obtain a normalized upper bound of the Euclidean distance between the axis-aligned bounding box center point of the point cloud cluster and the center point of the upper surface of the three-dimensional detection region of the hoisted object

[0106] The formula used to obtain the Euclidean distance between the axis-aligned bounding box center point of the point cloud cluster and the center point of the upper surface of the three-dimensional detection region of the hoisted object is as follows:

[0107] ​​

[0108] In the formula: The Euclidean distance from the center point of the axis-aligned bounding box of the point cloud cluster to the center point of the upper surface of the 3D detection area of ​​the suspended object; , , These are the X, Y, and Z coordinates of the center point of the upper surface of the three-dimensional detection area of ​​the hoisted object at the current moment;

[0109] The formula used to obtain the normalized upper bound of the Euclidean distance from the center point of the axis-aligned bounding box of the point cloud cluster to the center point of the upper surface of the 3D detection area of ​​the suspended object is as follows:

[0110]

[0111] In the formula: The normalized upper bound of the Euclidean distance from the center point of the axis-aligned bounding box of the point cloud cluster to the center point of the upper surface of the 3D detection area of ​​the suspended object;

[0112] S35: Obtain the Euclidean distance between the axis-aligned bounding box of the normalized point cloud cluster and the center point of the axis-aligned bounding box of the previously identified target load, the 3D size difference between the axis-aligned bounding box of the normalized point cloud cluster and the axis-aligned bounding box of the target load at the previous moment, and the Euclidean distance from the center point of the axis-aligned bounding box of the normalized point cloud cluster to the center point of the upper surface of the 3D detection area of ​​the load. The formula used is as follows:

[0113]

[0114]

[0115]

[0116] In the formula: The Euclidean distance between the axis-aligned bounding box of the normalized point cloud cluster and the center point of the axis-aligned bounding box of the previously identified target sling; The difference in three-dimensional dimensions between the axis-aligned bounding box of the normalized point cloud cluster and the axis-aligned bounding box of the target hoist at the previous moment. The Euclidean distance from the center point of the axis-aligned bounding box of the normalized point cloud cluster to the center point of the upper surface of the 3D detection area of ​​the suspended object; This is a parameter used to ensure that the denominator is not zero;

[0117] S36: When the size of a point cloud cluster is greater than a size threshold, a score for the point cloud cluster is obtained using the following formula:

[0118]

[0119] In the formula: They are respectively weight of the fourth region.

[0120] Further, the center of the anti-collision detection region bounding box is aligned with the center of the accurate hoisting load axis alignment bounding box and the center line, and the size of the anti-collision detection region bounding box is greater than the size of the accurate hoisting load axis alignment bounding box.

[0121] Further, the S5 comprises:

[0122] S51: determining a detection region arranged between the accurate hoisting load axis alignment bounding box and the anti-collision detection region bounding box according to the accurate hoisting load axis alignment bounding box and the anti-collision detection region bounding box, to determine a first detection region and a second detection region;

[0123] The first detection region comprises:

[0124] respectively taking the bth side face of the accurate hoisting load axis alignment bounding box as the bottom face, b=1, 2, 3, 4, and the top face is arranged on the 4 cuboids on the side face of the corresponding anti-collision detection region bounding box;

[0125] taking the bottom face of the accurate hoisting load axis alignment bounding box as the bottom face, and the top face is arranged on the 5th cuboid on the bottom face of the anti-collision detection region bounding box;

[0126] The second detection region is a region outside the first detection region between the accurate hoisting load axis alignment bounding box and the anti-collision detection region bounding box;

[0127] S52: determining whether there is a collision risk in the first detection region, and the method adopted is:

[0128] If the points in the fourth region three-dimensional point cloud in the bth cuboid, b=1, 2, 3, 4, exist between the bth side face of the accurate hoisting load axis alignment bounding box and the distance is less than the set distance threshold, there is a collision risk;

[0129] If the points in the fourth region three-dimensional point cloud in the 5th cuboid exist between the bottom face of the accurate hoisting load axis alignment bounding box and the distance is less than the set distance threshold, there is a collision risk;

[0130] S53: determining whether there is a collision risk in the second detection region, and the method adopted is:

[0131] obtaining a line segment with the point in the fourth region three-dimensional point cloud in the second detection region and the center point of the accurate hoisting load axis alignment bounding box as end points, to obtain the intersection point of the line segment and the accurate hoisting load axis alignment bounding box and the distance between the point in the fourth region three-dimensional point cloud in the second detection region, if less than the set distance threshold, there is a risk;

[0132] S54: When any of the first detection area and the second detection area has a collision risk, then the crane load has a collision risk.

[0133] Beneficial effects: The crane load anti-collision detection method based on real-time three-dimensional point cloud driving of the application obtains a three-dimensional detection area of the load through the three-dimensional position coordinates of the hook, clusters the third area three-dimensional point cloud in the three-dimensional detection area of the load, obtains a plurality of point cloud clusters and scores, obtains the optimal point cloud cluster with the lowest score, obtains an accurate load axis-aligned bounding box based on the optimal point cloud cluster, determines an anti-collision detection area bounding box based on the size of the accurate load axis-aligned bounding box, obtains the fourth area three-dimensional point cloud arranged between the accurate load axis-aligned bounding box and the anti-collision detection area bounding box according to the first area three-dimensional point cloud, and performs anti-collision detection according to the fourth area three-dimensional point cloud. The application can realize anti-collision detection of two different configurations of shipbuilding gantry cranes during operation, can perceive the accurate three-dimensional size of the load during hoisting, can perform omnidirectional anti-collision detection on the operation area, can cover the detection of objects with different shapes and sizes in the complex and diverse hoisting object scene of the shipyard, can accurately judge the relationship between the load and the surrounding obstacles, has high detection accuracy, and can meet the anti-collision requirements under complex working conditions. It can be applied to three-dimensional size recognition of the load during construction and anti-collision detection between the shipbuilding gantry crane body, the load and the surrounding potential obstacles. BRIEF DESCRIPTION OF DRAWINGS

[0134] In order to more clearly illustrate the technical solutions in the embodiments of the application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are some embodiments of the application, and those skilled in the art can also obtain other drawings according to these drawings without creative labor.

[0135] Figure 1 The flowchart of the crane load anti-collision detection method based on real-time three-dimensional point cloud driving of the application;

[0136] Figure 2 The lateral schematic diagram of the laser radar scanning range of the portal crane in the embodiment of the application;

[0137] Figure 3 The forward schematic diagram of the laser radar scanning range of the portal crane in the embodiment of the application;

[0138] Figure 4 The forward schematic diagram of the laser radar scanning range of the portal crane in the embodiment of the application;

[0139] Figure 5 A top view schematic diagram of a laser radar scanning range of a portal crane in an embodiment of the present application;

[0140] Figure 6 A schematic diagram of a collision avoidance detection area in an embodiment of the present application;

[0141] Figure 7 A schematic diagram of a line segment in spherical collision avoidance detection in an embodiment of the present application.

[0142] Wherein, 1, a boom; 2, a hoisted load; 31, a first laser radar; 32, a second laser radar; 4, a cab; A, a spatial point; B, a hoisted load bounding box center point. DETAILED DESCRIPTION

[0143] To make the objectives, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described below in connection with the drawings in the embodiments of the present application. Obviously, the described embodiments are some but not all of the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0144] The present embodiment introduces a crane hoisted load collision avoidance detection method based on real-time three-dimensional point cloud driving, including the following steps, as shown in Figure 1 .

[0145] S1: Obtain a first area three-dimensional point cloud in a shipbuilding gantry crane working area and a three-dimensional position coordinate of a hoist hook of the shipbuilding gantry crane; wherein the shipbuilding gantry crane is a portal crane or a gantry crane;

[0146] In the present embodiment, all three-dimensional point cloud data in the working area range of the crane and the three-dimensional position coordinates of all hoist hooks on the shipbuilding gantry crane (portal crane or gantry crane) are obtained; wherein the gantry crane (hereinafter referred to as portal crane) and the gantry crane in the field of hoisting equipment such as port or shipbuilding are collectively referred to as shipbuilding gantry crane.

[0147] In the present embodiment, the first area three-dimensional point cloud is the three-dimensional point cloud data of the working area range obtained based on two laser radars, the point clouds are merged in the gantry crane coordinate system, and the downsampling and cutting are performed. Specifically, the specified space includes the three-dimensional space in the working area and a certain distance on both sides. For the gantry crane, it is the three-dimensional space in the front facing direction of the cab 4 left and right and up and down. The specific working area range depends on the scanning range and installation angle of the laser radar, as shown in Figure 2 and Figure 3All objects such as parts, hulls, transport vehicles and people within the range of the crane working area are reflected in real time to obtain a three-dimensional point cloud in the crane designated space based on two laser radar scans.

[0148] The embodiment can collect real-time crane data through the PLC and encoder installed on the crane. For a gantry crane, the real-time crane data is the real-time load of all trolley positions, all hook heights and all hooks on the body. For a portal crane, the real-time crane data is the real-time load of the rotation angle of the rotating platform on the body, the pitch angle of the boom, all hook heights and all hooks. According to the real-time crane data and the kinematic model of the shipbuilding gantry crane, the original three-dimensional coordinates of all hooks in the gantry crane coordinate system are calculated.

[0149] Preferably, when the shipbuilding gantry crane is a gantry crane, the three-dimensional position coordinates of the hook of the shipbuilding gantry crane are obtained as follows:

[0150] Specifically, in the embodiment, the kinematic model of the shipbuilding gantry crane is based on the gantry crane coordinate system, i.e. the global coordinate system. The global coordinate system (i.e. the gantry crane coordinate system) in the embodiment takes the vertically upward direction as the positive direction of the Z direction and takes the plane of the gantry crane track as the XY plane, wherein the direction along the trolley track is the X direction and the direction perpendicular to the trolley track in the XY plane is the Y direction. When the shipbuilding gantry crane is a gantry crane, the three-dimensional position of the hook is obtained based on the kinematic model of the gantry crane as follows:

[0151] S11: The theoretical three-dimensional position coordinates of the hook are obtained as follows:

[0152] First, the position of the trolley coordinate system origin relative to the gantry coordinate system origin is obtained:

[0153]

[0154] In the formula: represents the position of the trolley coordinate system origin relative to the base, i.e. the position relative to the gantry coordinate system origin; represents the X direction coordinate of the trolley coordinate system origin; represents the Y direction coordinate of the trolley coordinate system origin; represents the Z direction coordinate of the trolley coordinate system origin;

[0155] Secondly, the theoretical three-dimensional position coordinates of the hook are obtained as follows:

[0156] 1. The theoretical position of the hook in the X direction is obtained as follows:

[0157] In the formula: represents the theoretical position coordinate of the hook in the X direction; This represents the X-axis coordinate of the origin of the trolley's coordinate system; This indicates the real-time position of the trolley in the X direction, that is, the real-time position of the trolley along the trolley track. This indicates the offset of the hook in the X direction;

[0158] In this embodiment, the direction in which the trolley travels along the track is defined as the X-axis direction.

[0159] 2. The theoretical position of the hook in the Y direction;

[0160] In the formula: This represents the theoretical position coordinates of the hook in the Y direction; This represents the Y-axis coordinate of the origin of the trolley's coordinate system; This indicates the offset of the hook in the Y direction;

[0161] 3. The theoretical position of the hook in the Z direction;

[0162] In the formula: This represents the theoretical position coordinates of the hook in the Z direction; Indicates the real-time height of the hook bottom;

[0163] Finally, the theoretical three-dimensional position of the hook (hook bottom) in the global coordinate system is obtained:

[0164] In the formula: This represents the theoretical three-dimensional position of the hook in the global coordinate system.

[0165] Specifically, when calculating the three-dimensional position coordinates of the hook, the offset of the hook in the X direction... The offset of the hook in the Y direction Real-time height of hook bottom Real-time position of the trolley along the track direction It is known.

[0166] S12: Based on the XY plane where the maximum coordinate of the hook top in the Z direction is located, the three-dimensional point cloud of the first region is segmented to obtain the three-dimensional point cloud of the second region above the XY plane where the maximum coordinate of the hook top in the Z direction is located.

[0167] S13: Cluster the 3D point cloud of the second region to obtain M rope-related point cloud clusters, and obtain the cluster center coordinates of the m-th rope-related point cloud cluster. Where m is the index of the suspending tether point cloud cluster; M is the total number of suspending tether point cloud clusters; These are the coordinates of the cluster center of the m-th suspending rope point cloud cluster in the X, Y, and Z directions, respectively.

[0168] S14: When the number of cloud clusters on the lifting rope M is less than the number of hooks N:

[0169] S141: Obtain the distance between the coordinates of the cluster center of the first sling point cloud cluster in the XY plane and the coordinates of the theoretical 3D positions of all hooks in the XY plane. The formula used is as follows:

[0170]

[0171] In the formula: Let be the distance between the cluster center coordinates of the m-th hoisting rope point cloud cluster and the theoretical three-dimensional position coordinates of the n-th hook in the XY plane; Let X be the theoretical position coordinates of the nth hook in the X direction; This represents the theoretical position coordinates of the nth hook in the Y direction;

[0172] S142: When When the minimum value is reached, the final three-dimensional position coordinates of the nth hook are obtained as follows:

[0173]

[0174] In the formula: These are the three-dimensional position coordinates of the nth hook in the X, Y, and Z directions, respectively, which are the three-dimensional position coordinates of the hook after the second calibration. These are the coordinates of the first suspending rope point cloud cluster in the X and Y directions, respectively; This represents the theoretical position coordinates of the nth hook in the Z direction;

[0175] S143: Based on the 2nd and 3rd sequentially For M suspending rope point cloud clusters, repeat S141-S142;

[0176] S144: At this point, the final three-dimensional position coordinates of the remaining NM hooks are as follows:

[0177]

[0178] S15: When the number of cloud clusters on the lifting rope M is greater than or equal to the number of hooks N:

[0179] Obtain the distance between the theoretical 3D position of the nth hook in the XY plane and the coordinates of the cluster center of all the rope point cloud clusters in the XY plane. ;

[0180] when When the minimum value is reached, the final three-dimensional position coordinates of the nth hook are as follows:

[0181] .

[0182] Specifically, the nearest distance pairing method is suitable for obtaining the final three-dimensional position coordinates of the hooks when the number of hooks is greater than 2 (especially when the number of hooks is 2 or 4) (wherein when the number of hooks is 1, the position of the hook does not need to be re-calibrated); first, the first region three-dimensional point cloud is segmented to obtain the second region three-dimensional point cloud and clustering is performed, a plurality of hook rope point cloud clusters can be obtained, and the cluster center coordinates of each cluster are calculated. In order to avoid the influence of the height disturbance of the hook rope on the matching, only the X and Y direction coordinate information of the cluster center is used during the matching. The specific pairing method is as follows: for each hook, find the cluster center closest to it in the XY plane among the "unassigned" cluster centers, and assign the cluster center to the hook. In order to ensure a one-to-one relationship, once each cluster center is assigned to a hook, it is removed from the candidate set and no longer participates in the matching of subsequent hooks. When the number of hook rope point cloud clusters M is greater than or equal to the number of hooks N (M≥N), the distance between the nth hook and the mth point cloud cluster is the smallest, and the X and Y direction coordinates of the mth point cloud cluster center replace the X and Y direction coordinates of the nth hook, thereby realizing the secondary correction of the horizontal position of the hook. The remaining point cloud clusters that are not matched with the hooks are ignored as interference or non-target point clouds. The Z direction coordinate of the hook is not modified. When the number of cluster centers is less than the number of hooks (M

[0183] The embodiment re-calibrates the position of the hook based on the theoretical three-dimensional position coordinates of the hook: the method of obtaining all the point cloud clusters of the hook rope above the hook through a point cloud clustering algorithm, specifically, constructing a Kd-Tree on the second region three-dimensional point cloud and completing clustering by calling PCL Euclidean clustering; wherein the clustering tolerance and the minimum / maximum cluster size are configurable parameters, which can be calibrated and adjusted according to the site conditions (such as beam spacing, hook rope diameter, point cloud density). In this embodiment, the tolerance is about 1.0 m, the minimum is 6 points, and the maximum is 1000 points (all are recommended values, not limited). The cluster centroids are extracted and calculated according to the cluster index to obtain the set of hook rope point cloud clusters. Then, the hook rope point cloud clusters above all the hooks are matched with the theoretical three-dimensional position coordinates of the hooks to update the secondary calibration three-dimensional coordinates of all the hooks in the portal crane coordinate system.

[0184] Preferably, when the shipbuilding portal crane is a gantry crane, the method for obtaining the three-dimensional position coordinates of the hook is as follows:

[0185] Specifically, the embodiment obtains the three-dimensional position of the hook based on the kinematic model of the hook of the gantry crane in the portal crane coordinate system: wherein the position of the cab (cab prime) relative to the origin (base) of the portal crane coordinate system is: , the position of the jib relative to the cabin is: is known, where, is the X-direction position coordinate of the cabin relative to the origin of the portal coordinate system; is the Y-direction position coordinate of the cabin relative to the origin of the portal coordinate system; is the Z-direction position coordinate of the cabin relative to the origin of the portal coordinate system.

[0186] S111: Obtain the position coordinates of the crane arm after rotation relative to the cabin:

[0187] Specifically, the position of the jib relative to the cabin is rotated with the cabin in the plane, and the position of the crane arm after rotation relative to the cabin is obtained:

[0188]

[0189] wherein: and are the X-direction coordinate and the Y-direction coordinate, respectively, of the position of the crane arm after rotation relative to the cabin; denotes transposition; and are the X-direction coordinate and the Y-direction coordinate, respectively, of the initial position of the crane arm relative to the cabin; is the two-dimensional rotation matrix of the crane around the Z-axis (acting on the plane); is the two-dimensional rotation angle of the cabin around the Z-axis. S112: Obtain the end displacement of the hook in the X-Y plane in the direction of extension of the crane boom;

[0190] Specifically, the end displacement in the plane in the forward extension direction of the crane boom (the length of the jib is modulated by the horizontal projection of the pitch angle of the jib relative to its own axis:

[0191]

[0192] wherein: denotes the end displacement of the hook in the X-Y plane in the direction of extension of the crane boom; denotes the two-dimensional rotation angle of the cabin around the Z-axis; denotes the pitch angle of the jib; denotes the length of the jib; denotes the component in the X-direction; denotes the component in the Y-direction;

[0193] S113: Obtain the three-dimensional position coordinates of the hook as follows: that is, the position of the hook (hook bottom) in the global coordinate system:

[0194] In the formula: is the X-direction position coordinate of the cabin relative to the origin of the door machine coordinate system; is the Y-direction position coordinate of the cabin relative to the origin of the door machine coordinate system;

[0195] The expansion is expressed as an explicit formula as follows:

[0196]

[0197]

[0198] ;

[0199] In the formula: represents the X-direction position coordinate of the hook; represents the Y-direction position coordinate of the hook; represents the Z-direction position coordinate of the hook.

[0200] S2: Determine the three-dimensional detection area of the hoisted object according to the three-dimensional position coordinates of the hook; to obtain the three-dimensional point cloud in the three-dimensional detection area of the hoisted object, that is, the third area three-dimensional point cloud;

[0201] Specifically, based on the three-dimensional position coordinates of the hook and the door machine body feature (that is, the number of hooks participating in hoisting), different three-dimensional detection areas of the hoisted object are determined, and the position of the three-dimensional detection area of the hoisted object is determined by the hook position and the door machine body feature. The hook position is the position after the secondary calibration for the portal crane; for the portal crane, since it does not involve secondary calibration, the position of the hook (hook bottom) in the global coordinate system is calculated according to the kinematic model of the hook of the portal crane.

[0202] The door machine body feature in this embodiment is the number of hooks that the door machine itself can participate in hoisting. For a portal crane, it is generally one; for a portal crane, there are multiple cases of one to four hooks participating in hoisting at the same time.

[0203] Preferably, when the shipbuilding door machine is a portal crane, the three-dimensional detection area of the hoisted object is obtained as follows:

[0204]

[0205] In the formula: is the three-dimensional detection area of the hoisted object; is the X-direction coordinate of any point in the three-dimensional detection area of the hoisted object in the global coordinate system; Y coordinate of any point in the three-dimensional detection area of the hoisted load in the global coordinate system; Z coordinate of any point in the three-dimensional detection area of the hoisted load in the global coordinate system; three-dimensional space; X-direction position coordinate of the hook; Y-direction position coordinate of the hook; Z-direction position coordinate of the hook; ground height value in the Z-axis direction in the global coordinate system; ground clearance height value;

[0206] wherein,

[0207]

[0208]

[0209] wherein: actual half-expansion in the X direction; actual half-expansion in the Y direction; scaling function varying with weight section; un-scaled reference half-expansion in the X direction; un-scaled reference half-expansion in the Y direction;

[0210]

[0211] wherein: current hoisted load weight; scaling coefficient; scaling limit value.

[0212] Specifically, the determination of the three-dimensional detection area of the hoisted load of the portal crane is only related to the position of the single hook in the current hoist, and the shape is a cuboid.

[0213] Specifically, for ease of description, a global coordinate system is adopted wherein X and Y are horizontal (XY plane) coordinate axes, and Z is the vertical direction. The position of the hook (hook bottom) in the global coordinate system is denoted as .

[0214] Suppose the ground is an isoplanar surface , the ground clearance height is , the lower boundary in the vertical direction is , and the upper boundary is satisfying the engineering constraint , and the minimum vertical thickness of the three-dimensional detection area of the hoisted load defined according to the actual working condition is introduced The ground clearance height is used to represent the minimum coordinate of the Z direction of the three-dimensional detection area of the hoisted object in the global coordinate system.

[0215] If , then ; if ≤ , then .

[0216] To give the horizontal (XY plane) extent, define the unscaled reference half- extent: its physical meaning corresponds to the reference half-length and half-width (obtained from the fixed distance configuration) along the X and Y directions, respectively.

[0217] , and 0, 0

[0218] In the formula: is the ground height value in the Z-axis direction of the global coordinate system; is the ground clearance height value; is the Z-axis height of the lower boundary of the three-dimensional detection area of the hoisted object in the global coordinate system; is the Z-axis height of the upper boundary of the three-dimensional detection area of the hoisted object in the global coordinate system; is the minimum vertical thickness of the three-dimensional detection area of the hoisted object in the global coordinate system; is the unscaled reference half-extent in the XY plane; is the unscaled reference half-extent in the X direction; is the unscaled reference half-extent in the Y direction; is the transpose;

[0219] Let the current hoisted object weight be , the scaling limit value , and the scaling coefficient (0, 1). Define the scaling function that changes with the weight segment as follows:

[0220]

[0221] In the formula: is the current hoisted object weight; is the scaling coefficient; is the scaling limit value; is the scaling function that changes with the weight segment;

[0222] Then the actual half-extent of the XY plane is:

[0223]

[0224] That is,

[0225]

[0226]

[0227] Accordingly, an axis-aligned rectangle centered at (x, y) is obtained in the XY plane:

[0228]

[0229] wherein: is the axis-aligned rectangle region; is the actual half extension of the XY plane; is the actual half extension of the X direction; is the actual half extension of the Y direction;

[0230] In this embodiment, the "suspended load three-dimensional detection region" (a rectangular parallelepiped aligned with the global coordinate axis, AABB) is defined as:

[0231]

[0232] wherein: is the region related to the three defined quantities; i.e.

[0233]

[0234]

[0235] wherein: is the suspended load three-dimensional detection region; is the X direction coordinate of any point in the suspended load three-dimensional detection region in the global coordinate system; is the Y direction coordinate of any point in the suspended load three-dimensional detection region in the global coordinate system; is the Z direction coordinate of any point in the suspended load three-dimensional detection region in the global coordinate system; is the three-dimensional space;

[0236] The above definitions ensure that the upper boundary is always greater than the lower boundary in actual engineering. Accordingly, the suspended load three-dimensional detection region R is obtained in the global coordinate system.

[0237] wherein, is a configurable parameter, which can be calibrated and adjusted according to the field conditions (such as beam spacing, sling diameter, point cloud density, etc.), and the algorithm form remains unchanged; when , the XY plane range is contracted by the scaling factor in proportion, otherwise the reference size is not scaled.

[0238] Preferably, when the shipbuilding gantry crane is a gantry crane, the suspended load three-dimensional detection region is:​​

[0239] When the number of hooks is 1, the hoisted object three-dimensional detection area is:

[0240]

[0241]

[0242]

[0243] In the formula: is the hoisted object three-dimensional detection area expansion distance set in the X direction; is the hoisted object three-dimensional detection area expansion distance set in the Y direction; is the hoisted object three-dimensional detection area expansion distance set in the Z direction;

[0244] When the number of hooks is 2, the hoisted object three-dimensional detection area is:

[0245]

[0246]

[0247]

[0248]

[0249] In the formula: is the position coordinate of the first hook in the X direction; is the position coordinate of the second hook in the X direction; is the position coordinate of the first hook in the Y direction; is the position coordinate of the second hook in the Y direction; is the position coordinate of the first hook in the Z direction; is the position coordinate of the second hook in the Z direction; is the maximum value of the coordinates of the first hook and the second hook in the Z direction;

[0250] When the number of hooks is greater than 2, the hoisted object three-dimensional detection area acquisition method is as follows:

[0251] First, the minimum rectangular area of the hooks in the XY plane (i.e. the horizontal plane) under the global coordinate system is obtained:

[0252]

[0253] Among them:

[0254]

[0255]

[0256]

[0257]

[0258] wherein: is the index of the hook; is the total number of hooks in the load; is the X direction coordinate of the th hook in the global coordinate system; is the Y direction coordinate of the th hook in the global coordinate system; is the minimum rectangular frame in the XY plane; is the minimum X direction coordinate of all hooks in the load in the global coordinate system; is the maximum X direction coordinate of all hooks in the load in the global coordinate system; is the minimum Y direction coordinate of all hooks in the load in the global coordinate system; is the maximum Y direction coordinate of all hooks in the load in the global coordinate system; is the two-dimensional space;

[0259] Then, the three-dimensional detection area of the load at this time is obtained as follows:

[0260]

[0261] wherein,

[0262] .

[0263] wherein: is the maximum Z direction coordinate of all hooks in the load in the global coordinate system.

[0264] Specifically, when the shipbuilding portal crane is a gantry crane, the three-dimensional detection area of the load of the gantry crane is related to the positions of the plurality of hooks in the current load, and the shape is a cuboid. Since the gantry crane may include two or four hooks according to different tonnages, the case of the two-hook type gantry crane is included in the scheme of the four-hook type gantry crane.

[0265] wherein, the height of the three-dimensional detection area of the load is the current height of the hook downward until the specified ground clearance height above the ground. When the number of hooks in the load is one, the three-dimensional detection area of the load is developed downward with the original three-dimensional coordinates of the hook in the portal crane coordinate system as the center, and the length and width in the horizontal direction are determined by the fixed distance. When the number of hooks in the load is two, the algorithm will calculate an equivalent position of the hook (hook bottom) in the global coordinate system The point is recorded as an equivalent hook three-dimensional space point. The calculation method of the equivalent hook three-dimensional space point is to calculate the average value of the X and Y coordinates of the two hooks in the horizontal plane direction, and expand downward with the maximum Z coordinate value in the three-dimensional coordinates of the two hooks as the center. The length and width in the XY plane direction are determined by the fixed distance set according to the project situation. When the number of hooks in the hoist load is three or four, the algorithm will be based on the minimum rectangular frame formed by the axes of the hooks in the horizontal plane in all hoist loads, add the fixed distance in the X and Y directions, and form the final length and width values in the horizontal direction.

[0266] According to the hoist load three-dimensional detection region, the third region three-dimensional point cloud inside the hoist load three-dimensional detection region is obtained by cutting from the first region three-dimensional point cloud, that is, the third region three-dimensional point cloud. The third region three-dimensional point cloud is a subset of the first region three-dimensional point cloud.

[0267] S3: According to the third region three-dimensional point cloud in the hoist load three-dimensional detection region, a plurality of point cloud clusters are obtained based on a clustering algorithm, a score of the plurality of point cloud clusters is obtained, and a point cloud cluster with the lowest score is obtained. Finally, an optimal point cloud cluster is obtained.

[0268] S31: According to the third region three-dimensional point cloud in the hoist load three-dimensional detection region, a plurality of point cloud clusters are obtained based on a clustering algorithm, a score of the plurality of point cloud clusters is obtained, and a point cloud cluster with the lowest score is obtained. Finally, an optimal point cloud cluster is obtained.

[0269] Preferably, the position of the point cloud cluster center point is obtained as follows:

[0270]

[0271] In the formula: is the position coordinate of the point cloud cluster center point; are the X direction, Y direction and Z direction coordinates of the position of the current point cloud cluster center point, respectively; are the minimum value and maximum value of the X direction coordinate of the axis-aligned bounding box of the point cloud cluster, respectively; are the minimum value and maximum value of the Y direction coordinate of the axis-aligned bounding box of the point cloud cluster, respectively; are the minimum value and maximum value of the Z direction coordinate of the axis-aligned bounding box of the point cloud cluster, respectively;

[0272] The size of the point cloud cluster is obtained as follows:

[0273]

[0274] In the formula: is the size of the axis-aligned bounding box of the point cloud cluster in the X direction at the current moment, that is, the width of the point cloud cluster at the current moment; a size of the axis-aligned bounding box of the point cloud cluster at the current time in the Y direction, i.e., a length of the point cloud cluster at the current time; a size of the axis-aligned bounding box of the point cloud cluster at the current time in the Z direction, i.e., a height of the point cloud cluster at the current time;

[0275] Specifically, the axis-aligned bounding box (AABB) is one of the most commonly used geometric concepts in computer graphics, collision detection, and space partitioning. The axis-aligned bounding box (AABB) of the embodiment is a smallest parallelepiped enclosing the target body, wherein a face parallel to the XY plane in the axis-aligned bounding box is taken as a bottom face, and a face perpendicular to the XY plane is taken as a side face.

[0276] S32: Obtain the Euclidean distance between the center point position of the point cloud cluster at the current time and the center point position of the hoisted object at the previous time; to obtain the normalized upper bound of the Euclidean distance between the axis-aligned bounding box of the point cloud cluster and the center point of the axis-aligned bounding box of the target hoisted object recognized last time

[0277] The formula used to obtain the Euclidean distance between the center point position of the point cloud cluster at the current time and the center point position of the hoisted object at the previous time is as follows:

[0278]

[0279] In the formula, distance is the Euclidean distance between the axis-aligned bounding box of the point cloud cluster and the center point of the axis-aligned bounding box of the target hoisted object recognized last time, i.e., the Euclidean distance between the center point position of the current point cloud cluster and the center point position of the hoisted object recognized last time; denotes square root; , , are the X direction, Y direction, and Z direction coordinates of the center point position of the current point cloud cluster, respectively; , , are the X direction, Y direction, and Z direction coordinates of the center point position of the target hoisted object bounding box at the previous time, respectively;

[0280] The formula used to obtain the normalized upper bound of the Euclidean distance between the axis-aligned bounding box of the point cloud cluster and the center point of the axis-aligned bounding box of the target hoisted object recognized last time is as follows:

[0281]

[0282] In the formula, ​a normalized upper bound of the Euclidean distance between the axis-aligned bounding box of the point cloud cluster and the axis-aligned bounding box center point of the target hoisted object identified last time; the axis-aligned bounding box of the target hoisted object identified last time; the axis-aligned bounding box of the target hoisted object identified last time; the index of the point cloud cluster; the total number of point cloud clusters;

[0283] obtaining a three-dimensional size difference value of the axis-aligned bounding box of the point cloud cluster at the current time and the axis-aligned bounding box of the target hoisted object at the last time, so as to obtain a normalized upper bound of the three-dimensional size difference value of the current point cloud cluster and the three-dimensional size of the axis-aligned bounding box of the target hoisted object at the last time

[0284] wherein the formula for obtaining the three-dimensional size difference value of the axis-aligned bounding box of the point cloud cluster at the current time and the axis-aligned bounding box of the target hoisted object at the last time is as follows:

[0285] wherein: is the three-dimensional size difference value of the current point cloud cluster and the three-dimensional size of the axis-aligned bounding box of the target hoisted object at the last time; are the width, length and height of the axis-aligned bounding box of the target hoisted object at the last time, i.e. the width, length and height of the axis-aligned bounding box of the target hoisted object at the last time;

[0286] the formula for obtaining the normalized upper bound of the three-dimensional size difference value of the current point cloud cluster and the three-dimensional size of the axis-aligned bounding box of the target hoisted object at the last time is as follows:

[0287]

[0288] wherein: is the normalized upper bound of the three-dimensional size difference value of the current point cloud cluster and the three-dimensional size of the axis-aligned bounding box of the target hoisted object at the last time;

[0289] S34: obtaining the Euclidean distance between the axis-aligned bounding box center point of the point cloud cluster and the center point of the upper surface of the hoisted object three-dimensional detection region (i.e. the third region), so as to obtain a normalized upper bound of the Euclidean distance between the axis-aligned bounding box center point of the point cloud cluster and the center point of the upper surface of the hoisted object three-dimensional detection region ; wherein the formula for obtaining the Euclidean distance between the axis-aligned bounding box center point of the point cloud cluster and the center point of the upper surface of the hoisted object three-dimensional detection region (i.e. the third region) is as follows:

[0290]

[0291] wherein: ​The Euclidean distance of the axis-aligned bounding box center point of the point cloud cluster to the center point of the upper surface of the hoisted object three-dimensional detection region (i.e., the third region) is obtained. , , The X direction, Y direction, and Z direction coordinates of the center point of the upper surface of the hoisted object three-dimensional detection region (i.e., the third region) at the current time are respectively obtained.

[0292] The formula used to obtain the normalized upper bound of the Euclidean distance of the axis-aligned bounding box center point of the point cloud cluster to the center point of the upper surface of the hoisted object three-dimensional detection region is as follows:

[0293]

[0294] In the formula, The normalized upper bound of the Euclidean distance of the axis-aligned bounding box center point of the point cloud cluster to the center point of the upper surface of the hoisted object three-dimensional detection region is obtained.

[0295] S35: The Euclidean distance between the normalized axis-aligned bounding box of the point cloud cluster and the axis-aligned bounding box center point of the target hoisted object identified last time, the three-dimensional size difference between the normalized axis-aligned bounding box of the point cloud cluster and the axis-aligned bounding box of the target hoisted object at the last time, and the Euclidean distance of the axis-aligned bounding box center point of the normalized point cloud cluster to the center point of the upper surface of the hoisted object three-dimensional detection region are obtained, and the formula used is as follows:

[0296]

[0297]

[0298]

[0299] In the formula, The Euclidean distance between the normalized axis-aligned bounding box of the point cloud cluster and the axis-aligned bounding box center point of the target hoisted object identified last time is obtained. The three-dimensional size difference between the normalized axis-aligned bounding box of the point cloud cluster and the axis-aligned bounding box of the target hoisted object at the last time is obtained. The Euclidean distance of the axis-aligned bounding box center point of the normalized point cloud cluster to the center point of the upper surface of the hoisted object three-dimensional detection region (i.e., the third region) is obtained. The parameter is used to ensure that the denominator is not zero, and in the present embodiment, .

[0300] S36: When the size of the point cloud cluster is greater than the size threshold, i.e., the size of the axis-aligned bounding box of the point cloud cluster in the X direction, the Y direction, and the Z direction is greater than the size threshold in the respective direction, the score of the point cloud cluster is obtained, and the formula used is as follows:

[0301]

[0302] wherein: are weight values.

[0303] Specifically, only when the three-dimensional size of the point cloud cluster is greater than the set length threshold, width threshold and height threshold respectively, the score of the obtained point cloud cluster is calculated, and small objects can be filtered out. The length threshold, width threshold and height threshold are given according to the actual situation of the project. Specifically, when the score of the point cloud cluster is the smallest, the obtained point cloud cluster is the optimal point cloud cluster.

[0304] Specifically, the third region three-dimensional point cloud in the hoisting load three-dimensional detection region is clustered and segmented to obtain all point cloud clusters, and a scoring function is used to score the point cloud clusters, and the only point cloud cluster with the lowest score is selected;

[0305] Specifically, the clustering and segmentation algorithm in the PCL library is used for the third region three-dimensional point cloud, and the clustering is completed by constructing a Kd-Tree on the third region three-dimensional point cloud and calling PCL Euclidean clustering; wherein the clustering tolerance and the minimum / maximum cluster size are configurable parameters, which are calibrated and adjusted according to the field conditions (such as beam spacing, hoisting object size range, point cloud density). The tolerance in this embodiment is about 1.3m, the minimum is 40 points, and the maximum is 10000 points.

[0306] Then the point cloud cluster obtained by clustering is used for scoring algorithm. Among them, the scoring algorithm will first eliminate the point cloud cluster whose minimum size of the three-dimensional size of the axis-aligned bounding box is less than the specified size, which can easily eliminate the hoisting rope point cloud recognized around the real hoisting load, avoiding its influence on the recognition result. Then the scoring function will calculate the Euclidean distance between the center point of the axis-aligned bounding box of all remaining point cloud clusters and the center point of the upper surface of the hoisting load three-dimensional detection region (i.e. the third region), the Euclidean distance between the axis-aligned bounding box of all point cloud clusters and the center point of the axis-aligned bounding box of the target hoisting load recognized last time, and the three-dimensional size difference of the axis-aligned bounding box. Then the score of each point cloud cluster is calculated according to the weight set in advance, and the point cloud cluster with the smallest score is finally selected as the only target point cloud cluster.

[0307] In this embodiment, the system can store the three-dimensional coordinates and three-dimensional size of the center point of the axis-aligned bounding box of the target hoisting load recognized each time in the global coordinate system, and the information of the target hoisting load recognized last time is obtained from the stored data.

[0308] S4: According to the optimal point cloud cluster, the accurate hoisting load axis-aligned bounding box is obtained, and the size of the accurate hoisting load axis-aligned bounding box is determined;

[0309] Specifically, in this embodiment, the point cloud above the optimal point cloud cluster with multiple suspension ropes is removed by a fixed percentage cutting method to obtain a precise load axis alignment bounding box, and the precise size of the load axis alignment bounding box is calculated.

[0310] Specifically, since the optimal point cloud cluster usually contains the lifting ropes directly connected to the load, this embodiment will remove the point cloud with the highest height from the unique target point cloud cluster according to a pre-set percentage rejection value. This percentage value is determined based on the actual project situation, generally ranging from 5% to 20%. This removes some meaningless lifting rope point clouds from the optimal point cloud cluster, and the remaining point clouds constitute the current target load point cloud cluster, i.e., the precise load point cloud cluster. Then, the precise load axis alignment bounding box, the three-dimensional coordinates of the bounding box's center point, and the three-dimensional dimensions of the bounding box are calculated.

[0311] The calculation of the axis-aligned bounding box (AABB), center, and 3D dimensions of the precise point cloud cluster D of the suspended load based on PCL is as follows: Let the point cloud cluster be:

[0312] In the formula: For the precise location of the load in the point cloud cluster The three-dimensional coordinates of a point in the global coordinate system; For precise indexing of points within a point cloud cluster of suspended loads; The total number of points in the precise point cloud cluster of the suspended load; For the precise location of the load in the point cloud cluster The X-coordinate of a point in the global coordinate system; For the precise location of the load in the point cloud cluster The Y-coordinate of a point in the global coordinate system; For the precise location of the load in the point cloud cluster The Z-axis coordinates of each point in the global coordinate system; D represents the precise cloud of suspended object points;

[0313] Extreme value scanning (using the PCL function pcl::getMinMax3D to find the extreme values ​​of the following formula):

[0314]

[0315]

[0316] In the formula: To accurately align the load axis bounding box (AABB) to the minimum X-axis boundary in the global coordinate system; To accurately align the load axis bounding box (AABB) to the maximum X-axis boundary in the global coordinate system; The minimum Y-axis boundary of the accurate hoist load axis-aligned bounding box (AABB) in the global coordinate system; The maximum Y-axis boundary of the accurate hoist load axis-aligned bounding box (AABB) in the global coordinate system; The minimum Z-axis boundary of the accurate hoist load axis-aligned bounding box (AABB) in the global coordinate system; The maximum Z-axis boundary of the accurate hoist load axis-aligned bounding box (AABB) in the global coordinate system;

[0317] 2) Accurate hoist load axis-aligned bounding box (AABB) definition:

[0318]

[0319] Wherein the symbol represents the superposition of the ranges of different axial regions in the global coordinate system. represents the accurate hoist load axis-aligned bounding box.

[0320] 3) Accurate hoist load axis-aligned bounding box center (three-dimensional coordinates):

[0321] Wherein, is the accurate hoist load axis-aligned bounding box center;

[0322] 4) Accurate hoist load axis-aligned bounding box three-dimensional size (length, width, height):

[0323] S5: Based on the accurate hoist load axis-aligned bounding box, determine the anti-collision detection region bounding box to obtain the three-dimensional point cloud between the accurate hoist load axis-aligned bounding box and the anti-collision detection region bounding box according to the first region three-dimensional point cloud, that is, the fourth region three-dimensional point cloud, and perform anti-collision detection according to the fourth region three-dimensional point cloud.

[0324] S51: According to the accurate hoist load axis-aligned bounding box and the anti-collision detection region bounding box, determine the detection region set between the accurate hoist load axis-aligned bounding box and the anti-collision detection region bounding box to determine the first detection region and the second detection region; wherein the first detection region and the second detection region are both sub-regions of the detection region.

[0325] The first detection region includes:

[0326] respectively take the b-th side face of the accurate hoist load axis-aligned bounding box as the bottom face, b=1, 2, 3, 4, and the top face is set on the side face of the corresponding anti-collision detection region bounding box four cuboids;

[0327] The bottom surface of the precise hoisting load axis-aligned bounding box is the bottom surface of the anti-collision detection region bounding box, and the top surface is the fifth cuboid on the bottom surface of the anti-collision detection region bounding box.

[0328] The second detection region is a region in the fourth region between the precise hoisting load axis-aligned bounding box and the anti-collision detection region bounding box, and is a region other than the first detection region.

[0329] S52: Determine whether there is a collision risk in the first detection region, and the method used is:

[0330] If the distance between the point in the fourth region three-dimensional point cloud in the bth cuboid and the bth side surface of the precise hoisting load axis-aligned bounding box is less than the set distance threshold, b = 1, 2, 3, 4, there is a collision risk.

[0331] If the distance between the point in the fourth region three-dimensional point cloud in the fifth cuboid and the bottom surface of the precise hoisting load axis-aligned bounding box is less than the set distance threshold, there is a collision risk.

[0332] S53: Determine whether there is a collision risk in the second detection region, and the method used is:

[0333] Obtain the line segment with the point in the fourth region three-dimensional point cloud in the second detection region and the center point of the precise hoisting load axis-aligned bounding box as end points, to obtain the intersection point of the line segment and the precise hoisting load axis-aligned bounding box, and the distance between the point in the fourth region three-dimensional point cloud in the second detection region, if less than the set distance threshold, there is a risk.

[0334] S54: When there is a collision risk in any of the first detection region and the second detection region, the crane hoisting load has a collision risk.

[0335] Specifically, according to the current center point three-dimensional coordinates and three-dimensional size of the precise hoisting load axis-aligned bounding box, the size of the current hoisting load axis-aligned bounding box is expanded outward in the -X, +X, -Y, +Y, -Z five directions of the portal coordinate system according to the set anti-collision detection region length value, to obtain the anti-collision detection region bounding box. The anti-collision detection region length value is artificially given and related to the actual needs of the project. The first region three-dimensional point cloud is cut using the anti-collision detection region bounding box and the current hoisting load axis-aligned bounding box, to obtain all three-dimensional point clouds between the two axis-aligned bounding boxes, i.e. the fourth region three-dimensional point cloud.

[0336] Finally, according to the anti-collision detection method in Figure 6 and Figure 7 , the minimum distance of all fourth region three-dimensional point clouds to the current hoisting load axis-aligned bounding box is calculated for safety identification.

[0337] Specifically, the collision avoidance detection method is divided into two cases: axial detection and spherical supplementary detection. The collision avoidance detection method first performs axial detection, and then performs spherical supplementary detection.

[0338] 1) Axial inspection results as follows Figure 6 As shown, Figure 6 The areas marked in black are the first detection areas, and the areas not marked in black are the second detection areas.

[0339] Define the nearest surface of the load's axis-aligned bounding box in the gantry crane coordinate system in the five directions of -X, +X, -Y, +Y, -Z as the nearest surface of the load's bounding box perpendicular to that direction. Figure 6 As shown in the diagram, the corresponding side surface in the -X direction is the nearest surface in the -X direction. Based on the corresponding surface in each direction as the base and with a pre-given anti-collision detection area length value as the height, a cuboid space can be constructed. The set of five cuboid spaces in the five directions (-X, +X, -Y, +Y, -Z) constitutes the first detection area. This area is a subset of the anti-collision detection area.

[0340] For all point clouds within the first detection area, the collision avoidance detection method will perform nearest-distance detection. For example, in the -X direction, it will detect the distance of all point clouds within the corresponding cuboid space in the -X direction to the corresponding surface in the X direction. ,in This represents the X-axis coordinate of the point cloud within any corresponding cuboid space. This represents the X-coordinate of the corresponding plane (this coordinate changes with different planes, for example, in the -Y direction). This represents the Y-axis coordinate of the corresponding plane, and so on. The distance to each point cloud is obtained... Then, by calculating the minimum value in this set, the nearest distance between all point clouds within the first detection area and the bounding box of the suspended object can be obtained. If the nearest distance is less than a predefined threshold, a collision risk is considered to exist.

[0341] 2) Spherical supplementary testing status as follows Figure 7 As shown.

[0342] Define the difference between the collision avoidance detection region and the first detection region as the second detection region. Within this region, the nearest distance for all point clouds cannot be directly calculated using the method described in the axial detection case above; instead, spatial distance needs to be calculated, such as... Figure 7 As shown. Taking any spatial point A as an example, the straight line segment from it to the center point B of the suspended object enclosure is... The straight line segment Two line segments obtained by cutting through the enclosure of the suspended load. These are straight line segments inside the enclosure. and external straight line segments This supplementary spherical test will take straight line segments. The length is the spatial distance. By calculating the spatial distances of all point clouds within the second detection area, the minimum value in this set is taken as the nearest spatial distance. If the nearest spatial distance is less than a predefined threshold, a collision risk is considered present.

[0343] If either the axial detection scenario or the spherical supplementary detection scenario presents a collision risk, the collision avoidance detection method will consider it an overall collision risk. In this case, an audible and visual alarm will be issued to the driver.

[0344] Specifically, for the loads lifted by shipbuilding gantry cranes, the anti-collision distance in this embodiment is a directional anti-collision measure for precise alignment of the load's axis with the surrounding box. The direction includes the sides of the surrounding box in the +X, -X, +Y, -Y, and -Z axial directions, calculated based on the nearest axial distance from the obstacle point cloud within the anti-collision range to each of the sides. For example... Figure 6 As shown. Furthermore, for all spaces below the top surface of the hoisting enclosure (excluding the five areas mentioned above), the collision avoidance distance is calculated as the straight-line distance from each point cloud (spatial point 1) to the center point of the hoisting enclosure, obtained by cutting off the intersection point from a side of the enclosure, and then to that spatial point. For example... Figure 7 As shown.

[0345] Specifically, in this embodiment, an industrial control computer with sufficient computing power is deployed on the vehicle side to collect real-time PLC data and LiDAR point cloud data. The PLC data is directly read from the PLC data processed by the CPE via the S7 protocol. The LiDAR data is used to acquire 3D point clouds based on the driver of different LiDAR models. The PLC data on the gantry crane includes essential components: the rotation angle of the rotating platform, the boom luffing, the main hook lifting height, the main hook load capacity, the auxiliary hook lifting height, and the auxiliary hook load capacity. The PLC data on the gantry crane also includes essential components: the upper trolley position, the lower trolley position, the main hook lifting height, the main hook load capacity, the auxiliary hook lifting height, the auxiliary hook load capacity, the hook lifting height, and the hook load capacity. Optional components include: trolley gear position, upper trolley gear position, lower trolley gear position, main hook gear position, auxiliary hook gear position, and hook gear position. The S7 protocol is a proprietary communication protocol developed by Siemens for its SIMATIC S7 series programmable logic controllers (PLCs). It is mainly used for data exchange and control between PLCs and HMIs (human-machine interfaces), programming software (such as TIA Portal and STEP 7), SCADA systems, and other intelligent devices.

[0346] Specifically, the laser radar point cloud data is consistent on the gantry and the portal, and is real-time three-dimensional point cloud data scanned by two laser radars from different perspectives, only the installation position and the radar model are different.

[0347] Specifically, the industrial computer end accesses data: first, based on the Robot Operation System (ROS) processing, it is converted into real-time data in the distributed system in multiple nodes, including the three-dimensional coordinates of all hooks under the shipbuilding gantry crane body under different kinematic models of the shipbuilding gantry crane (gantry and portal), the Boolean quantity of whether all hooks are loaded, and the merged three-dimensional point cloud data under the shipbuilding gantry crane coordinate system. According to the three-dimensional coordinates of all hooks, the hook load Boolean quantity and the merged three-dimensional point cloud data, the real-time three-dimensional point cloud of the hoisted load of the shipbuilding gantry crane, the accurate hoisted load axis-aligned bounding box and the hoisted object obstacle point cloud are finally segmented. The obstacle point cloud can be divided into two categories according to the requirements: on the ship body and other obstacles.

[0348] The embodiment can return the center coordinates of the hoisted load axis-aligned bounding box and whether the hoisted load and the obstacle collide to the digital twin platform based on the TCP protocol according to the requirements.

[0349] The laser radar installation scheme of the embodiment is as follows: 1) Laser radar installation scheme for portal crane:

[0350] The portal crane has two-dimensional rotation movement, and in addition to the anti-collision of the hoisted load 2, the body's hoist arm 1 also needs a larger anti-collision range, so the portal crane needs to install two wide-angle laser radars, which are the first laser radar 31 and the second laser radar 32. As shown in Figure 3 , the two laser radars are installed vertically, and each scans an area of 75° in the horizontal direction. When installing, it is necessary to ensure that the two have a certain overlapping area to facilitate mutual position calibration. As shown in Figure 2 , the laser radars each scan an area of 120° in the vertical direction. When installing, it is necessary to ensure that the lower edge of the scanning range of each laser radar can just cover the area below the portal crane. 2) Laser radar installation scheme for portal crane: The portal crane only has translational motion and only needs to realize the anti-collision of the hoisted load and the surrounding obstacles. In order to accurately obtain the three-dimensional size of the hoisted load and the surrounding obstacle information, two laser radars installed vertically on opposite sides are necessary. Figure 4 is a front view installation diagram, and the two laser radars are respectively located below the cross beams of the rigid leg side and the flexible leg side, and each laser radar scans an area of 120° in the vertical direction. Figure 5 is a top view, and each laser radar scans an area of 25.4° in the horizontal direction.

[0351] The radar arrangement of the embodiment solves the problem that the sensor detection range is limited, the installation position is fixed, only a specific direction and a local area can be monitored, and the whole coverage of the gantry crane operation space cannot be formed in the traditional infrared or ultrasonic scheme; the embodiment is not affected by environmental light, temperature, surface material and other factors, has high detection accuracy, and can adapt to the variable operation environment of the shipyard.

[0352] Meanwhile, the embodiment solves the problem that when the laser radar is installed directly above the trolley in the traditional tower crane safety monitoring scheme, it is mainly designed for the vertical lifting and rotary motion of the tower crane, and for large gantry cranes such as shipyard gantries, there is a problem that due to the large height and lateral size of large hoisted objects, if the sensor is only installed on the top of the trolley, it is difficult to cover the side of the hoisted object and the area near the ground, there is a monitoring blind area, and the spatial perception model and control logic are usually designed for tower crane structures, which is difficult to directly transplant to the operation mode of gantry cranes. In the radar installation scheme of the embodiment:

[0353] Wide adaptability to different types of gantries: based on the double laser radar scheme, different types of gantries only need to be simply changed in the installation position to ensure that the hoisted object recognition area is not blocked, and the same algorithm can be used to recognize the hoisted object and perform anti-collision.

[0354] Wide adaptability of the scheme to different types of hoisted objects: as long as the laser radar can scan the hoisted object with a certain reflectivity, regardless of the complexity of the structure and the type of the object, the size can be accurately recognized and anti-collision can be performed.

[0355] The scheme reduces the dependence on the accuracy of PLC and encoder sensors: in the scheme of the embodiment for gantry cranes, the results obtained by using high-precision laser radar scanning are used for secondary calibration of the hook position, effectively solving the problems of encoder numerical drift and inaccuracy, and achieving centimeter-level positioning accuracy in the horizontal direction.

[0356] The scheme does not require the support of a data set: the anti-collision detection of the embodiment does not require training models (such as hook position recognition or specific hoisted object type recognition) for different types of scenes, and can be directly deployed.

[0357] In summary, the arrangement scheme of the embodiment can meet the requirements of high-precision, full coverage, real-time anti-collision and size recognition in terms of dealing with complex and diverse hoisted objects in shipyards, full-space monitoring of large hoisted objects, and adaptability to the special structure of gantry cranes.

[0358] Application Example One: Anti-collision between the hoisted object below the portal crane and the surrounding lower ship body during hoisting:

[0359] Scenario and goal: When using a gantry crane for lifting operations in a shipyard, the load needs to be moved from the ground to above or around the ship body. Due to the low decks or segmented structures of the ship body during construction, the load is prone to collision during translation or descent. The goal of this application example is to detect the spatial distance between the load and the surrounding low ship body structure and other obstacles in real time, ensuring collision avoidance during lifting.

[0360] Deployment and parameters: Two wide-angle lidars are installed above the gantry crane cab 4, and an industrial computer is set up inside the cab to receive, process, and send data. The scheme will collect point cloud data in real time, PLC and encoder mechanism state, lifting height, and load weight parameters. The collision avoidance threshold is 5 meters.

[0361] Running process: During the lifting process, the system first generates a real-time three-dimensional bounding box of the load in combination with PLC data and point cloud, and continuously tracks its position and calculates the distance to the nearest obstacle point cloud in each direction. When the load is lifted near the ship body, if the distance in any anti-collision direction is less than the collision avoidance threshold, an alarm will be triggered to remind the driver to drive carefully. Finally, when the load reaches above the target position and slowly descends, when the minimum distance directly below is less than the collision avoidance threshold, an alarm will be triggered to remind the driver to drive carefully. Until the load is safely landed.

[0362] Specifically, through this application example, the system can effectively prevent the load from colliding with the ship body structure and surrounding potential obstacles during the lifting process, avoiding damage to the load or the ship body. This method achieves real-time monitoring at the centimeter level, improves the safety and reliability of lifting operations, and provides a reusable solution for safety control in complex shipbuilding scenarios.

[0363] Application example two: collision avoidance between the load below the gantry crane and the surrounding higher ship body during lifting:

[0364] Scenario and goal: When using a gantry crane for lifting operations in a shipyard, when encountering ship types with a height higher than the cab, the load needs to cross or approach the higher ship body area. At this time, not only is the load itself at risk of collision, but the boom is also prone to interference with the ship body structure during swing or luffing. The goal of this application example is to detect the distance between the load and the boom in three-dimensional space and the surrounding higher ship body structure, ensuring full-range collision avoidance under complex lifting paths.

[0365] Deployment and parameters: Two wide-angle laser radars are installed above the driver's cab of the portal crane to cover the space above the load and the ship's side. The industrial computer receives and processes point cloud and PLC data in real time, including hook position, boom angle, luffing state, and load information. In addition to calculating the bounding box of the load, the system also establishes the directional bounding box of the boom and updates its position in real time. The anti-collision threshold is set to 5 meters.

[0366] Operation process: During the lifting process, the system generates the load bounding box and the boom directional bounding box based on PLC parameters and point cloud data, and continuously tracks the spatial relationship between the two and the ship's point cloud. When the load moves close to the higher ship, the system calculates the closest distance between the bounding box and the ship's point cloud; if the minimum distance between any part of the load or boom and the ship is less than the set threshold, an alarm will be triggered immediately to remind the driver to adjust the operation. During the entire lifting process, the system will monitor the spatial position of the boom and the load simultaneously until the load safely passes through or reaches the target area.

[0367] Specifically, through this application example, the system not only prevents the load from colliding with the high ship during movement, but also detects the safety space of the boom in large-angle luffing or rotation in real time, avoiding accidents caused by interference between the boom and the ship. This method realizes the joint monitoring of the boom and the load, further improving the safety and robustness of the lifting operation, and providing reliable protection for adapting to various shipbuilding body heights and complex environments.

[0368] Application example three: Anti-collision between the load below the portal crane and the shipbuilding body during lifting:

[0369] Scene and target: When using a portal crane for lifting operations in a shipyard, the load is usually larger and heavier, and needs to be placed directly on the designated location of the ship. In such scenarios, the safe distance between the load and the lower and surrounding shipbuilding bodies is particularly critical, as a collision could cause serious damage to the load or the ship. The goal of this application example is to detect the minimum distance between the load and the lower shipbuilding body in real time, ensuring smooth and safe positioning during lifting.

[0370] Deployment and parameters: One laser radar is installed below each of the portal crane's beam rigid leg and flexible leg beam, covering the entire field of view of the load. The industrial computer collects point cloud data and PLC parameters in real time, including the positions of the upper and lower trolleys and the lifting height of the hook. Since the laser radars are arranged on the opposite side, they can effectively reduce the obstruction of the load during scanning, and even obtain point cloud information below the load. The anti-collision threshold is also set to 5 meters.

[0371] Operation process: During the lifting process, the system generates a real-time three-dimensional bounding box of the lifted object according to the PLC data and the point cloud, and continuously calculates the minimum distance of the bounding box to the surrounding obstacle point cloud in each anti-collision direction. When the lifted object gradually moves near the shipbuilding body, the system can use the perspective advantage of the dual laser radar to obtain the point cloud information under the lifted object, thereby ensuring safety in all directions, especially in the downward direction. When the minimum distance in any direction is less than the anti-collision threshold, the system will trigger an alarm to remind the driver to adjust the operation, and ultimately ensure the smooth and safe positioning of the lifted object.

[0372] Specifically, through this application example, the system can effectively avoid the collision between the gantry crane and the shipbuilding body below during the lifting of large-volume and heavy-weight lifted objects. The installation scheme significantly reduces point cloud occlusion and can scan the bottom area of the lifted object, thereby achieving higher precision real-time monitoring. Ultimately, this method ensures the safety and reliability of large-scale lifting operations and provides a replicable solution for anti-collision in high-risk scenarios for shipyards. It is especially suitable for port and shipbuilding application scenarios and can meet the requirements for three-dimensional size recognition of lifted objects and anti-collision detection between the shipbuilding gantry crane body, the lifted object, and the surrounding potential obstacles. It can achieve lifted object anti-collision safety functions during the operation of two different configurations of shipbuilding gantries. And the identification and anti-collision results can be combined with the digital twin platform to achieve higher-dimensional safety protection for shipbuilding construction operations.

[0373] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or make equivalent substitutions for part or all of the technical features; and these modifications or substitutions do not make the essence of the corresponding technical solution deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A crane load collision detection method based on real-time three-dimensional point cloud driving, characterized in that, The method comprises the following steps: S1: obtaining a first region three-dimensional point cloud in a working area of a shipbuilding portal crane and a three-dimensional position coordinate of a hook of the shipbuilding portal crane; wherein the shipbuilding portal crane is a portal crane or a portal and seated crane; S2: determining a hoisted load three-dimensional detection region according to the three-dimensional position coordinate of the hook to obtain a three-dimensional point cloud in the hoisted load three-dimensional detection region, i.e. a third region three-dimensional point cloud; S3: obtaining a plurality of point cloud clusters based on a clustering algorithm according to the third region three-dimensional point cloud in the hoisted load three-dimensional detection region, obtaining scores of the plurality of point cloud clusters, and then obtaining a point cloud cluster with the lowest score, and finally obtaining an optimal point cloud cluster; S4: obtaining an accurate hoisted load axis-aligned bounding box according to the optimal point cloud cluster to determine the size of the accurate hoisted load axis-aligned bounding box; S5: determining a collision avoidance detection region bounding box based on the size of the accurate hoisted load axis-aligned bounding box to obtain a three-dimensional point cloud between the accurate hoisted load axis-aligned bounding box and the collision avoidance detection region bounding box according to the first region three-dimensional point cloud, i.e. a fourth region three-dimensional point cloud, and performing collision avoidance detection according to the fourth region three-dimensional point cloud; The center of the collision avoidance detection region bounding box coincides with the center and center line of the accurate hoisted load axis-aligned bounding box, and the size of the collision avoidance detection region bounding box is greater than the size of the accurate hoisted load axis-aligned bounding box; The S5 comprises: S51: determining a detection region between the accurate hoisted load axis-aligned bounding box and the collision avoidance detection region bounding box according to the accurate hoisted load axis-aligned bounding box and the collision avoidance detection region bounding box to determine a first detection region and a second detection region; The first detection region comprises: respectively taking a bth side of the accurate hoisted load axis-aligned bounding box as a bottom face, b=1, 2, 3, 4, and four cuboids are arranged on the corresponding side of the collision avoidance detection region bounding box; taking the bottom face of the accurate hoisted load axis-aligned bounding box as a bottom face, and a fifth cuboid is arranged on the bottom face of the collision avoidance detection region bounding box; The second detection region is a region outside the first detection region between the accurate hoisted load axis-aligned bounding box and the collision avoidance detection region bounding box; S52: determining whether there is a collision risk in the first detection region, and the method used is: if a point in the fourth region three-dimensional point cloud in the bth cuboid, b=1, 2, 3, 4, has a distance less than a set distance threshold from the bth side of the accurate hoisted load axis-aligned bounding box, then there is a collision risk; if a point in the fourth region three-dimensional point cloud in the fifth cuboid has a distance less than a set distance threshold from the bottom face of the accurate hoisted load axis-aligned bounding box, then there is a collision risk; S53: determining whether there is a collision risk in the second detection region, and the method used is: Obtaining a line segment with an end point being a point in the fourth region three-dimensional point cloud in the second detection region and a center point of the accurate hoist load axis-aligned bounding box, to obtain an intersection point of the line segment and the accurate hoist load axis-aligned bounding box and a distance between the point in the fourth region three-dimensional point cloud in the second detection region, if less than a set distance threshold, there is a risk; S54: When any region of the first detection region and the second detection region has a collision risk, the crane hoist load has a collision risk.

2. The crane load collision detection method based on real-time three-dimensional point cloud driving according to claim 1, characterized in that, When the shipbuilding gantry crane is a gantry crane, the three-dimensional position coordinates of the hook of the shipbuilding gantry crane are obtained as follows: S11: The theoretical three-dimensional position coordinates of the hook are obtained as follows: In the formula: represents the coordinate of the theoretical position of the hook in the X direction; represents the coordinate of the theoretical position of the hook in the Y direction; represents the coordinate of the theoretical position of the hook in the Z direction; represents the X direction coordinate of the trolley coordinate system origin; represents the Y direction coordinate of the trolley coordinate system origin; represents the real-time height of the hook bottom; represents the real-time position of the trolley in the X direction, i.e. the real-time position of the trolley along the trolley track direction; represents the offset of the hook in the X direction; represents the offset of the hook in the Y direction; S12: According to the XY plane where the maximum Z-direction coordinate of the hook top is located, the first region three-dimensional point cloud is segmented to obtain the second region three-dimensional point cloud above the XY plane where the maximum Z-direction coordinate of the hook top is located; S13: clustering the second region three-dimensional point cloud to obtain M sling point cloud clusters, to obtain a cluster center coordinate of an mth sling point cloud cluster ; wherein m is an index of the sling point cloud cluster; M is the total number of the sling point cloud clusters; respectively, are coordinates of the cluster center of the mth sling point cloud cluster in the X direction, the Y direction and the Z direction. S14: When the number M of the hoisting rope point cloud clusters is less than the number N of the hooks: S141: Obtain the distance between the coordinate of the cluster center of the first hanging rope point cloud cluster in the XY plane and the coordinate of the theoretical three-dimensional position of all hooks in the XY plane ; S142: When At the minimum time, the final three-dimensional position coordinates of the n th hook at this time are as follows: In the formula: X, Y, Z are the three-dimensional position coordinates of the first hook in the X, Y, Z directions, respectively; X1, Y1 are the coordinates of the first sling point cloud cluster in the X, Y directions, respectively; Zn represents the theoretical position coordinate of the n th hook in the Z direction. S143: sequentially based on the 2, 3 M hanging rope point cloud clusters, repeatedly perform S141-S142; S144: At this time, the final three-dimensional position coordinates of the remaining N-M hooks are as follows: ； S15: When the number M of the hoisting rope point cloud clusters is greater than or equal to the number N of the hooks: the distance between the coordinate of the theoretical three-dimensional position of the n-th hook in the XY plane and the coordinate of the cluster center of all the sling point cloud clusters in the XY plane ; When At the minimum time, the final three-dimensional position coordinates of the nth hook are acquired as follows: 。 3. The crane load collision detection method based on real-time three-dimensional point cloud driving according to claim 1, characterized in that, When the shipbuilding gantry crane is a gantry crane, the three-dimensional position coordinates of the hook are obtained as follows; S111: The position coordinates of the crane arm after relative rotation of the cabin are obtained: wherein: respectively the coordinates in the X direction and in the Y direction of the position of the crane boom after the rotation relative to the cabin; denotes the transpose; respectively the coordinates in the X direction and in the Y direction of the initial position of the crane boom relative to the cabin; is a two-dimensional rotation matrix of the crane around the Z axis; is a two-dimensional rotation angle of the cabin around the Z axis; S112: The end displacement of the hook in the X-Y plane in the direction of extension of the crane boom is obtained; wherein: denotes the end displacement of the hook in the X-Y plane in the direction of extension of the crane jib; denotes the two-dimensional rotation angle of the cabin around the Z axis; denotes the pitch angle of the jib; denotes the jib length; denotes the component in the X direction; denotes the component in the Y direction; S113: The three-dimensional position coordinates of the hook are obtained as follows: In the formula: is the X-direction position coordinate of the cabin relative to the original point of the door machine coordinate system; is the Y-direction position coordinate of the cabin relative to the original point of the door machine coordinate system; The expansion is expressed as an explicit formula as follows: In the formulae: denotes the position coordinate of the hook in the X direction; denotes the position coordinate of the hook in the Y direction; denotes the position coordinate of the hook in the Z direction.

4. The crane load collision detection method based on real-time three-dimensional point cloud driving according to claim 1, characterized in that, When the shipbuilding gantry crane is a gantry crane, the hoist load three-dimensional detection region is obtained as follows: In the formula: is a three-dimensional detection area of a load; is a coordinate of an X direction of any point in the three-dimensional detection area of the load in a global coordinate system; is a coordinate of a Y direction of any point in the three-dimensional detection area of the load in the global coordinate system; is a coordinate of a Z direction of any point in the three-dimensional detection area of the load in the global coordinate system; is a three-dimensional space; represents a position coordinate of the hook in the X direction; represents a position coordinate of the hook in the Y direction; represents a position coordinate of the hook in the Z direction; is a ground height value in the Z axis direction in the global coordinate system; is a ground clearance height value; represents a transpose; wherein, where: is the actual half spread in the X direction; is the actual half spread in the Y direction; is a scaling function that varies with weight segment; is the un-scaled reference half spread in the X direction; is the un-scaled reference half spread in the Y direction; wherein: is the current load weight; is a scaling factor; is a scaling limit value.

5. The crane load collision detection method based on real-time three-dimensional point cloud driving according to claim 4, characterized in that, When the shipbuilding gantry crane is a gantry crane, the hoist load three-dimensional detection region is: When the number of hooks is 1, the hoist load three-dimensional detection region is: In the formula: is the spread distance of the three-dimensional detection area of the hoisted load set in the X direction; is the spread distance of the three-dimensional detection area of the hoisted load set in the Y direction; is the spread distance of the three-dimensional detection area of the hoisted load set in the Z direction; When the number of hooks is 2, the hoist load three-dimensional detection region is: wherein: is the position coordinate of the first hook in the X direction; is the position coordinate of the second hook in the X direction; is the position coordinate of the first hook in the Y direction; is the position coordinate of the second hook in the Y direction; is the position coordinate of the first hook in the Z direction; is the position coordinate of the second hook in the Z direction; is the maximum value of the coordinates of the first hook and the second hook in the Z direction; When the number of hooks is greater than 2, the hoist load three-dimensional detection region is obtained as follows: First, the minimum rectangular region of the hook in the XY plane under the global coordinate system is obtained: wherein: In the formula: is the index of the hook; is the total number of hooks in the hoist load; is the X-direction coordinate of the th hook in the global coordinate system; is the Y-direction coordinate of the th hook in the global coordinate system; is the minimum rectangular frame in the XY plane; is the minimum X-direction coordinate of the hooks in all hoist loads in the global coordinate system; is the maximum X-direction coordinate of the hooks in all hoist loads in the global coordinate system; is the minimum Y-direction coordinate of the hooks in all hoist loads in the global coordinate system; is the maximum Y-direction coordinate of the hooks in all hoist loads in the global coordinate system; denotes transposition; is a two-dimensional space; Then, the hoist load three-dimensional detection region at this time is obtained as follows: wherein, In the formula: is the maximum Z coordinate of all hooks in the global coordinate system.

6. The crane load collision detection method based on real-time three-dimensional point cloud driving according to claim 1, characterized in that, The method for obtaining the score of the point cloud cluster is as follows: S31: Based on the third region three-dimensional point cloud in the hoist load three-dimensional detection region, a plurality of point cloud clusters are obtained based on a clustering algorithm to obtain a point cloud cluster center point position and a point cloud cluster size; The position of the point cloud cluster center point is obtained as follows: In the formula: is the position coordinate of the point cloud cluster center point; is the coordinate of the position of the current point cloud cluster center point in the X direction, Y direction, and Z direction, respectively; is the minimum and maximum value of the X direction coordinate of the axis-aligned bounding box of the point cloud cluster, respectively; is the minimum and maximum value of the Y direction coordinate of the axis-aligned bounding box of the point cloud cluster, respectively; is the minimum and maximum value of the Z direction coordinate of the axis-aligned bounding box of the point cloud cluster, respectively; The size of the point cloud cluster is obtained as follows: In the formula: is the size of the axis-aligned bounding box of the point cloud cluster at the current time point in the X direction, that is, the width of the point cloud cluster at the current time point; is the size of the axis-aligned bounding box of the point cloud cluster at the current time point in the Y direction, that is, the length of the point cloud cluster at the current time point; is the size of the axis-aligned bounding box of the point cloud cluster at the current time point in the Z direction, that is, the height of the point cloud cluster at the current time point; S32: Obtain the Euclidean distance between the current time point cloud cluster center point position and the center point position of the last time's hoisted object; to obtain the normalized upper bound of the Euclidean distance between the axis-aligned bounding box of the point cloud cluster and the axis-aligned bounding box center point of the target hoisted object identified last time ; The formula for obtaining the Euclidean distance between the current time point cloud cluster center point position and the center point position of the hoist load at the last time is as follows: In the formula, distance is the Euclidean distance between the axis-aligned bounding box of the point cloud cluster and the axis-aligned bounding box center point of the target load recognized last time, that is, the Euclidean distance between the center point position of the current point cloud cluster and the center point position of the load recognized last time; represents square root; , , X direction, Y direction and Z direction coordinates of the center point position of the current point cloud cluster respectively; , , X direction, Y direction and Z direction coordinates of the center point position of the target load bounding box at the last moment respectively; The formula for obtaining the normalized upper bound of the Euclidean distance between the axis-aligned bounding box of the point cloud cluster and the axis-aligned bounding box center point of the target hoist load recognized last time is as follows: In the formula: is a normalized upper bound of the Euclidean distance between the axis-aligned bounding box center point of the point cloud cluster and the axis-aligned bounding box center point of the last identified target hoisted object; is the Euclidean distance between the axis-aligned bounding box center point of the point cloud cluster and the axis-aligned bounding box center point of the last identified target hoisted object; is the Euclidean distance between the axis-aligned bounding box center point of the point cloud cluster and the axis-aligned bounding box center point of the last identified target hoisted object; is the index of the point cloud cluster; is the total number of point cloud clusters; S33: Obtain the axis-aligned bounding box three-dimensional size difference value of the current time point cloud cluster and the axis-aligned bounding box of the target hoisted object at the previous time: to obtain the normalized upper bound of the three-dimensional size difference value of the current point cloud cluster and the three-dimensional size of the axis-aligned bounding box of the target hoisted object at the previous time ; The formula for obtaining the difference value of the three-dimensional size of the axis-aligned bounding box of the current time point cloud cluster and the three-dimensional size of the axis-aligned bounding box of the target hoist load at the last time is as follows: In the formula: is the difference between the three-dimensional size of the current point cloud cluster and the three-dimensional size of the axis-aligned bounding box of the target hoisted object at the previous time; is the width, length and height of the axis-aligned bounding box of the target hoisted object at the previous time, respectively, i.e. the width, length and height of the axis-aligned bounding box of the target hoisted object at the previous time. The formula for obtaining the normalized upper bound of the difference value of the three-dimensional size of the current time point cloud cluster and the three-dimensional size of the axis-aligned bounding box of the target hoist load at the last time is as follows: In the formula: is the normalized upper bound of the difference between the three-dimensional size of the current point cloud cluster and the three-dimensional size of the axis-aligned bounding box of the target hoisted object at the previous time; S34: Obtain the Euclidean distance from the axis-aligned bounding box center point of the point cloud cluster to the center point of the upper surface of the three-dimensional detection region of the hoisted load: to obtain the normalized upper bound of the Euclidean distance from the axis-aligned bounding box center point of the point cloud cluster to the center point of the upper surface of the three-dimensional detection region of the hoisted load ; The formula for obtaining the Euclidean distance from the axis-aligned bounding box center point of the point cloud cluster to the center point of the upper surface of the three-dimensional detection area of the hoisted object is as follows: In the formula: is the Euclidean distance from the axis-aligned bounding box center point of the point cloud cluster to the center point of the upper surface of the load three-dimensional detection area; , , are the X direction, Y direction, and Z direction coordinates of the center point of the upper surface of the load three-dimensional detection area at the current time, respectively. The formula for obtaining the normalized upper bound of the Euclidean distance from the axis-aligned bounding box center point of the point cloud cluster to the center point of the upper surface of the three-dimensional detection area of the hoisted object is as follows: In the formula: is the normalized upper bound of the Euclidean distance from the axis-aligned bounding box center point of the point cloud cluster to the center point of the upper surface of the three-dimensional detection region of the load. S35: Obtain the Euclidean distance between the axis-aligned bounding box of the normalized point cloud cluster and the axis-aligned bounding box center point of the target hoisted object identified last time, the three-dimensional size difference between the axis-aligned bounding box of the normalized point cloud cluster and the axis-aligned bounding box of the target hoisted object at the last moment, and the Euclidean distance from the axis-aligned bounding box center point of the normalized point cloud cluster to the center point of the upper surface of the three-dimensional detection area of the hoisted object, using the formula as follows: In the formula: is the Euclidean distance between the axis-aligned bounding box center point of the normalized point cloud cluster and the axis-aligned bounding box center point of the target load recognized last time; is the three-dimensional size difference between the axis-aligned bounding box of the normalized point cloud cluster and the axis-aligned bounding box of the target load at the last time; is the Euclidean distance from the axis-aligned bounding box center point of the normalized point cloud cluster to the center point of the upper surface of the three-dimensional detection region of the load; is a parameter for ensuring that the denominator is not zero; S36: When the size of the point cloud cluster is greater than the size threshold, obtain the score of the point cloud cluster, using the formula as follows: wherein: are weight values, respectively. are weight values, respectively.

Images