A method for predicting the movement trajectories of group tower cranes to prevent collisions based on an attribute-enhanced PointLSTM model

Abstract

The present invention relates to a method for predicting the movement trajectories of tower cranes to prevent collisions based on an attribute-enhanced PointLSTM model, belonging to the technical field of tower crane safety control. This method uses lidar to detect the surrounding environment of the tower crane in real time to form three-dimensional point cloud data. At the same time, the spatial characteristics of the tower crane such as the boom length, hook height, trolley position, etc. are obtained through tower crane sensors. According to the above characteristic information, the relative position information of the point cloud is calculated to establish an attribute-enhanced feature matrix. Then, the PointLSTM model is used to learn and analyze the spatial and temporal characteristics of the above feature matrix, so as to realize the prediction of the future movement trajectory of the point cloud and analyze through the point cloud data coincidence degree and the change trend of the coincidence degree, so as to judge whether a collision event is about to occur, significantly improving the solution efficiency and accuracy of the tower crane anti-collision problem.

Description

Technical Field

[0001] The present invention relates to a method for predicting the movement trajectories of tower cranes to prevent collisions based on an attribute-enhanced PointLSTM model, belonging to the technical field of tower crane safety control. Background Art

[0002] In the research and application of the problem of preventing collisions between tower cranes, the general method is based on the dynamic and static parameters of the tower cranes collected by sensors, including the size parameters, movement parameters, and lifted object parameters of the tower cranes, etc. By calculating the distance information between tower cranes through three-dimensional space data modeling and setting a safety threshold, a warning or intervention shutdown operation is performed when the calculation result is less than the threshold, so as to solve the problem of preventing collisions between tower cranes. The above method mainly has two problems:

[0003] 1. The general method has a lag in taking warning measures after calculating the distance information between tower cranes, and does not consider the characteristic information of the relative movement of tower cranes in the time series. In order to ensure that warning measures are taken in a timely manner, it is necessary to strictly set the safety threshold, and it is impossible to avoid misjudgment caused by too high a safety threshold.

[0004] 2. Due to the changes in the movement trajectories of tower cranes and the sizes of lifted objects, some methods do not bring the size of the lifted object into the three-dimensional space data modeling calculation process, resulting in the situation where collisions between the lifted object and the tower crane during operation cannot be avoided; other methods add the parameters of each tower crane hoisting task, including the task trajectory, the size of the lifted object, etc., to the model for input calculation. Obviously, this is not an end-to-end detection method, which has a greater impact on the task operation complexity, work cost, and efficiency. Summary of the Invention

[0005] To solve the technical problems existing in the prior art, the present invention provides a method for predicting the movement trajectories of tower cranes to prevent collisions based on an attribute-enhanced PointLSTM model.

[0006] To achieve the above object, the technical solution adopted by the present invention is a method for predicting the movement trajectories of tower cranes to prevent collisions based on an attribute-enhanced PointLSTM model, which is specifically operated according to the following steps:

[0007] S1. Install a lidar at the front end of the tower arm, and collect the three-dimensional point cloud environment data under and in front of its own tower arm during the operation of the tower crane. That is, the point cloud data at time t is P t ∈R n*3 , where n*3 is the number of point clouds * three-dimensional space information;

[0008] S2. Obtain the spatial characteristics of the tower crane such as the height of the high tower hook and the position of the trolley through the tower crane sensor, and calculate the relative planar distance, relative height distance, and spatial distance between the point cloud data and the high tower hook and the trolley. That is, the point cloud spatial attribute enhancement feature matrix at time t is Xt ∈R n*5 , where n * 5 is the number of point clouds * the number of attributes;

[0009] S3. Concatenate the above data in a time series to form spatio-temporal input data P ∈ R T*N*3 , X ∈ R T*N*5 , (P, X) ∈ R T*N*8 , where T is the dimension of the time series node number, and N is the number of points in the point cloud data at time t;

[0010] S4. Use the point cloud long short-term memory network model PointLSTM to analyze the spatial and temporal features of the spatio-temporal input data (P, X) to obtain the trajectory prediction result;

[0011] S5. Analyze the point cloud trajectory prediction results from time t + 1 to time t + ∆t, judge the degree of point cloud overlap and trend, so as to judge whether there is a collision situation and take warning measures in advance.

[0012] Preferably, in step S2, when calculating the relative plane distance, relative height distance and spatial distance between the point cloud data and the high tower hook and trolley, the calculation is as follows:

[0013] a. Let the three-dimensional spatial information of point i in the point cloud at time t be (x1, y1, z1), let the position of the high tower hook be (x2, y2, z2), and the position of the high tower trolley be (x2, y2, z3), then the relative plane distance of a certain point at time t from the high tower trolley and hook can be obtained: ;

[0014] The relative height distance from the high tower hook: ;

[0015] The relative height distance from the high tower trolley: ;

[0016] The spatial distance from the high tower hook: ;

[0017] The spatial distance from the high tower trolley: ;

[0018] b. Concatenate the above relative distance features to obtain the attribute enhancement feature matrix of point i at time t: , that is, the attribute enhancement feature matrix of the point cloud at time t is X t ∈R n*5 , n * 5 is the number of point clouds * the number of attributes.

[0019] Preferably, in step S4, the calculation process of the point cloud long short-term memory network model is as follows:

[0020] ;

[0021] ;

[0022] ;

[0023] ;

[0024] ;

[0025] ;

[0026] ;

[0027] where is the input gate, is the forget gate, is the output gate, is the spatio-temporal feature of the point cloud input at time t, is the spatio-temporal feature of the point cloud at the previous time, is the state passed from the previous time to this time, , , , , are the parameter matrices of the corresponding functions respectively; , are activation functions, is the state of the previous unit, is the control input of the current unit, is the state of the current unit, is the hidden state of the current unit, and also the output at the current time, that is, the motion prediction of the point cloud data.

[0028] Meanwhile, when performing PointLSTM calculation, it is necessary to calculate the previous unit , that is, the correlation between the previous unit and the current input P t , and then calculate the state of the current unit and the hidden state of the current unit through the gate control method, which is also the output at the current time, that is, the motion prediction of the point cloud data.

[0029] Preferably, the PointRNN model structure and calculation process:

[0030] ;

[0031] Due to the disorder of the point cloud data, for each point in , the k-NN method is used to select the k nearest points ([[]] ) based on the spatio-temporal proximity principle during the calculation process; secondly, for ( , ( ) are concatenated to obtain the intermediate feature at the current moment ;

[0032] ;

[0033] Finally, the k features are comprehensively obtained through a fully connected layer and a pooling operation to obtain ( );

[0034] Thus, the model result can be obtained:

[0035] ;

[0036] ;

[0037] Among them is the spatio-temporal neighboring point set i of , is the point cloud feature matrix of the current point, is the attribute enhanced feature matrix of the current point, is the state of the adjacent point at the previous moment, is the point cloud feature matrix of the adjacent point at the previous moment.

[0038] Compared with the prior art, the present invention has the following technical effects: The present invention forms three-dimensional point cloud data by real-time detecting the surrounding environment of the tower crane through a lidar, and at the same time obtains the spatial features of the tower crane such as the boom length, hook height, trolley position, etc. through the tower crane sensors. According to the above feature information, the relative position information of the point cloud is calculated to establish an attribute enhanced feature matrix. Then, the PointLSTM model is used to learn and analyze the spatial features and temporal features of the above feature matrix, so as to realize the prediction of the future point cloud movement trajectory and analyze through the point cloud data coincidence degree and the change trend of the coincidence degree, so as to judge whether a collision event is about to occur. This method fully considers the influence of operation variables such as the construction environment, the size of the suspended object, and the task trajectory, effectively reduces the operation complexity of the analysis task, and is an end-to-end anti-collision monitoring method; at the same time, the prediction of the movement trajectory can ensure that early warning measures are taken in advance, effectively solve the defects and problems existing in the above existing general methods, and significantly improve the solution efficiency and accuracy of the tower crane anti-collision problem. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 is a schematic diagram of the position of the tower crane in the present invention.

[0040] Figure 2 is a three-dimensional spatial diagram of the relative distance attribute of the present invention.

[0041] Figure 3 Schematic diagram of the attribute enhancement feature matrix of the present invention.

[0042] Figure 4 Schematic diagram of the PointLSTM model architecture of the present invention. Detailed implementation manners

[0043] In order to make the technical problems, technical solutions and beneficial effects to be solved by the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0044] A method for predicting the movement trajectory of tower cranes to prevent collision based on an attribute-enhanced PointLSTM model is specifically operated according to the following steps.

[0045] S1. Determine whether a collision occurs in the relative movement of the high tower crane and the low tower crane. As Figure 1 shown, install a lidar at the front end of the tower arm, and collect the three-dimensional point cloud environment data under and in front of its own tower arm during the operation of the tower crane. That is, the point cloud data at time t is P t ∈R n*3 , where n*3 is the number of point clouds * three-dimensional space information;

[0046] S2. Obtain the spatial characteristics of the tower crane such as the height of the high tower crane hook and the position of the trolley through the tower crane sensor, and calculate the relative planar distance, relative height distance and spatial distance between the point cloud data and the high tower crane hook and the trolley. That is, the spatial attribute enhancement feature matrix of the point cloud at time t is X t ∈R n*5 , where n*5 is the number of point clouds * the number of attributes;

[0047] S3. Concatenate the above data in a time series to form spatio-temporal input data as P∈R T*N*3 , X∈R T*N*5 , (P, X)∈R T*N*8 , where T is the dimension of the time series node number, and N is the number of points in the point cloud data at time t;

[0048] S4. Use the point cloud long short-term memory network model PointLSTM to analyze the spatial and temporal characteristics of the spatio-temporal input data (P, X) to obtain the trajectory prediction result;

[0049] S5. Analyze the point cloud trajectory prediction results from time t+1 to time t+∆t, judge the degree of point cloud overlap and trend, so as to judge whether there is a collision situation and take warning measures in advance.

[0050] Among them, as Figure 2 , Figure 3As shown in the figure, when calculating the relative planar distance, relative height distance, and spatial distance between the point cloud data and the high tower crane hook and trolley, the calculations are as follows:

[0051] a. Let the three-dimensional spatial information of point i in the point cloud at time t be (x1, y1, z1). Since the relative planar positions of the trolley and the hook are the same, and there is only a height difference. Let the position of the high tower crane hook be (x2, y2, z2), and the position of the high tower trolley be (x2, y2, z3). Then, the relative planar distance between a certain point at time t and the high tower trolley and hook can be obtained:

[0052] ;

[0053] The relative height distance from the high tower crane hook: ;

[0054] The relative height distance from the high tower trolley: ;

[0055] The spatial distance from the high tower crane hook: ;

[0056] The spatial distance from the high tower trolley: ;

[0057] b. Concatenate the above relative distance features to obtain the attribute enhancement feature matrix of point i at time t: , that is, the attribute enhancement feature matrix of the point cloud at time t is X t ∈R n*5 , where n*5 is the number of point clouds * the number of attributes.

[0058] During the operation of the tower crane, the high tower crane always remains relatively stationary with the hook when lifting objects, and there is only a change in the relative height distance with the trolley; while the relative height distance between the tower body structure of the low tower and the high tower trolley always remains unchanged, and the distances between the low tower hook and the lifted object and the high tower will all change significantly. Therefore, the above attribute enhancement feature matrix of the point cloud is beneficial for the model to identify the relevant points of the low tower that may collide, making the collision prediction more accurate.

[0059] As Figure 4 shown in the figure, the calculation process of the point cloud long short-term memory network model is as follows:

[0060] ;

[0061] ;

[0062] ;

[0063] ;

[0064] ;

[0065] ;

[0066] ;

[0067] where is the input gate, is the forget gate, is the output gate, is the spatio-temporal feature of the point cloud input at time t, is the spatio-temporal feature of the point cloud at the previous time, is the state passed from the previous time to this time, , , , , are the parameter matrices of the corresponding functions respectively; , are activation functions, is the state of the previous unit, is the control input of the current unit, is the state of the current unit, is the hidden state of the current unit, and also the output at the current time, that is, the motion prediction of the point cloud data.

[0068] At the same time, when performing PointLSTM calculation, it is necessary to calculate the previous unit , that is, the correlation between the previous unit and the current input P t , and then calculate the state of the current unit and the hidden state of the current unit through the gate control method, which is also the output at the current time, that is, the motion prediction of the point cloud data.

[0069] Among them, the PointRNN model structure and calculation process: ,

[0070] Due to the disorder of the point cloud data, for each point in , the k nearest points ([[]] ) are selected based on the spatio-temporal proximity principle during the calculation; secondly, ([[]] , ( )) are concatenated to obtain the intermediate feature at the current time;

[0071] ;

[0072] Finally, the k features are synthesized through the fully connected layer and pooling operation to obtain ([[]] ).

[0073] Thus, the model results can be obtained as follows:

[0074] ;

[0075] ;

[0076] wherein is the spatio-temporal proximity point set i , is the point cloud feature matrix of the current point, is the attribute enhanced feature matrix of the current point, is the state of the adjacent point at the previous moment, is the point cloud feature matrix of the adjacent point at the previous moment.

[0077] First, the present invention proposes a new relative spatial distance attribute feature for the problem of tower crane anti-collision, which effectively reduces the difficulty of identifying objects that may collide. Secondly, the method adopts an end-to-end anti-collision prediction method of the PointLSTM model, and the temporal prediction of the point cloud movement trajectory in three-dimensional space can ensure that early warning measures are taken in advance, significantly improving the solution efficiency and accuracy of the tower crane anti-collision problem.

[0078] The above are only the preferred embodiments of the present invention, and are not intended to limit the present invention. Any modifications, equivalent replacements, and improvements made within the spirit and principle of the present invention shall be included within the scope of the present invention.

Claims

1. A method for predicting the movement trajectory of tower crane anti-collision based on an attribute-enhanced PointLSTM model, characterized in that: The specific operations are carried out according to the following steps: S1. Install a lidar at the front end of the tower arm, and collect the 3D point cloud environmental data below and in front of its own tower arm during the operation of the tower crane. That is, the point cloud data at time t is P t ∈R n*3 , where n*3 is the number of point clouds * three-dimensional space information; S2. Obtain the spatial characteristics of the tower crane such as the height of the high tower crane hook and the position of the trolley through the tower crane sensors, and calculate the relative planar distance, relative height distance and spatial distance between the point cloud data and the high tower crane hook and trolley. That is, the enhanced feature matrix of the point cloud spatial attributes at time t is X t ∈R n*5 , where n*5 is the number of point clouds * the number of attributes; S3. Concatenate the above data in a time series to form spatio-temporal input data \(P\in\mathbb{R}\) T*N*3 , \(X\in\mathbb{R}\) T*N*5 , \((P, X)\in\mathbb{R}\) T *N*8 , where \(T\) is the dimension of the time series node number, and \(N\) is the number of points in the point cloud data at time \(t\); S4. Use the point cloud long short-term memory network model PointLSTM to analyze the spatial and temporal features of the spatio-temporal input data (P, X) to obtain the trajectory prediction result; S5. Analyze the point cloud trajectory prediction results from the (t + 1)th moment to the (t + ∆t)th moment, judge the degree of point cloud overlap and trend, so as to judge whether there is a collision situation and take warning measures in advance; In the step S4, the calculation process of the point cloud long short-term memory network model is as follows: ； ； ； ； ； ； ； Among them is the input gate, is the forget gate, is the output gate, is the spatio-temporal feature of the point cloud input at time t, is the spatio-temporal feature of the point cloud at the previous time, is the state transmitted from the previous time to this time, , , , , are the parameter matrices of the corresponding functions respectively; , are activation functions, is the state of the previous unit, is the control input of the current unit, is the state of the current unit, is the hidden state of the current unit, which is also the output at the current time, that is, the motion prediction of the point cloud data; Meanwhile, when performing PointLSTM calculations, it is necessary to calculate the previous unit That is, the relevance between the previous unit and the current input P t Subsequently, the current unit state is calculated through the gate control method , the hidden state of the current unit , which is also the output at the current moment, that is, the motion prediction of the point cloud data.

2. The method for predicting the movement trajectory of tower crane anti-collision based on the attribute-enhanced PointLSTM model according to claim 1, wherein: In the step S2, when calculating the relative planar distance, relative height distance and spatial distance between the point cloud data and the high tower hook and trolley, the calculation is as follows: a. Let the three-dimensional spatial information of the i-th point in the point cloud at the t-th moment be (x1, y1, z1), let the position of the high tower hook be (x2, y2, z2), and the position of the high tower trolley be (x2, y2, z3), then the relative planar distance between a certain point at the t-th moment and the high tower trolley and hook can be obtained: ； Relative height distance from the high tower crane hook: ; Relative height distance from the high tower trolley: ; Spatial distance from the high tower crane hook: ; Spatial distance from the high tower trolley: ; b. Concatenate the above relative distance features to obtain the attribute enhanced feature matrix of the i-th point at the t-th moment: , the enhanced feature matrix of the point cloud attributes at time t is obtained as X t ∈R n*5 , where n*5 is the number of point clouds * the number of attributes.

3. A method for predicting the movement trajectory of tower crane anti-collision based on an attribute-enhanced PointLSTM model according to claim 1, characterized in that: The structure and calculation process of the PointRNN model: ； Due to the disorderliness of the point cloud data, for each point in it, the k-NN method is used to select the k nearest points ([[]] ) based on the spatio-temporal proximity principle during the calculation process; secondly, ([[]] , ( )) are spliced to obtain the intermediate feature at the current moment; ； Finally, the k features are comprehensively obtained through a fully connected layer and a pooling operation ( ); Thus, the model result can be obtained: ； ； Among them is the spatio-temporal neighboring point set i , is the point cloud feature matrix of the current point, is the attribute enhancement feature matrix of the current point, is the state at the previous moment of the adjacent point, is the point cloud feature matrix of the adjacent point at the previous moment.

Images