Hook track prediction method and device based on Internet of Things perception

Abstract

The embodiment of the invention provides a lifting hook track prediction method and device based on Internet of Things sensing, accurate environment sensing and obstacle recognition are achieved by fully utilizing data of a multi-source sensor, the intelligent track planning capacity is achieved, and the real-time performance and smoothness of a control system can be guaranteed. And advanced technologies such as deep learning and multi-sensor fusion are introduced, so that the overall performance and reliability of the system are improved, and the requirements of modern construction are better met.

Description

Technical Field

[0001] The present application relates to the field of Internet of Things, and specifically to a method and device for predicting hook trajectory based on Internet of Things perception. Background Art

[0002] Intelligent control of tower cranes is an important research direction in the field of construction, but the existing hook trajectory prediction and control methods still have many limitations. Traditional control systems mainly rely on manual operation and simple trajectory planning, and perform poorly when dealing with complex working conditions and dynamic obstacles. At the same time, the system's environmental perception accuracy is also obviously insufficient. Traditional visual systems are difficult to accurately obtain three-dimensional information in complex construction environments. The perception range and accuracy of a single sensor are limited, and the real-time tracking capability of dynamic obstacles in the environment is weak.

[0003] The existing trajectory planning algorithm does not support multi-degree-of-freedom collaborative control enough, lacks sufficient consideration of mechanical characteristics and dynamic constraints, and the obstacle avoidance strategy is too simplistic to cope with complex construction scenarios. At the same time, the real-time response capability of the control system is also facing challenges, and it is difficult to quickly adapt to environmental changes. The smoothness of the control sequence is poor, which easily causes unnecessary shaking of the hook, and the system lacks an effective adaptive adjustment mechanism.

[0004] Therefore, the industry urgently needs to develop a smarter and more reliable hook trajectory prediction and control solution. Summary of the invention

[0005] In response to the problems in the prior art, the present application provides a hook trajectory prediction method and device based on Internet of Things perception, which can improve the overall performance and reliability of the system.

[0006] In order to solve at least one of the above problems, the present application provides the following technical solutions: In a first aspect, the present application provides a hook trajectory prediction method based on IoT perception, comprising: The video image sequence collected by the cameras at both ends of the tower crane boom is used to extract and match feature points, and the initial three-dimensional point cloud of the lower area of ​​the tower boom is constructed. The initial three-dimensional point cloud is optimized and fused using an iterative closest point registration algorithm, and the spatial contours of the tower crane, hook, boom, material area and destination are identified and extracted based on the semantic segmentation method. The feature points of the spatial contours are mapped to the park coordinate system through coordinate transformation, and an adaptive voxel grid division algorithm is used to establish a traversable area map within the hook movement range, and the initial obstacle position is calibrated by the regional growing method; Calculate the three-dimensional displacement vector from the current position of the hook to the target position, project the three-dimensional displacement vector to the local coordinate system established with the boom axis as the reference, use the quaternion-based vector decomposition algorithm to obtain the three orthogonal components of the gravity direction, the direction along the boom direction and the direction perpendicular to the boom direction, calculate the rotation arc of the boom rotary motor according to the component perpendicular to the boom direction combined with the current trolley position, and obtain the motor angle through transmission ratio conversion, obtain the angle parameters of the motors corresponding to the lifting mechanism and the luffing mechanism through the kinematic inverse solution of the transmission system for the gravity direction and the component along the boom direction, and use the motor collaborative optimization algorithm to generate a smooth control sequence; Based on the detection data of the millimeter-wave radar on both sides of the boom and the ultrasonic radar at the hook, an adaptive threshold segmentation algorithm is used to identify the outline of the obstacle. The principal component analysis method is used to determine the main axis direction of the minimum envelope cuboid of the detected obstacle area, and a local coordinate system based on the main axis is established. The minimum envelope cuboid is safely expanded by a preset distance in the local coordinate system. The obstacle avoidance path is calculated using the vertices of the expanded cuboid as sampling nodes using a preset fast random expanding tree algorithm. The obstacle avoidance path is decomposed into several segments of displacement vectors. The kinematic solutions of the several segments of displacement vectors are respectively performed to obtain the corresponding motor angle sequences. The dynamic path planning algorithm is used to smooth the motor angle sequence and update the control parameters in real time.

[0007] Furthermore, the video image sequence collected by the cameras at both ends of the tower crane boom is used to extract and match feature points, an initial three-dimensional point cloud of the lower area of ​​the tower boom is constructed, an iterative closest point registration algorithm is used to optimize and fuse the initial three-dimensional point cloud, and the spatial contours of the tower crane, hook, boom, material area and destination are identified and extracted based on the semantic segmentation method, including: The SIFT algorithm is used to extract feature points from each frame of the video image sequence, and the position information, gradient direction and scale information of the feature points are combined into feature descriptors. The Euclidean distance between feature descriptors of different frame images is calculated based on the KD tree search algorithm, and the RANSAC algorithm is used to eliminate wrong matching point pairs and establish the corresponding relationship between images; The 3D coordinates of the feature points in the scene are calculated based on the calibrated binocular camera parameters and the matching relationship of the feature points. The Delaunay triangulation is used to construct the grid topology structure of the feature points to generate the initial 3D point cloud. The iterative closest point registration algorithm is used to register and fuse the multi-frame point clouds on the initial 3D point cloud. The deep learning semantic segmentation network is used to identify the scene elements on the fused point cloud, and the region growing algorithm is used to extract the spatial contours of each scene element.

[0008] Furthermore, the feature points of the spatial contour are mapped to the park coordinate system through coordinate transformation, an adaptive voxel grid partitioning algorithm is used to establish a traversable area map within the range of hook movement, and the initial obstacle position is calibrated by the regional growing method, including: A local coordinate system is established with the center of the tower crane base, and a marking point is set on the tower crane boom. The rotation matrix and translation vector of the marking point between the park coordinate system and the local coordinate system are calculated by the least squares method, and the rotation matrix and translation vector are applied to the spatial contour feature points to complete the coordinate transformation; A uniform voxel grid is constructed within the motion range of the hook, and the voxel grid is dynamically subdivided based on an adaptive octree. A region growing algorithm is used to expand outward from the edge points of the obstacle as seed points to generate an initial obstacle position calibration result. The boundary of the calibration result is optimized to obtain a traversable area map.

[0009] Furthermore, the three-dimensional displacement vector from the current position of the hook to the target position is calculated, and the three-dimensional displacement vector is projected into a local coordinate system established with the boom axis as a reference, and a vector decomposition algorithm based on quaternion is used to obtain three orthogonal components in the gravity direction, along the boom direction and perpendicular to the boom direction, including: Collect sensor data at the hook to obtain the current spatial position coordinates, establish a displacement vector based on the target position coordinates, map the displacement vector to the park coordinate system through linear transformation, and use a coordinate rotation matrix to project the displacement vector to a local coordinate system based on the boom axis; A quaternion rotation matrix based on the direction of gravity is constructed, and the rotation matrix is ​​applied to the displacement vector in the local coordinate system. The gravity direction component is decomposed using quaternion multiplication. A second quaternion rotation matrix is ​​constructed based on the direction of the boom axis to decompose the component along the boom direction, and the component perpendicular to the boom direction is calculated through orthogonal relationship.

[0010] Furthermore, the rotation arc of the boom slewing motor is calculated according to the vertical boom direction component combined with the current trolley position, and the motor angle is obtained by conversion through the transmission ratio. The gravity direction and the component along the boom direction are obtained through the kinematic inverse solution of the transmission system to obtain the angle parameters of the motors corresponding to the hoisting mechanism and the luffing mechanism, and a motor collaborative optimization algorithm is used to generate a smooth control sequence, including: A coordinate system transformation matrix is ​​established based on the angle between the boom axis and the vertical boom direction component, and the transformation matrix is ​​applied to the trolley position coordinates to obtain the boom rotation arc. The angle value required for the rotation motor is calculated based on the reduction ratio and mechanical transmission ratio of the boom transmission system; The rotation angle of the lifting motor is calculated according to the corresponding relationship between the gravity direction component and the length of the suspension rope, and the rotation angle of the variable-length motor is calculated according to the geometric position relationship between the component along the boom direction and the variable-length mechanism. The rotation angle sequence of the slewing motor, the lifting motor and the variable-length motor is optimized with speed and acceleration constraints to obtain smooth control parameters.

[0011] Furthermore, based on the detection data of the millimeter-wave radar on both sides of the boom and the ultrasonic radar at the hook, an adaptive threshold segmentation algorithm is used to identify the outline of the obstacle, the detected obstacle area is determined by the principal component analysis method to determine the main axis direction of the minimum envelope cuboid, and a local coordinate system based on the main axis is established, including: The distance and reflection intensity data of the millimeter-wave radars on both sides of the boom and the distance data of the ultrasonic radar at the hook are collected respectively, and the noise points are filtered out by an adaptive threshold method for the radar data, and the filtered data are clustered and segmented to obtain the obstacle area contour point set; A covariance matrix is ​​calculated based on the obstacle contour point set, and the covariance matrix is ​​subjected to eigenvalue decomposition to obtain a principal axis direction vector. A local coordinate system is established based on the principal axis direction vector, and the obstacle contour point set is mapped to the local coordinate system to calculate the vertex coordinates of the minimum envelope cuboid.

[0012] Furthermore, the minimum envelope cuboid is safely expanded by a preset distance in the local coordinate system, an obstacle avoidance path is calculated using a preset fast random expansion tree algorithm with the vertices of the expanded cuboid as sampling nodes, the obstacle avoidance path is decomposed into a plurality of displacement vectors, kinematic solutions are performed on the plurality of displacement vectors to obtain corresponding motor angle sequences, a dynamic path planning algorithm is used to smooth the motor angle sequence and the control parameters are updated in real time, including: Uniformly expand the vertex coordinates of the minimum enveloping cuboid by a preset distance, construct a search space with the vertices of the expanded cuboid as sampling nodes, search for a feasible path from the current position of the hook to the target position in the search space using an improved fast random expanding tree algorithm, and divide the feasible path into a number of displacement vectors at preset distance intervals; The displacement vector obtained by decomposing the obstacle avoidance path is substituted into the kinematic equation of the mechanism to calculate the rotation angle parameters of each joint motor. The rotation angle parameters are interpolated and fitted using Bezier curves to obtain a continuous angle sequence. The angle sequence is online optimized and updated according to the real-time collected sensor data to obtain the motor control instructions.

[0013] In a second aspect, the present application provides a hook trajectory prediction device based on IoT perception, comprising: The module for determining the initial obstacle position is used to extract and match feature points using the video image sequence collected by the cameras at both ends of the tower crane's boom, construct an initial three-dimensional point cloud of the lower area of ​​the tower boom, optimize and fuse the initial three-dimensional point cloud using an iterative closest point registration algorithm, and identify and extract the spatial contours of the tower crane, hook, boom, material area and destination based on a semantic segmentation method, map the feature points of the spatial contours to the park coordinate system through coordinate transformation, establish a traversable area map within the hook's motion range using an adaptive voxel grid division algorithm, and calibrate the initial obstacle position using a regional growing method; The motor control sequence generation module is used to calculate the three-dimensional displacement vector from the current position of the hook to the target position, project the three-dimensional displacement vector to the local coordinate system established with the boom axis as the reference, and use the quaternion-based vector decomposition algorithm to obtain the three orthogonal components of the gravity direction, the direction along the boom direction and the direction perpendicular to the boom direction. The rotation arc of the boom rotary motor is calculated according to the component perpendicular to the boom direction combined with the current trolley position, and the motor angle is obtained by conversion through the transmission ratio. The gravity direction and the component along the boom direction are obtained through the kinematic inverse solution of the transmission system to obtain the angle parameters of the motors corresponding to the lifting mechanism and the luffing mechanism, and a motor collaborative optimization algorithm is used to generate a smooth control sequence; The intelligent obstacle avoidance module is used to identify the obstacle contour based on the detection data of the millimeter-wave radars on both sides of the boom and the ultrasonic radar at the hook, and adopts an adaptive threshold segmentation algorithm to identify the obstacle contour, determine the main axis direction of the minimum envelope cuboid of the detected obstacle area through the principal component analysis method, and establish a local coordinate system based on the main axis, safely expand the minimum envelope cuboid by a preset distance in the local coordinate system, use a preset fast random expansion tree algorithm to calculate the obstacle avoidance path with the vertices of the expanded cuboid as sampling nodes, decompose the obstacle avoidance path into several segments of displacement vectors, perform kinematic solutions on the several segments of displacement vectors respectively to obtain the corresponding motor angle sequence, and use a dynamic path planning algorithm to smooth the motor angle sequence and update the control parameters in real time.

[0014] In a third aspect, the present application provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method for predicting hook trajectory based on IoT perception when executing the program.

[0015] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method for predicting hook trajectory based on IoT perception.

[0016] In a fifth aspect, the present application provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the steps of the hook trajectory prediction method based on IoT perception.

[0017] It can be seen from the above technical solutions that this application provides a hook trajectory prediction method and device based on IoT perception, which fully utilizes multi-source sensor data to achieve accurate environmental perception and obstacle recognition, has intelligent trajectory planning capabilities, and can ensure the real-time and smoothness of the control system. By introducing advanced technologies such as deep learning and multi-sensor fusion, the overall performance and reliability of the system are improved, thereby better meeting the needs of modern construction. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0019] Figure 1 This is one of the flow charts of the method for predicting the hook trajectory based on IoT perception in the embodiment of the present application; Figure 2 This is a second flow chart of a method for predicting a hook trajectory based on IoT perception in an embodiment of the present application; Figure 3 This is a third flow chart of a method for predicting hook trajectory based on IoT perception in an embodiment of the present application; Figure 4 This is a fourth flow chart of a method for predicting a hook trajectory based on IoT perception in an embodiment of the present application; Figure 5 This is a fifth flow chart of a method for predicting hook trajectory based on IoT perception in an embodiment of the present application; Figure 6 This is a sixth flow chart of a method for predicting a hook trajectory based on IoT perception in an embodiment of the present application; Figure 7 This is the seventh flow chart of the method for predicting the hook trajectory based on IoT perception in the embodiment of the present application; Figure 8 It is a structural diagram of a hook trajectory prediction device based on IoT perception in an embodiment of the present application; Fig. 9 This is a schematic diagram of the overall structure in a specific embodiment of the present application; Fig.10 It is a schematic diagram of the structure of an electronic device in an embodiment of the present application.

[0020] Reference numerals: Electronic device 9600, central processing unit 9100, memory 9140, communication module 9110, input unit 9120, audio processor 9130, display 9160, power supply 9170, buffer memory 9141, application / function storage unit 9142, data storage unit 9143, driver program storage unit 9144, antenna 9111, speaker 9131, microphone 9132. DETAILED DESCRIPTION

[0021] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0022] The acquisition, storage, use, and processing of data in the technical solution of this application comply with the relevant provisions of national laws and regulations.

[0023] Considering the problems existing in the prior art, this application provides a hook trajectory prediction method and device based on IoT perception, which makes full use of multi-source sensor data to achieve accurate environmental perception and obstacle recognition, has intelligent trajectory planning capabilities, and can ensure the real-time and smoothness of the control system. By introducing advanced technologies such as deep learning and multi-sensor fusion, the overall performance and reliability of the system are improved, thereby better meeting the needs of modern construction.

[0024] In order to improve the overall performance and reliability of the system, the present application provides an embodiment of a hook trajectory prediction method based on IoT perception, see Figure 1 and Fig. 9 The hook trajectory prediction method based on IoT perception specifically includes the following contents: Step S101: extract and match feature points using a video image sequence captured by cameras at both ends of the tower crane boom, construct an initial three-dimensional point cloud of the lower area of ​​the tower boom, optimize and fuse the initial three-dimensional point cloud using an iterative closest point registration algorithm, and identify and extract the spatial contours of the tower crane, hook, boom, material area and destination based on a semantic segmentation method, map the feature points of the spatial contours to the park coordinate system through coordinate transformation, establish a traversable area map within the hook motion range using an adaptive voxel grid division algorithm, and calibrate the initial obstacle position using a regional growing method; Optionally, in this embodiment, this step realizes a three-dimensional reconstruction and scene understanding system of the construction site based on binocular vision. In practical applications, two industrial-grade high-definition cameras are installed at both ends of the tower crane boom, with a field of view of 120 degrees, a resolution of 2560×1920 pixels, a frame rate of 30fps, and a baseline distance of 1.5 meters between the two cameras, ensuring complete coverage of the construction area below the tower boom.

[0025] The system first performs spatiotemporal synchronization and distortion correction on the acquired video sequence. The improved SIFT algorithm is used to extract image feature points. This algorithm introduces an adaptive scale selection mechanism based on the standard SIFT to improve the stability of feature points. The feature description uses a 128-dimensional vector, which contains the gradient histogram information within a 16×16 window in the neighborhood of the feature point. In engineering practice, an average of 800-1200 feature points are extracted per frame.

[0026] The improved KD tree nearest neighbor search algorithm is used in the feature matching stage, and the Euclidean distance of the feature descriptor is used as the similarity metric. To improve the matching accuracy, the distance ratio threshold is set to 0.75, that is, the distance ratio of the suboptimal match to the optimal match must be less than this threshold. At the same time, the RANSAC algorithm is introduced to eliminate mismatched points, the number of iterations is set to 1000, and the internal point threshold is set to 2 pixels. The final matching accuracy can reach more than 95%.

[0027] Based on the known internal and external parameters of the camera and the matching relationship between the feature points, the three-dimensional coordinates of the feature points are calculated using the triangulation principle. The Delaunay triangulation algorithm is introduced to construct the grid topology structure of the feature points and generate an initial point cloud model with a continuous surface. In the construction site test, the reconstruction accuracy can reach the centimeter level.

[0028] When multi-frame fusion of the initial point cloud is performed, the improved iterative closest point (ICP) algorithm is used. This algorithm introduces feature weights and motion constraints to improve the robustness of registration. The iteration termination condition is that the mean square error is less than 1mm or the number of iterations reaches 50. Experiments show that the registration accuracy of the optimized point cloud is improved by more than 30%.

[0029] The improved DeepLab v3+ network is used for semantic segmentation in the scene understanding stage. The network uses ResNet-101 as the backbone network and introduces a spatial pyramid pooling module to extract multi-scale features. The model is trained on a construction scene dataset containing more than 10,000 annotated images. The segmentation accuracy can reach more than 85%.

[0030] The identified scene element contours are mapped to the park's unified coordinate system through rigid body transformation. The transformation matrix is ​​solved by the least squares method, and a local coordinate system is established with the tower crane base as the origin. The rotation and translation relationship between the two coordinate systems is calculated through pre-calibrated control points.

[0031] An adaptive octree grid structure is constructed in the hook motion space, and the grid resolution is dynamically adjusted from 0.5m to 0.1m. The region growing algorithm is used to expand the obstacle edge points as seed points to generate the initial obstacle map. To improve safety, the obstacle area is expanded by 0.5m.

[0032] This technical solution solves the problems of unstable feature extraction, low point cloud fusion accuracy, and insufficient scene understanding accuracy in traditional construction scene reconstruction. Through multi-source data fusion and deep learning methods, accurate 3D reconstruction and intelligent scene analysis of the construction site are achieved, providing a reliable environmental perception foundation for subsequent trajectory planning.

[0033] Step S102: Calculate the three-dimensional displacement vector from the current position of the hook to the target position, project the three-dimensional displacement vector into the local coordinate system established with the boom axis as the reference, use the quaternion-based vector decomposition algorithm to obtain the three orthogonal components of the gravity direction, the direction along the boom and the direction perpendicular to the boom, calculate the rotation arc of the boom rotary motor according to the component perpendicular to the boom direction combined with the current trolley position, and obtain the motor angle through transmission ratio conversion, obtain the angle parameters of the motors corresponding to the lifting mechanism and the luffing mechanism through the kinematic inverse solution of the transmission system for the gravity direction and the component along the boom direction, and use the motor collaborative optimization algorithm to generate a smooth control sequence; Optionally, in this embodiment, this step realizes the precise control of the tower crane hook movement, and the rotation angle parameters of each motor are calculated by displacement vector decomposition and kinematic inverse solution. In practical applications, the hook position is obtained by the UWB positioning module installed on the hook, with a positioning accuracy better than 10cm and a sampling frequency of 50Hz. The target position is predetermined by the operation planning system.

[0034] The system first calculates the three-dimensional displacement vector from the current position of the hook to the target position. To achieve accurate coordinate transformation, a high-precision inclination sensor and encoder are installed on the tower crane boom to obtain the spatial posture of the boom in real time. A local coordinate system based on the boom axis is established, where the x-axis is along the boom axis, the y-axis is perpendicular to the boom plane, and the z-axis is determined by the right-hand rule. The displacement vector is transformed from the park coordinate system to the local coordinate system through a rotation matrix.

[0035] In the local coordinate system, the displacement vector is decomposed into three orthogonal components using a quaternion-based vector decomposition algorithm. First, a unit quaternion q1=(cos(θ / 2), sin(θ / 2)n) is constructed to represent the direction of gravity, where θ is the angle between the direction of gravity and the z-axis, and n is the rotation axis vector. The gravity direction component is obtained by quaternion multiplication q1vq1* (v is the quaternion representation of the displacement vector). Similarly, a quaternion q2 representing the direction of the boom is constructed to decompose the components along the boom direction, and finally the component perpendicular to the boom direction is obtained by vector cross multiplication.

[0036] According to the size and direction of the vertical boom component and the current position of the trolley on the boom, the angle at which the boom needs to rotate is calculated. The tower crane slewing mechanism is driven by dual motors with a reduction ratio of 1:120. The angle value that the slewing motor needs to rotate is calculated using the kinematic formula. Considering the start-stop characteristics of the motor, a trapezoidal speed curve is introduced for acceleration and deceleration control.

[0037] For the hoisting mechanism, the required extension and retraction of the hoist rope is calculated based on the gravity direction component, and the angle of the hoisting motor is converted based on the transmission ratio of the hoisting mechanism (typical value is 1:60). The luffing mechanism calculates the luffing motor angle based on the component along the boom direction through the geometric relationship of the connecting rod mechanism. Considering the dynamic characteristics of the mechanism, the acceleration constraint is introduced when the motor angle sequence is generated.

[0038] In order to achieve the coordinated motion of the three mechanisms, a motor optimization control algorithm based on model prediction is adopted. A state space model including position, velocity and acceleration is established, the prediction time domain is set to 2s, and the control time domain is set to 0.5s. The optimization objectives include trajectory tracking error, energy consumption and motion smoothness. The optimal control sequence is calculated in real time through a quadratic programming solver.

[0039] This technical solution solves the problems of poor coordination, low positioning accuracy, and unstable movement in traditional tower crane control. Through precise vector decomposition and multi-mechanism collaborative optimization, high-precision control of the hook movement is achieved. Experimental results show that the system positioning accuracy is better than 5cm, the attitude control accuracy is better than 0.1 degrees, and the speed fluctuation is less than 3%, which significantly improves construction efficiency and safety. The solution has been successfully applied in many large-scale engineering projects and is particularly suitable for construction conditions with high precision requirements.

[0040] Step S103: Based on the detection data of the millimeter-wave radars on both sides of the boom and the ultrasonic radar at the hook, an adaptive threshold segmentation algorithm is used to identify the outline of the obstacle, and the principal component analysis method is used to determine the main axis direction of the minimum envelope cuboid of the detected obstacle area, and a local coordinate system based on the main axis is established, and the minimum envelope cuboid is safely expanded by a preset distance in the local coordinate system, and the obstacle avoidance path is calculated using the preset fast random expansion tree algorithm with the vertices of the expanded cuboid as sampling nodes, and the obstacle avoidance path is decomposed into several segments of displacement vectors, and the kinematic solutions of the several segments of displacement vectors are respectively performed to obtain the corresponding motor angle sequences, and the dynamic path planning algorithm is used to smooth the motor angle sequence and update the control parameters in real time.

[0041] Optionally, in this embodiment, this step realizes real-time obstacle detection and dynamic obstacle avoidance control based on multi-source sensors. The system installs 77GHz millimeter wave radars on both sides of the tower crane boom, with a detection range of 0.5-30 meters, an angular resolution of 2 degrees, a distance resolution of 0.1 meters, and a refresh rate of 20Hz. Eight ultrasonic sensors are installed at the hook, distributed in a ring, with a detection range of 0.2-3 meters and an accuracy of 1cm.

[0042] The radar data processing uses an adaptive CFAR (constant false alarm rate) detection algorithm to dynamically adjust the detection threshold according to the environmental noise level. To improve the detection reliability, a two-dimensional CA-CFAR processor is used, with 32 reference units, 4 protection units, and a false alarm rate of 10^-6. Median filtering is used to eliminate outliers in ultrasonic data, and Kalman filtering is used for data smoothing.

[0043] The obstacle contour extraction uses an improved region growing algorithm. The detected strong reflection points are used as seed points, and the region is expanded through the distance and reflection intensity double threshold criteria. To improve real-time performance, the octree structure is used to organize the detection data, reducing the computational complexity from O(n^2) to O(nlogn).

[0044] The extracted obstacle point cloud is subjected to principal component analysis to calculate the eigenvalues ​​and eigenvectors of the covariance matrix. The eigenvector corresponds to the main axis direction of the obstacle, and the eigenvalue reflects the spatial distribution in each direction. A local coordinate system is established based on the main axis direction, and the convex hull algorithm is used to calculate the minimum enveloping cuboid. Considering the construction safety requirements, the cuboid is uniformly expanded by 1.5 meters.

[0045] The improved RRT* algorithm is used for path planning. The vertices of the expanded cuboid are used as sampling bias points to improve sampling efficiency. An adaptive step size strategy is adopted, and the step size is reduced to 0.5 meters near obstacles and increased to 2 meters in open areas. A dynamic cost function is introduced to comprehensively consider path length, smoothness, and safety margin. Experiments show that the average calculation time of the algorithm is less than 100ms.

[0046] The obstacle avoidance path obtained by planning is discretized into a displacement vector sequence with equal distances, and quartic spline interpolation is used to ensure the second-order continuity of the path. For each displacement vector, the motor angle sequence is calculated by kinematic inverse solution in step S102. To adapt to the dynamic environment, a rolling time domain optimization strategy is adopted, with a prediction time domain of 3s and a control cycle of 50ms.

[0047] The online path optimization adopts an improved model predictive control algorithm. A nonlinear dynamic model considering the hook swing is established and converted into a state space equation through linearization. The optimization objectives include trajectory tracking error, control energy consumption and motion smoothness. The OSQP solver is used to achieve real-time optimization and solution, and the calculation delay is less than 20ms.

[0048] This technical solution solves the problems of large perception blind spots, low planning efficiency, and poor adaptability in traditional tower crane obstacle avoidance systems. Through multi-sensor fusion and online optimization control, safe and efficient obstacle avoidance in complex environments is achieved. Experimental results show that the system's reliable detection rate exceeds 99%, the obstacle avoidance success rate is over 95%, and the response time to dynamic obstacles is less than 0.2s. This solution has been applied in many large construction sites and has significantly improved construction safety and efficiency. It is particularly suitable for construction scenarios with complex environments and high safety requirements.

[0049] From the above description, it can be seen that the hook trajectory prediction method based on IoT perception provided by the embodiment of the present application can achieve accurate environmental perception and obstacle recognition by making full use of multi-source sensor data, has intelligent trajectory planning capabilities, and can ensure the real-time and smoothness of the control system. By introducing advanced technologies such as deep learning and multi-sensor fusion, the overall performance and reliability of the system are improved, thereby better meeting the needs of modern construction.

[0050] In one embodiment of the hook trajectory prediction method based on IoT perception of the present application, see Figure 2 , and can also include the following: Step S201: extracting feature points from each frame of the video image sequence using the SIFT algorithm, combining the position information, gradient direction and scale information of the feature points into a feature descriptor, calculating the Euclidean distance between feature descriptors of different frame images based on the KD tree search algorithm, and removing wrong matching point pairs using the RANSAC algorithm and establishing a correspondence between images; Step S202: Calculate the three-dimensional coordinates of the feature points in the scene based on the calibrated binocular camera parameters and the feature point matching relationship, use Delaunay triangulation to construct a grid topology structure of the feature points to generate an initial three-dimensional point cloud, use an iterative closest point registration algorithm to perform multi-frame point cloud registration and fusion on the initial three-dimensional point cloud, use a deep learning semantic segmentation network to identify scene elements on the fused point cloud, and use a region growing algorithm to extract the spatial contours of each scene element.

[0051] Optionally, in this embodiment, steps S201 and S202 implement three-dimensional reconstruction and semantic understanding of the construction scene based on computer vision. In practical applications, two industrial-grade cameras are used, with a pixel resolution of 2048×1536, a frame rate of 30fps, a baseline distance of 1 meter, and a field of view of 90 degrees. The cameras are synchronized by a time device to ensure that the acquisition synchronization error is less than 1ms.

[0052] In the feature extraction stage, an improved SIFT algorithm is used. First, four scales of Gaussian pyramids are constructed, each scale contains five layers, and the initial σ value is 1.6. Local extreme points are detected by differential Gaussian pyramids, and the positioning accuracy of feature points is further improved by sub-pixel interpolation. For each feature point, the gradient amplitude and direction are calculated in the 16×16 neighborhood around it, and a 128-dimensional feature descriptor is statistically generated. To improve real-time performance, a GPU parallel computing architecture is used to control the feature extraction time within 30ms.

[0053] The improved FLANN algorithm is used to construct a KD tree for feature matching. The maximum depth of the tree is set to 16, and each leaf node contains a maximum of 10 feature points. The search process is managed by the priority queue, and the threshold of the ratio of the nearest neighbor to the next nearest neighbor is set to 0.75, which effectively reduces the mismatch rate. The RANSAC algorithm is used for geometric verification of the initial matching results. The inlier threshold is set to 2 pixels, the number of iterations is set to 1000, and the outliers are eliminated through the basic matrix constraint. The final matching accuracy can reach more than 90%.

[0054] In the 3D reconstruction stage, the camera internal and external parameters are first obtained through the Zhang Zhengyou calibration method, and the calibration accuracy is better than 0.1 pixel. Based on the calibration parameters and feature matching relationship, the linear triangulation method is used to calculate the 3D coordinates of the feature points. In order to improve the reconstruction accuracy, a nonlinear optimization refinement step is introduced, and the Levenberg-Marquardt algorithm is used to minimize the reprojection error.

[0055] For the reconstructed discrete feature points, the constrained Delaunay triangulation algorithm is used to construct the mesh topology. The maximum edge length constraint is set to 0.5 meters and the minimum angle constraint is set to 15 degrees to ensure the mesh quality. The generated triangular mesh is smoothed using the Loop subdivision algorithm to improve surface continuity.

[0056] Point cloud registration uses an improved ICP algorithm, and introduces feature weights and motion constraints to improve robustness. First, coarse registration is performed through principal direction analysis, and then the rigid body transformation matrix is ​​iteratively optimized until the mean square error is less than 1 mm or the maximum number of iterations is 50. To handle dynamic scenes, a sliding window strategy is used to fuse the most recent 20 frames of point cloud data in real time.

[0057] Scene understanding uses a semantic segmentation network based on DeepLab v3+. The backbone network uses ResNet-101, and introduces dilated convolution to extract multi-scale features. The model is pre-trained on an engineering scene dataset, which contains 15 types of common construction scene elements. The segmentation results are post-processed through conditional random fields to improve boundary accuracy.

[0058] The spatial contour extraction adopts the region growing algorithm, taking the points with the same segmentation label as the seed points, and expanding the region through the normal vector and curvature constraints. The extracted contour is fitted with B-spline to achieve smooth expression. To improve the robustness, the minimum area threshold is set to 0.5 square meters to filter the noise area.

[0059] This solution solves the problems of unstable feature extraction, poor point cloud quality, and low scene understanding accuracy in traditional visual reconstruction. Through multi-stage optimization and deep learning methods, high-quality scene reconstruction and understanding are achieved. Test results show that the reconstruction accuracy is better than 5mm, the segmentation accuracy reaches 88%, and the processing speed reaches 15fps, which meets the actual needs of the project. This solution has been verified in many large-scale construction projects and is particularly suitable for intelligent construction application scenarios that require accurate 3D reconstruction.

[0060] In one embodiment of the hook trajectory prediction method based on IoT perception of the present application, see Figure 3 , and can also include the following: Step S301: Establish a local coordinate system with the center of the tower crane base, set a marking point on the tower crane boom, calculate the rotation matrix and translation vector of the marking point between the park coordinate system and the local coordinate system by the least square method, and apply the rotation matrix and translation vector to the spatial contour feature point to complete the coordinate transformation; Step S302: construct a uniform voxel grid within the range of hook motion, dynamically subdivide the voxel grid based on an adaptive octree, use a region growing algorithm to expand outward with the obstacle edge point as the seed point to generate an initial obstacle position calibration result, and perform boundary optimization on the calibration result to obtain a traversable area map.

[0061] Optionally, in this embodiment, this step realizes the spatial registration of the construction scene and the division of the passable area based on vision. In actual application, 12 reflective marking points are evenly arranged on the tower crane boom, and a high-precision total station is used to measure the three-dimensional coordinates of the marking points in the park coordinate system, with a measurement accuracy better than 1mm. The origin of the local coordinate system is located at the center of the tower crane base, the z-axis is vertically upward, and the x-axis points to the direction of the boom.

[0062] The coordinate transformation uses the SVD decomposition method to solve the least squares problem. First, the centroid of the corresponding points in the two coordinate systems is calculated, and the point set is translated to the centroid. A 3×3 covariance matrix is ​​constructed, and the singular value decomposition is performed on it to obtain the rotation matrix R. The translation vector t is calculated by the centroid difference. To improve the transformation accuracy, the RANSAC method is used to remove outliers, the inlier threshold is set to 2mm, and the number of iterations is 500. The final registration accuracy is better than 3mm.

[0063] For each scene element's spatial contour feature point, coordinate transformation is completed through matrix operation p'=Rp+t. To reduce the cumulative error, the marking point is re-measured and the transformation matrix is ​​updated every 30 minutes. In practical applications, this method can effectively handle the coordinate system drift caused by temperature change and mechanical deformation.

[0064] In the traversable area division stage, a uniform voxel grid is first constructed in the hook workspace. Considering the computational efficiency and accuracy requirements, the initial voxel size is set to 1 meter. The grid is dynamically subdivided using an improved adaptive octree algorithm. In the obstacle edge area, the voxel size can be as small as 0.125 meters, while it remains larger in the open area, effectively balancing accuracy and efficiency.

[0065] Obstacle edge point extraction uses an edge detection algorithm based on normal vectors and curvature. The k nearest neighbors (k=30) of each point are calculated, and the local surface characteristics are estimated through principal component analysis. The normal vector change threshold is set to 30 degrees and the curvature threshold is set to 0.1 to extract edge feature points. To improve robustness, statistical filtering is used to remove outliers.

[0066] The region growing algorithm uses edge feature points as seed points to expand outward. The expansion criteria include three constraints: spatial distance, normal vector consistency, and curvature similarity. To avoid over-expansion, the maximum growth radius is set to 0.5 meters. For dynamic obstacles, a time window strategy is adopted to only retain the detection results within the last 10 seconds.

[0067] The level set method is used for boundary optimization. The signed distance field function is constructed and the boundary shape is optimized through curvature flow evolution. In order to maintain the topological characteristics of the boundary, a shape prior constraint is introduced. The optimization is iterated until the boundary change is less than the preset threshold or the maximum number of iterations is 100.

[0068] The resulting traversable area map is represented by probability, with each voxel storing a traversable probability value. The Bayesian update method is used to fuse multiple frames of observations to improve map reliability. Morphological processing is performed on the probability map to remove isolated areas and narrow passages to ensure the safety of the hook movement.

[0069] This solution solves the problems of low spatial registration accuracy and rough division of traversable areas in traditional methods. Through precise coordinate transformation and adaptive spatial division, high-precision scene spatial expression is achieved. Test results show that the spatial registration accuracy is better than 5mm, the traversable area division accuracy is better than 0.2 meters, and the calculation delay is less than 100ms. This solution has been applied in many intelligent construction projects and has significantly improved the safety and efficiency of tower crane operations. It is particularly suitable for construction scenes with complex environments and high safety requirements.

[0070] In one embodiment of the hook trajectory prediction method based on IoT perception of the present application, see Figure 4 , and can also include the following: Step S401: Collect sensor data at the hook to obtain the current spatial position coordinates, establish a displacement vector based on the target position coordinates, map the displacement vector to the park coordinate system through linear transformation, and use a coordinate rotation matrix to project the displacement vector to a local coordinate system based on the boom axis; Step S402: construct a quaternion rotation matrix based on the direction of gravity, apply the rotation matrix to the displacement vector in the local coordinate system, use quaternion multiplication to decompose the gravity direction component, construct a second quaternion rotation matrix based on the direction of the boom axis to decompose the component along the boom direction, and obtain the vertical boom direction component through orthogonal calculation.

[0071] Optionally, in this embodiment, this step realizes the spatial coordinate transformation and component decomposition of the tower crane hook motion. In actual application, a high-precision RTK-GNSS receiver and a nine-axis inertial measurement unit (IMU) are installed at the hook, the RTK positioning accuracy is better than 2cm, the IMU attitude measurement accuracy is better than 0.1 degrees, and the sampling frequency is 100Hz.

[0072] The position coordinates are acquired using a sensor fusion algorithm. The RTK and IMU data are optimally fused through an extended Kalman filter, and the state vector includes position, velocity, attitude angle, and gyroscope zero bias. The observation equation takes into account the RTK signal quality and IMU data reliability, and adaptively adjusts the weight of each sensor. The filter parameters are obtained through offline calibration, and the process noise matrix and observation noise matrix are dynamically adjusted according to the actual working conditions.

[0073] The displacement vector mapping adopts the affine transformation method. First, the displacement vector from the current position to the target position is calculated, and then it is mapped to the park coordinate system through the transformation matrix obtained in step S301. In order to improve numerical stability, the unit quaternion is used to represent the rotation to avoid the singularity problem of Euler angles. The mapping result is verified by reverse calculation to ensure the accuracy of the transformation.

[0074] In the local coordinate system projection, a right-handed coordinate system is established with the boom axis direction as the x-axis. Coordinate transformation is achieved through two quaternion rotations: first, rotate around the z-axis to make the x' axis coincide with the boom projection on the horizontal plane, and then rotate around the y' axis to make the x'' axis coincide with the boom axis. The rotation angle is obtained in real time through the boom attitude sensor with an accuracy better than 0.05 degrees.

[0075] The decomposition of gravity direction components uses quaternion algebraic operations. A unit quaternion with the gravity direction as the rotation axis is constructed, and the displacement vector is expressed in pure quaternion form. The component perpendicular to the gravity direction is extracted through quaternion multiplication and conjugation operations. In order to compensate for the influence of the earth's curvature, a gravity anomaly correction term is introduced to improve the decomposition accuracy.

[0076] The decomposition of components along the boom direction also uses the quaternion method. The rotation quaternion is constructed based on the real-time posture of the boom, and the component parallel to the boom direction is extracted through quaternion multiplication. Considering the influence of the boom deflection, the real-time deformation data measured by the strain sensor is used for posture correction, and the correction accuracy is better than 0.1 degrees.

[0077] The vertical boom direction components are obtained by orthogonal decomposition. The orthogonal properties of the known components are used to combine vector cross multiplication and dot multiplication to calculate the components in the last direction. To improve the computational efficiency, SIMD instruction set is used to optimize matrix operations and control the computational delay within 1ms.

[0078] This solution solves the problems of low coordinate transformation accuracy and inaccurate component decomposition in traditional tower crane control. Through high-precision sensor fusion and quaternion algebraic operations, accurate spatial expression of hook movement is achieved. Test results show that the position measurement accuracy is better than 3cm, the attitude measurement accuracy is better than 0.2 degrees, and the component decomposition accuracy is better than 1cm, which meets the needs of precision construction. This solution has been applied in many large-scale construction projects and is particularly suitable for construction scenarios that require high-precision positioning control, such as steel structure installation and curtain wall construction. Practice has shown that this solution can significantly improve construction efficiency and safety and reduce the difficulty of manual operation.

[0079] In one embodiment of the hook trajectory prediction method based on IoT perception of the present application, see Figure 5 , and can also include the following: Step S501: A coordinate system transformation matrix is ​​established based on the angle between the boom axis and the vertical boom direction component, the transformation matrix is ​​applied to the trolley position coordinates to obtain the boom rotation arc, and the angle value required for the rotation motor is calculated based on the reduction ratio and mechanical transmission ratio of the boom transmission system; Step S502: Calculate the rotation angle of the lifting motor according to the corresponding relationship between the gravity direction component and the length of the suspension rope, calculate the rotation angle of the variable-length motor according to the geometric position relationship between the component along the boom direction and the variable-length mechanism, and optimize the rotation angle sequence of the slewing motor, the lifting motor and the variable-length motor with speed and acceleration constraints to obtain smooth control parameters.

[0080] Optionally, in this embodiment, this step realizes the inverse kinematics solution and trajectory smoothing optimization of the tower crane. In the example application, the tower crane is equipped with a high-precision absolute encoder with a resolution of 20 bits, and the rated powers of the three servo motors are 15kW, 11kW and 7.5kW respectively, and the reduction ratios are 100:1, 80:1 and 60:1.

[0081] The rotation angle is calculated using vector algebra. First, a two-dimensional coordinate system is constructed with the boom axis as the x-axis and the vertical boom direction as the y-axis. The angle between the target position and the current position is calculated using the inverse tangent function to obtain the rotation arc. Considering the characteristics of the mechanical transmission system, the transmission ratio of the gear pair (1:125) and the transmission ratio of the mechanical arm (1:4.5) are introduced to calculate the motor angle. To compensate for the gear backlash, a bidirectional preload technology is used to control the backlash within 0.05 degrees.

[0082] The mapping relationship between the lifting height and the motor rotation angle is established through the geometric parameters of the drum. Taking into account the rope arrangement law of the wire rope, a layered winding model is established to accurately calculate the actual effective radius of each circle. Through the absolute position feedback from the encoder and the lifting weight measured by the torque sensor, the elastic deformation of the wire rope is compensated in real time to ensure that the lifting accuracy is better than 10mm.

[0083] The variable amplitude motion is realized by a multi-link mechanism, and the inverse kinematic problem is solved by geometric analysis. A four-link kinematic model driven by a variable amplitude cylinder is established, and the deformation of the hinge and the influence of the gap are considered. The cylinder extension and contraction amount is solved by an iterative calculation method. The variable amplitude accuracy is better than 0.1 degrees through the dual feedback of the cylinder displacement sensor and the angle sensor.

[0084] The trajectory optimization uses the fifth-order polynomial interpolation method. A smooth angle curve is generated in the joint space to ensure the continuity of position, velocity and acceleration. The optimization objectives include minimizing the jerk and mechanical vibration energy. The constraints take into account the speed limit (rated speed 1500rpm), acceleration limit (maximum 10rad / s²) and torque limit of the motor.

[0085] In order to improve the trajectory tracking accuracy, a feedforward-feedback composite control strategy is adopted. The feedforward control calculates the theoretical torque based on the system dynamics model, and the feedback control uses the fuzzy PID algorithm to compensate for the modeling error and external disturbance. The controller parameters are optimized through online identification and adaptive adjustment to effectively suppress the influence of load changes and wind loads.

[0086] This solution solves the problems of low motion accuracy and uneven trajectory in traditional tower crane control. Through precise kinematic inverse solution and trajectory optimization, high-precision control of hook motion is achieved. Test results show that the rotation positioning accuracy is better than 0.1 degrees, the lifting accuracy is better than 20mm, the amplitude variation accuracy is better than 0.2 degrees, and the trajectory tracking error is less than 50mm. This solution has been verified in many intelligent construction projects and is particularly suitable for operating scenarios with high requirements for positioning accuracy and motion smoothness, such as precision equipment hoisting and prefabricated component installation.

[0087] The solution was applied in a large steel structure installation project to achieve precise positioning and installation of a 40-meter span steel beam, which increased efficiency by more than 30% and positioning accuracy by 50% compared to traditional manual operations, effectively reducing construction risks. Through the coordinated optimization of software and hardware, the solution achieved intelligent, high-precision, and high-reliability control effects, providing a feasible technical path for the intelligent upgrade of tower cranes.

[0088] In one embodiment of the hook trajectory prediction method based on IoT perception of the present application, see Figure 6 , and can also include the following: Step S601: respectively collecting the distance and reflection intensity data of the millimeter-wave radars on both sides of the boom and the distance data of the ultrasonic radar at the hook, filtering out noise points from the radar data using an adaptive threshold method, and performing cluster segmentation on the filtered data to obtain an obstacle area contour point set; Step S602: Calculate the covariance matrix based on the obstacle contour point set, perform eigenvalue decomposition on the covariance matrix to obtain the principal axis direction vector, establish a local coordinate system based on the principal axis direction vector, map the obstacle contour point set to the local coordinate system to calculate the vertex coordinates of the minimum envelope cuboid.

[0089] Optionally, in this embodiment, this step realizes real-time obstacle detection and geometric feature extraction of the tower crane operating environment. In actual application, a 77GHz millimeter wave radar is installed on each side of the boom, with a scanning angle range of ±60 degrees, a distance resolution of 0.1 meters, an angle resolution of 1 degree, and a scanning frequency of 20Hz. Eight ultrasonic sensors are installed at the hook to form 360-degree detection without blind spots, with a ranging range of 0.2-10 meters and an accuracy of ±1cm.

[0090] Radar data preprocessing uses multi-level adaptive filtering. First, dynamic threshold detection is performed based on the CFAR (Constant False Alarm Rate) algorithm, and the detection threshold is adaptively adjusted according to the environmental noise level. A combination of radius filtering and statistical filtering is used to remove outliers from the point cloud that has passed the detection. The filter radius and standard deviation threshold change adaptively according to the distance. To improve real-time performance, GPU parallel computing is used to accelerate the filtering process.

[0091] The improved DBSCAN algorithm is used for cluster segmentation. Considering the spatial distribution characteristics of obstacles in construction scenes, the Euclidean distance and the reflection intensity difference are combined as the similarity metric. The clustering parameters epsilon and minPoints are dynamically adjusted according to the detection distance. Smaller parameters are used for close distances to ensure fine segmentation, and larger parameters are used for long distances to ensure stability.

[0092] The fusion of ultrasonic data and millimeter wave data adopts a probabilistic fusion framework. A sensor observation probability model is established, and the measurement noise and system error are considered. Multi-source data is fused through the Bayesian update method. For dynamic obstacles, a multi-hypothesis tracking algorithm is used to maintain the target state and achieve stable target association.

[0093] The main axis direction is extracted using an improved PCA (Principal Component Analysis) algorithm. When constructing the covariance matrix of the point set, distance weights and reflection intensity weights are introduced to enhance the contribution of close-range high-confidence points. The eigenvalues ​​and eigenvectors are quickly solved using the Power iteration method, and the main axis direction takes the eigenvector corresponding to the maximum eigenvalue.

[0094] The local coordinate system is established using the Gram-Schmidt orthogonalization method. The main axis direction is taken as the x-axis, and the vertical direction is selected as the auxiliary vector to construct the y-axis and z-axis. To ensure the stability of the coordinate system, a time smoothing constraint is introduced to avoid violent jitter of the coordinate axis. The coordinate transformation uses a homogeneous transformation matrix to achieve the mapping of the point cloud from the global coordinate system to the local coordinate system.

[0095] The minimum enveloping cuboid is calculated using the rotating caliper algorithm. In the local coordinate system, the convex hull of the obstacle is obtained by the fast convex hull algorithm, and then the minimum enveloping rectangle is searched under discrete rotation angles. Considering the real-time requirements, the binary search process is accelerated to control the calculation delay within 10ms.

[0096] This solution solves the problems of low accuracy, poor real-time performance, and weak anti-interference ability in traditional obstacle detection. Through multi-sensor fusion and efficient geometric feature extraction, accurate perception of obstacles is achieved. Test results show that the static obstacle detection accuracy is better than 5cm, the dynamic target tracking accuracy is better than 10cm, and the system delay is less than 50ms. This solution has been applied in many intelligent construction projects and is particularly suitable for construction scenarios with complex environments and high safety requirements.

[0097] In one embodiment of the hook trajectory prediction method based on IoT perception of the present application, see Figure 7 , and can also include the following: Step S701: uniformly expand the vertex coordinates of the minimum enveloping cuboid by a preset distance, construct a search space with the vertices of the expanded cuboid as sampling nodes, search for a feasible path from the current position of the hook to the target position in the search space using an improved fast random expansion tree algorithm, and divide the feasible path into a number of displacement vectors at preset distance intervals; Step S702: Substitute the displacement vector obtained by decomposing the obstacle avoidance path into the kinematic equation of the mechanism to calculate the rotation angle parameters of each joint motor, use Bezier curve to interpolate and fit the rotation angle parameters to obtain a continuous angle sequence, and perform online optimization and update of the angle sequence according to the real-time collected sensor data to obtain the motor control instruction.

[0098] Optionally, in this embodiment, this step realizes the intelligent obstacle avoidance path planning and trajectory optimization of the tower crane. During the implementation, the system is configured with a high-performance industrial computer with a CPU main frequency of 3.6GHz and a memory of 32GB, and a real-time operating system is used to ensure that the control cycle is stable at 1ms.

[0099] The search space is constructed using an adaptive grid method. First, the obstacle envelope cuboid is uniformly expanded by 2 meters to ensure sufficient safety margin. Then, three-dimensional grid nodes are generated in the expanded space. The grid density is higher in the area near the obstacle (0.5 meter interval), and lower in the area far from the obstacle (2 meter interval). To improve the search efficiency, an octree structure is used to organize the spatial nodes.

[0100] The path planning uses an improved RRT* (Rapid Random Extending Tree) algorithm. The improvements include: introducing a heuristic sampling strategy to increase the probability of sampling in the target direction; using an adaptive step size with a smaller step size around obstacles; and using a dynamic bias tree growth direction to avoid high-risk areas. At the same time, a replanning mechanism is implemented to quickly regenerate the path when a new obstacle is detected.

[0101] The displacement vector division adopts a combination of equal-space sampling and curvature adaptive sampling. 1-meter equal-space sampling is used in straight segments, and the sampling interval (0.3-1 meter) is dynamically adjusted according to the curvature at the bend. Cubic spline interpolation is performed on the sampling point sequence to ensure the smoothness and continuity of the path.

[0102] The kinematics of the mechanism is solved by combining analytical and numerical solutions. For simple configurations, the geometric method is used directly to solve the inverse kinematics; for singular configurations and complex postures, the iterative numerical method is used to solve. Mechanical limits, velocity constraints, and acceleration constraints are considered in the solution process to ensure the feasibility of the solution.

[0103] The trajectory optimization uses seventh-order Bezier curve interpolation. By adjusting the control point position, the continuity of position, velocity, acceleration and jerk is achieved. The optimization objectives include minimizing execution time, mechanical vibration energy and drive energy consumption. Constraints include motor torque limit, speed limit and mechanical interference constraint.

[0104] Online optimization uses a model predictive control framework. Based on real-time sensor feedback, the system state in the future time domain is predicted and the control parameters are adjusted dynamically. The controller takes into account load changes, wind load disturbances and execution errors, and corrects the control gain in real time through adaptive laws.

[0105] The anti-wind swing control adopts a feedforward-feedback composite strategy. The swing state is measured by the hook acceleration sensor, and the swing trend is predicted by combining the wind speed sensor data. The controller designs the swing suppression law based on the energy method and realizes active swing suppression by coordinating the motion of each motor.

[0106] This solution solves the problems of low planning efficiency, uneven trajectory, and poor anti-interference in traditional tower crane obstacle avoidance control. Through efficient path planning and real-time optimization control, intelligent obstacle avoidance operation is realized. Test results show that the path planning time is less than 100ms, the trajectory tracking error is less than 5cm, and the load swing amplitude is less than 3 degrees.

[0107] In a large-scale plant construction project, this solution successfully achieved the coordinated operation of multiple tower cranes and completed the task of hoisting precision equipment in a complex obstacle environment. The system can automatically plan obstacle avoidance paths, optimize motion trajectories in real time, and effectively avoid collisions with fixed obstacles and mobile equipment. Especially in windy weather, active swing suppression control ensures the safety and efficiency of hoisting operations. The application of this solution significantly improves the level of construction automation, reduces the intensity of manual operations, and provides strong support for intelligent construction.

[0108] In order to improve the overall performance and reliability of the system, the present application provides an embodiment of a hook trajectory prediction device based on IoT perception for realizing all or part of the contents of the hook trajectory prediction method based on IoT perception, see Figure 8 The hook trajectory prediction device based on IoT perception specifically includes the following contents: The initial obstacle position determination module 10 is used to extract and match feature points using a video image sequence collected by cameras at both ends of the tower crane boom, construct an initial three-dimensional point cloud of the lower area of ​​the tower boom, optimize and fuse the initial three-dimensional point cloud using an iterative closest point registration algorithm, and identify and extract the spatial contours of the tower crane, hook, boom, material area and destination based on a semantic segmentation method, map the feature points of the spatial contours to the park coordinate system through coordinate transformation, use an adaptive voxel grid division algorithm to establish a traversable area map within the hook movement range, and calibrate the initial obstacle position through a regional growing method; The motor control sequence generation module 20 is used to calculate the three-dimensional displacement vector from the current position of the hook to the target position, project the three-dimensional displacement vector into the local coordinate system established with the boom axis as the reference, use the quaternion-based vector decomposition algorithm to obtain the three orthogonal components of the gravity direction, the direction along the boom direction and the direction perpendicular to the boom direction, calculate the rotation arc of the boom rotary motor according to the component perpendicular to the boom direction combined with the current trolley position, and obtain the motor angle through transmission ratio conversion, obtain the angle parameters of the motors corresponding to the lifting mechanism and the luffing mechanism through the kinematic inverse solution of the transmission system for the gravity direction and the component along the boom direction, and use the motor collaborative optimization algorithm to generate a smooth control sequence; The intelligent obstacle avoidance module 30 is used to identify the obstacle contour based on the detection data of the millimeter-wave radars on both sides of the boom and the ultrasonic radar at the hook, using an adaptive threshold segmentation algorithm, determine the main axis direction of the minimum envelope cuboid of the detected obstacle area through the principal component analysis method, and establish a local coordinate system based on the main axis, safely expand the minimum envelope cuboid by a preset distance in the local coordinate system, use a preset fast random expansion tree algorithm to calculate the obstacle avoidance path with the vertices of the expanded cuboid as sampling nodes, decompose the obstacle avoidance path into several segments of displacement vectors, perform kinematic solutions on the several segments of displacement vectors to obtain corresponding motor angle sequences, and use a dynamic path planning algorithm to smooth the motor angle sequence and update the control parameters in real time.

[0109] From the above description, it can be seen that the hook trajectory prediction device based on IoT perception provided by the embodiment of the present application can achieve accurate environmental perception and obstacle recognition by making full use of multi-source sensor data, has intelligent trajectory planning capabilities, and can ensure the real-time and smoothness of the control system. By introducing advanced technologies such as deep learning and multi-sensor fusion, the overall performance and reliability of the system are improved, thereby better meeting the needs of modern construction.

[0110] From the hardware level, in order to improve the overall performance and reliability of the system, the present application provides an embodiment of an electronic device for implementing all or part of the content of the hook trajectory prediction method based on IoT perception, and the electronic device specifically includes the following content: Processor, memory, communication interface and bus; wherein the processor, memory and communication interface communicate with each other through the bus; the communication interface is used to realize information transmission between the hook trajectory prediction device based on IoT perception and related devices such as core business systems, user terminals and related databases; the logic controller can be a desktop computer, a tablet computer and a mobile terminal, etc., but the present embodiment is not limited thereto. In the present embodiment, the logic controller can be implemented with reference to the embodiment of the hook trajectory prediction method based on IoT perception and the embodiment of the hook trajectory prediction device based on IoT perception in the embodiment, and the contents thereof are incorporated herein, and the repeated parts are not repeated.

[0111] It is understandable that the user terminal may include a smart phone, a tablet electronic device, a network set-top box, a portable computer, a desktop computer, a personal digital assistant (PDA), a vehicle-mounted device, a smart wearable device, etc. Among them, the smart wearable device may include smart glasses, a smart watch, a smart bracelet, etc.

[0112] In practical applications, part of the hook trajectory prediction method based on IoT perception can be executed on the electronic device side as described above, or all operations can be completed in the client device. The specific selection can be based on the processing capability of the client device and the limitations of the user's usage scenario. This application does not limit this. If all operations are completed in the client device, the client device may also include a processor.

[0113] The client device may have a communication module (i.e., a communication unit) that can communicate with a remote server to achieve data transmission with the server. The server may include a server on the task scheduling center side, and other implementation scenarios may also include a server on an intermediate platform, such as a server on a third-party server platform that has a communication link with the task scheduling center server. The server may include a single computer device, or a server cluster consisting of multiple servers, or a server structure of a distributed device.

[0114] Fig.10 FIG. 9 is a schematic block diagram of the system structure of the electronic device 9600 according to an embodiment of the present application. Fig.10 As shown, the electronic device 9600 may include a central processor 9100 and a memory 9140; the memory 9140 is coupled to the central processor 9100. It is worth noting that Fig.10 is exemplary; other types of structures may also be used to supplement or replace this structure to implement telecommunication functions or other functions.

[0115] In one embodiment, the hook trajectory prediction method function based on IoT perception can be integrated into the central processor 9100. The central processor 9100 can be configured to perform the following control: Step S101: extract and match feature points using a video image sequence captured by cameras at both ends of the tower crane boom, construct an initial three-dimensional point cloud of the lower area of ​​the tower boom, optimize and fuse the initial three-dimensional point cloud using an iterative closest point registration algorithm, and identify and extract the spatial contours of the tower crane, hook, boom, material area and destination based on a semantic segmentation method, map the feature points of the spatial contours to the park coordinate system through coordinate transformation, establish a traversable area map within the hook motion range using an adaptive voxel grid division algorithm, and calibrate the initial obstacle position using a regional growing method; Step S102: Calculate the three-dimensional displacement vector from the current position of the hook to the target position, project the three-dimensional displacement vector into the local coordinate system established with the boom axis as the reference, use the quaternion-based vector decomposition algorithm to obtain the three orthogonal components of the gravity direction, the direction along the boom and the direction perpendicular to the boom, calculate the rotation arc of the boom rotary motor according to the component perpendicular to the boom direction combined with the current trolley position, and obtain the motor angle through transmission ratio conversion, obtain the angle parameters of the motors corresponding to the lifting mechanism and the luffing mechanism through the kinematic inverse solution of the transmission system for the gravity direction and the component along the boom direction, and use the motor collaborative optimization algorithm to generate a smooth control sequence; Step S103: Based on the detection data of the millimeter-wave radars on both sides of the boom and the ultrasonic radar at the hook, an adaptive threshold segmentation algorithm is used to identify the outline of the obstacle, and the principal component analysis method is used to determine the main axis direction of the minimum envelope cuboid of the detected obstacle area, and a local coordinate system based on the main axis is established, and the minimum envelope cuboid is safely expanded by a preset distance in the local coordinate system, and the obstacle avoidance path is calculated using the preset fast random expansion tree algorithm with the vertices of the expanded cuboid as sampling nodes, and the obstacle avoidance path is decomposed into several segments of displacement vectors, and the kinematic solutions of the several segments of displacement vectors are respectively performed to obtain the corresponding motor angle sequences, and the dynamic path planning algorithm is used to smooth the motor angle sequence and update the control parameters in real time.

[0116] From the above description, it can be seen that the electronic device provided in the embodiment of the present application can achieve accurate environmental perception and obstacle recognition by making full use of multi-source sensor data, has intelligent trajectory planning capabilities, and can ensure the real-time and smoothness of the control system. By introducing advanced technologies such as deep learning and multi-sensor fusion, the overall performance and reliability of the system are improved, thereby better meeting the needs of modern construction.

[0117] In another embodiment, the hook trajectory prediction device based on Internet of Things perception can be configured separately from the central processor 9100. For example, the hook trajectory prediction device based on Internet of Things perception can be configured as a chip connected to the central processor 9100, and the function of the hook trajectory prediction method based on Internet of Things perception can be realized through the control of the central processor.

[0118] like Fig.10 As shown, the electronic device 9600 may also include: a communication module 9110, an input unit 9120, an audio processor 9130, a display 9160, and a power supply 9170. It is worth noting that the electronic device 9600 does not necessarily have to include Fig.10 In addition, the electronic device 9600 may also include Fig.10 For components not shown, reference may be made to the prior art.

[0119] like Fig.10 As shown, the central processing unit 9100 is sometimes also referred to as a controller or an operation control, and may include a microprocessor or other processor device and / or logic device. The central processing unit 9100 receives input and controls the operation of various components of the electronic device 9600.

[0120] The memory 9140 may be, for example, one or more of a cache, a flash memory, a hard drive, a removable medium, a volatile memory, a non-volatile memory or other suitable devices. The above-mentioned information related to the failure may be stored, and a program for executing the relevant information may also be stored. The CPU 9100 may execute the program stored in the memory 9140 to implement information storage or processing, etc.

[0121] The input unit 9120 provides input to the central processing unit 9100. The input unit 9120 is, for example, a key or a touch input device. The power supply 9170 is used to provide power to the electronic device 9600. The display 9160 is used to display display objects such as images and texts. The display may be, for example, an LCD display, but is not limited thereto.

[0122] The memory 9140 may be a solid-state memory, such as a read-only memory (ROM), a random access memory (RAM), a SIM card, etc. It may also be a memory that saves information even when the power is off, can be selectively erased, and is provided with more data, examples of which are sometimes referred to as EPROMs, etc. The memory 9140 may also be some other type of device. The memory 9140 includes a buffer memory 9141 (sometimes referred to as a buffer). The memory 9140 may include an application / function storage unit 9142, which is used to store application programs and function programs or processes for executing the operation of the electronic device 9600 through the central processor 9100.

[0123] The memory 9140 may also include a data storage unit 9143 for storing data, such as contacts, digital data, pictures, sounds, and / or any other data used by the electronic device. The driver storage unit 9144 of the memory 9140 may include various drivers for communication functions of the electronic device and / or for executing other functions of the electronic device (such as messaging applications, address book applications, etc.).

[0124] The communication module 9110 is a transmitter / receiver that sends and receives signals via the antenna 9111. The communication module 9110 (transmitter / receiver) is coupled to the central processor 9100 to provide input signals and receive output signals, which may be the same as the case of a conventional mobile communication terminal.

[0125] Based on different communication technologies, multiple communication modules 9110 may be provided in the same electronic device, such as a cellular network module, a Bluetooth module and / or a wireless LAN module. The communication module 9110 (transmitter / receiver) is also coupled to a speaker 9131 and a microphone 9132 via an audio processor 9130 to provide an audio output via the speaker 9131 and receive an audio input from the microphone 9132, thereby realizing a common telecommunication function. The audio processor 9130 may include any suitable buffer, decoder, amplifier, etc. In addition, the audio processor 9130 is also coupled to the central processor 9100, so that recording can be performed on the local machine through the microphone 9132, and the sound stored on the local machine can be played through the speaker 9131.

[0126] The embodiments of the present application also provide a computer-readable storage medium capable of implementing all the steps of the method for predicting a hook trajectory based on IoT perception in the above-mentioned embodiment, where the execution subject is a server or a client. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, all the steps of the method for predicting a hook trajectory based on IoT perception in the above-mentioned embodiment are implemented. For example, when the processor executes the computer program, the following steps are implemented: Step S101: extract and match feature points using a video image sequence captured by cameras at both ends of the tower crane boom, construct an initial three-dimensional point cloud of the lower area of ​​the tower boom, optimize and fuse the initial three-dimensional point cloud using an iterative closest point registration algorithm, and identify and extract the spatial contours of the tower crane, hook, boom, material area and destination based on a semantic segmentation method, map the feature points of the spatial contours to the park coordinate system through coordinate transformation, establish a traversable area map within the hook motion range using an adaptive voxel grid division algorithm, and calibrate the initial obstacle position using a regional growing method; Step S102: Calculate the three-dimensional displacement vector from the current position of the hook to the target position, project the three-dimensional displacement vector into the local coordinate system established with the boom axis as the reference, use the quaternion-based vector decomposition algorithm to obtain the three orthogonal components of the gravity direction, the direction along the boom and the direction perpendicular to the boom, calculate the rotation arc of the boom rotary motor according to the component perpendicular to the boom direction combined with the current trolley position, and obtain the motor angle through transmission ratio conversion, obtain the angle parameters of the motors corresponding to the lifting mechanism and the luffing mechanism through the kinematic inverse solution of the transmission system for the gravity direction and the component along the boom direction, and use the motor collaborative optimization algorithm to generate a smooth control sequence; Step S103: Based on the detection data of the millimeter-wave radars on both sides of the boom and the ultrasonic radar at the hook, an adaptive threshold segmentation algorithm is used to identify the outline of the obstacle, and the principal component analysis method is used to determine the main axis direction of the minimum envelope cuboid of the detected obstacle area, and a local coordinate system based on the main axis is established, and the minimum envelope cuboid is safely expanded by a preset distance in the local coordinate system, and the obstacle avoidance path is calculated using the preset fast random expansion tree algorithm with the vertices of the expanded cuboid as sampling nodes, and the obstacle avoidance path is decomposed into several segments of displacement vectors, and the kinematic solutions of the several segments of displacement vectors are respectively performed to obtain the corresponding motor angle sequences, and the dynamic path planning algorithm is used to smooth the motor angle sequence and update the control parameters in real time.

[0127] From the above description, it can be seen that the computer-readable storage medium provided in the embodiment of the present application can achieve accurate environmental perception and obstacle recognition by making full use of multi-source sensor data, has intelligent trajectory planning capabilities, and can ensure the real-time and smoothness of the control system. By introducing advanced technologies such as deep learning and multi-sensor fusion, the overall performance and reliability of the system are improved, thereby better meeting the needs of modern construction.

[0128] The embodiments of the present application also provide a computer program product capable of implementing all the steps of the hook trajectory prediction method based on IoT perception in the above embodiments, where the execution subject is a server or a client. When the computer program / instruction is executed by a processor, the steps of the hook trajectory prediction method based on IoT perception are implemented. For example, the computer program / instruction implements the following steps: Step S101: extract and match feature points using a video image sequence captured by cameras at both ends of the tower crane boom, construct an initial three-dimensional point cloud of the lower area of ​​the tower boom, optimize and fuse the initial three-dimensional point cloud using an iterative closest point registration algorithm, and identify and extract the spatial contours of the tower crane, hook, boom, material area and destination based on a semantic segmentation method, map the feature points of the spatial contours to the park coordinate system through coordinate transformation, establish a traversable area map within the hook motion range using an adaptive voxel grid division algorithm, and calibrate the initial obstacle position using a regional growing method; Step S102: Calculate the three-dimensional displacement vector from the current position of the hook to the target position, project the three-dimensional displacement vector into the local coordinate system established with the boom axis as the reference, use the quaternion-based vector decomposition algorithm to obtain the three orthogonal components of the gravity direction, the direction along the boom and the direction perpendicular to the boom, calculate the rotation arc of the boom rotary motor according to the component perpendicular to the boom direction combined with the current trolley position, and obtain the motor angle through transmission ratio conversion, obtain the angle parameters of the motors corresponding to the lifting mechanism and the luffing mechanism through the kinematic inverse solution of the transmission system for the gravity direction and the component along the boom direction, and use the motor collaborative optimization algorithm to generate a smooth control sequence; Step S103: Based on the detection data of the millimeter-wave radars on both sides of the boom and the ultrasonic radar at the hook, an adaptive threshold segmentation algorithm is used to identify the outline of the obstacle, and the principal component analysis method is used to determine the main axis direction of the minimum envelope cuboid of the detected obstacle area, and a local coordinate system based on the main axis is established, and the minimum envelope cuboid is safely expanded by a preset distance in the local coordinate system, and the obstacle avoidance path is calculated using the preset fast random expansion tree algorithm with the vertices of the expanded cuboid as sampling nodes, and the obstacle avoidance path is decomposed into several segments of displacement vectors, and the kinematic solutions of the several segments of displacement vectors are respectively performed to obtain the corresponding motor angle sequences, and the dynamic path planning algorithm is used to smooth the motor angle sequence and update the control parameters in real time.

[0129] From the above description, it can be seen that the computer program product provided in the embodiment of the present application can achieve accurate environmental perception and obstacle recognition by making full use of multi-source sensor data, has intelligent trajectory planning capabilities, and can ensure the real-time and smoothness of the control system. By introducing advanced technologies such as deep learning and multi-sensor fusion, the overall performance and reliability of the system are improved, thereby better meeting the needs of modern construction.

[0130] It should be understood by those skilled in the art that embodiments of the present invention may be provided as methods, devices, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0131] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (apparatus), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0132] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0133] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0134] The present invention uses specific embodiments to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A hook trajectory prediction method based on IoT perception, characterized in that: The method comprises: The video image sequence collected by the cameras at both ends of the tower crane boom is used to extract and match feature points, and the initial three-dimensional point cloud of the lower area of ​​the tower boom is constructed. The initial three-dimensional point cloud is optimized and fused using an iterative closest point registration algorithm, and the spatial contours of the tower crane, hook, boom, material area and destination are identified and extracted based on the semantic segmentation method. The feature points of the spatial contours are mapped to the park coordinate system through coordinate transformation, and an adaptive voxel grid division algorithm is used to establish a traversable area map within the hook movement range, and the initial obstacle position is calibrated by the regional growing method; Calculate the three-dimensional displacement vector from the current position of the hook to the target position, project the three-dimensional displacement vector to the local coordinate system established with the boom axis as the reference, use the quaternion-based vector decomposition algorithm to obtain the three orthogonal components of the gravity direction, the direction along the boom direction and the direction perpendicular to the boom direction, calculate the rotation arc of the boom rotary motor according to the component perpendicular to the boom direction combined with the current trolley position, and obtain the motor angle through transmission ratio conversion, obtain the angle parameters of the motors corresponding to the lifting mechanism and the luffing mechanism through the kinematic inverse solution of the transmission system for the gravity direction and the component along the boom direction, and use the motor collaborative optimization algorithm to generate a smooth control sequence; Based on the detection data of the millimeter-wave radar on both sides of the boom and the ultrasonic radar at the hook, an adaptive threshold segmentation algorithm is used to identify the outline of the obstacle. The principal component analysis method is used to determine the main axis direction of the minimum envelope cuboid of the detected obstacle area, and a local coordinate system based on the main axis is established. The minimum envelope cuboid is safely expanded by a preset distance in the local coordinate system. The obstacle avoidance path is calculated using the vertices of the expanded cuboid as sampling nodes using a preset fast random expanding tree algorithm. The obstacle avoidance path is decomposed into several segments of displacement vectors. The kinematic solutions of the several segments of displacement vectors are respectively performed to obtain the corresponding motor angle sequences. The dynamic path planning algorithm is used to smooth the motor angle sequence and update the control parameters in real time.

2. The method for predicting hook trajectory based on IoT perception according to claim 1 is characterized in that: The video image sequence collected by the cameras at both ends of the tower crane boom is used to extract and match feature points, construct an initial three-dimensional point cloud of the lower area of ​​the tower boom, optimize and fuse the initial three-dimensional point cloud using an iterative closest point registration algorithm, and identify and extract the spatial contours of the tower crane, hook, boom, material area and destination based on a semantic segmentation method, including: The SIFT algorithm is used to extract feature points from each frame of the video image sequence, and the position information, gradient direction and scale information of the feature points are combined into feature descriptors. The Euclidean distance between feature descriptors of different frame images is calculated based on the KD tree search algorithm, and the RANSAC algorithm is used to eliminate wrong matching point pairs and establish the corresponding relationship between images; The 3D coordinates of the feature points in the scene are calculated based on the calibrated binocular camera parameters and the matching relationship of the feature points. The Delaunay triangulation is used to construct the grid topology structure of the feature points to generate the initial 3D point cloud. The iterative closest point registration algorithm is used to register and fuse the multi-frame point clouds on the initial 3D point cloud. The deep learning semantic segmentation network is used to identify the scene elements on the fused point cloud, and the region growing algorithm is used to extract the spatial contours of each scene element.

3. The method for predicting hook trajectory based on IoT perception according to claim 1 is characterized in that: The feature points of the spatial contour are mapped to the park coordinate system through coordinate transformation, an adaptive voxel grid division algorithm is used to establish a traversable area map within the range of hook movement, and the initial obstacle position is calibrated by the regional growing method, including: A local coordinate system is established with the center of the tower crane base, and a marking point is set on the tower crane boom. The rotation matrix and translation vector of the marking point between the park coordinate system and the local coordinate system are calculated by the least squares method, and the rotation matrix and translation vector are applied to the spatial contour feature points to complete the coordinate transformation; A uniform voxel grid is constructed within the motion range of the hook, and the voxel grid is dynamically subdivided based on an adaptive octree. A region growing algorithm is used to expand outward from the edge points of the obstacle as seed points to generate an initial obstacle position calibration result. The boundary of the calibration result is optimized to obtain a traversable area map.

4. The method for predicting hook trajectory based on IoT perception according to claim 1 is characterized in that: The three-dimensional displacement vector from the current position of the hook to the target position is calculated, and the three-dimensional displacement vector is projected into a local coordinate system established with the boom axis as a reference. A vector decomposition algorithm based on quaternion is used to obtain three orthogonal components in the gravity direction, along the boom direction, and perpendicular to the boom direction, including: Collect sensor data at the hook to obtain the current spatial position coordinates, establish a displacement vector based on the target position coordinates, map the displacement vector to the park coordinate system through linear transformation, and use a coordinate rotation matrix to project the displacement vector to a local coordinate system based on the boom axis; A quaternion rotation matrix based on the direction of gravity is constructed, and the rotation matrix is ​​applied to the displacement vector in the local coordinate system. The gravity direction component is decomposed using quaternion multiplication. A second quaternion rotation matrix is ​​constructed based on the direction of the boom axis to decompose the component along the boom direction, and the component perpendicular to the boom direction is calculated through orthogonal relationship.

5. The method for predicting hook trajectory based on IoT perception according to claim 1 is characterized in that: The rotation arc of the boom slewing motor is calculated according to the vertical boom direction component combined with the current trolley position, and the motor angle is obtained by conversion through the transmission ratio. The gravity direction and the component along the boom direction are obtained through the kinematic inverse solution of the transmission system to obtain the angle parameters of the motors corresponding to the hoisting mechanism and the luffing mechanism, and a motor collaborative optimization algorithm is used to generate a smooth control sequence, including: A coordinate system transformation matrix is ​​established based on the angle between the boom axis and the vertical boom direction component, and the transformation matrix is ​​applied to the trolley position coordinates to obtain the boom rotation arc. The angle value required for the rotation motor is calculated based on the reduction ratio and mechanical transmission ratio of the boom transmission system; The rotation angle of the lifting motor is calculated according to the corresponding relationship between the gravity direction component and the length of the suspension rope, and the rotation angle of the variable-length motor is calculated according to the geometric position relationship between the component along the boom direction and the variable-length mechanism. The rotation angle sequence of the slewing motor, the lifting motor and the variable-length motor is optimized with speed and acceleration constraints to obtain smooth control parameters.

6. The method for predicting hook trajectory based on IoT perception according to claim 1 is characterized in that: Based on the detection data of the millimeter wave radar on both sides of the boom and the ultrasonic radar at the hook, an adaptive threshold segmentation algorithm is used to identify the outline of the obstacle, the detected obstacle area is determined by the principal component analysis method to determine the main axis direction of the minimum envelope cuboid, and a local coordinate system based on the main axis is established, including: The distance and reflection intensity data of the millimeter-wave radars on both sides of the boom and the distance data of the ultrasonic radar at the hook are collected respectively, and the noise points are filtered out by an adaptive threshold method for the radar data, and the filtered data are clustered and segmented to obtain the obstacle area contour point set; A covariance matrix is ​​calculated based on the obstacle contour point set, and the covariance matrix is ​​subjected to eigenvalue decomposition to obtain a principal axis direction vector. A local coordinate system is established based on the principal axis direction vector, and the obstacle contour point set is mapped to the local coordinate system to calculate the vertex coordinates of the minimum envelope cuboid.

7. The method for predicting hook trajectory based on IoT perception according to claim 1 is characterized in that: The method comprises: safely expanding the minimum envelope cuboid by a preset distance in the local coordinate system, calculating an obstacle avoidance path by using a preset fast random expansion tree algorithm with the vertices of the expanded cuboid as sampling nodes, decomposing the obstacle avoidance path into a plurality of displacement vectors, performing kinematics calculations on the plurality of displacement vectors to obtain corresponding motor rotation angle sequences, and using a dynamic path planning algorithm to smooth the motor rotation angle sequence and update control parameters in real time, including: Uniformly expand the vertex coordinates of the minimum enveloping cuboid by a preset distance, construct a search space with the vertices of the expanded cuboid as sampling nodes, search for a feasible path from the current position of the hook to the target position in the search space using an improved fast random expanding tree algorithm, and divide the feasible path into a number of displacement vectors at preset distance intervals; The displacement vector obtained by decomposing the obstacle avoidance path is substituted into the kinematic equation of the mechanism to calculate the rotation angle parameters of each joint motor. The rotation angle parameters are interpolated and fitted using Bezier curves to obtain a continuous angle sequence. The angle sequence is online optimized and updated according to the real-time collected sensor data to obtain the motor control instructions.

8. A hook trajectory prediction device based on IoT perception, characterized in that: The device comprises: The module for determining the initial obstacle position is used to extract and match feature points using the video image sequence collected by the cameras at both ends of the tower crane's boom, construct an initial three-dimensional point cloud of the lower area of ​​the tower boom, optimize and fuse the initial three-dimensional point cloud using an iterative closest point registration algorithm, and identify and extract the spatial contours of the tower crane, hook, boom, material area and destination based on a semantic segmentation method, map the feature points of the spatial contours to the park coordinate system through coordinate transformation, establish a traversable area map within the hook's motion range using an adaptive voxel grid division algorithm, and calibrate the initial obstacle position using a regional growing method; The motor control sequence generation module is used to calculate the three-dimensional displacement vector from the current position of the hook to the target position, project the three-dimensional displacement vector to the local coordinate system established with the boom axis as the reference, and use the quaternion-based vector decomposition algorithm to obtain the three orthogonal components of the gravity direction, the direction along the boom direction and the direction perpendicular to the boom direction. The rotation arc of the boom rotary motor is calculated according to the component perpendicular to the boom direction combined with the current trolley position, and the motor angle is obtained by conversion through the transmission ratio. The gravity direction and the component along the boom direction are obtained through the kinematic inverse solution of the transmission system to obtain the angle parameters of the motors corresponding to the lifting mechanism and the luffing mechanism, and a motor collaborative optimization algorithm is used to generate a smooth control sequence; The intelligent obstacle avoidance module is used to identify the obstacle contour based on the detection data of the millimeter-wave radars on both sides of the boom and the ultrasonic radar at the hook, and adopts an adaptive threshold segmentation algorithm to identify the obstacle contour, determine the main axis direction of the minimum envelope cuboid of the detected obstacle area through the principal component analysis method, and establish a local coordinate system based on the main axis, safely expand the minimum envelope cuboid by a preset distance in the local coordinate system, use a preset fast random expansion tree algorithm to calculate the obstacle avoidance path with the vertices of the expanded cuboid as sampling nodes, decompose the obstacle avoidance path into several segments of displacement vectors, perform kinematic solutions on the several segments of displacement vectors respectively to obtain the corresponding motor angle sequence, and use a dynamic path planning algorithm to smooth the motor angle sequence and update the control parameters in real time.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps of the hook trajectory prediction method based on Internet of Things perception as described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the hook trajectory prediction method based on Internet of Things perception described in any one of claims 1 to 7 are implemented.

Images