A tower crane path planning method based on edge computing
By using edge computing and an improved deep Q-network (RDQN), combined with tower crane environmental data and operational data, a stable tower crane path planning method is generated, which solves the problems of unstable and inefficient path planning in existing technologies and achieves safe and efficient path planning in complex construction site environments.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- CHINA TELECOM DIGITAL INTELLIGENCE TECHNOLOGY (ANHUI) CO LTD
- Filing Date
- 2026-03-10
- Publication Date
- 2026-06-02
AI Technical Summary
Existing tower crane path planning methods rely on operator experience or static rules, making it difficult to ensure the safety and repeatability of hook movement paths in complex construction site environments. Furthermore, reinforcement learning methods suffer from sparse learning signals and unstable training in continuous state scenarios, making it difficult to generate reasonable control action sequences.
An improved deep Q-network RDQN is constructed based on edge computing. By combining tower crane environmental data and operational data, and through multi-dimensional reward mechanisms and weighted parameter synchronization, a stable tower crane path planning method is generated, thereby improving the reliability of path planning and operational efficiency.
Achieving stability and safety in tower crane path planning in complex construction site environments improves the reliability and efficiency of path planning, reduces collision risks, and enhances learning efficiency.
Smart Images
Figure CN122126753A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of smart construction site technology, and in particular to a tower crane path planning method based on edge computing. Background Technology
[0002] Tower cranes are key equipment for material handling on construction sites. Their operation involves the coordinated control of multiple mechanisms, including hoisting, luffing, and slewing. The movement path of the hook between the lifting point and the target landing point is often affected by factors such as narrow site space, complex obstacle distribution, and variations in the weight and rope length. Currently, tower crane operation on construction sites largely relies on the operator's experience for visual judgment and manual control, or on pre-set operating procedures and simple trajectory rules to complete lifting tasks.
[0003] However, traditional experience-based control methods rely heavily on driver proficiency. When faced with dense obstacles, dynamically changing work environments, or limited visibility, it's difficult to consistently ensure the safety and repeatability of the hook's movement path, leading to problems such as the hook being too close to obstacles, unreasonable path detours, or reduced work efficiency. Rule-based or static planning methods typically require idealized environmental modeling and lack the ability to adapt to real-time changes in the construction site environment, updated obstacle positions, and changes in equipment status, making it difficult to continuously output reasonable control action sequences in complex scenarios.
[0004] Furthermore, while existing reinforcement learning methods commonly used in path planning, such as DQN, can learn decision-making strategies through the main Q-network and the target Q-network, in continuous state scenarios like tower crane path planning, if the state space and reward mechanism are not designed sufficiently, problems such as sparse learning signals, unstable training, and slow convergence speed can easily occur. For example, if a reward is only given when the target point is reached or a collision occurs, most steps in the training process will lack effective feedback, making it difficult to guide the model to gradually approach the target and effectively avoid obstacles.
[0005] Therefore, how to provide a tower crane path planning method based on edge computing is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention
[0006] One objective of this invention is to propose a tower crane path planning method based on edge computing. This invention is based on edge computing and an improved deep Q-network (RDQN), and constructs a state space by combining tower crane environmental data and operation data. Stable training and online calculation are achieved through a multi-dimensional reward mechanism and weighted parameters. This method can safely and efficiently generate tower crane operation paths in complex construction site environments, thereby improving the reliability of path planning and operational efficiency.
[0007] A tower crane path planning method based on edge computing according to an embodiment of the present invention includes the following steps:
[0008] Training data and environmental data for tower crane path planning are acquired at edge computing nodes on the construction site.
[0009] Construct a state space based on environmental data;
[0010] An improved deep Q-network RDQN is constructed, and the RDQN is trained using training data and state space to obtain a network model for tower crane path planning. The RDQN includes a main Q-network and a target Q-network.
[0011] During the training of the network model, a corresponding reward value is generated based on the state space. The reward value is synthesized from dynamic reward, target point reward and obstacle reward, and the parameters of the network model are updated based on the reward value.
[0012] The network parameters of the main Q network are synchronized to the network parameters of the target Q network, and the network parameters of the target Q network are updated in a weighted manner.
[0013] Based on the trained network model, online path planning is performed, the current state space is input into the main Q network, and the corresponding tower crane control action is output.
[0014] The tower crane control actions are converted into tower crane control commands to drive the tower crane to perform operations. After continuously outputting and executing the tower crane control actions, an action set consisting of tower crane control actions is formed. When the target landing point arrival condition is met, the action set is output as the tower crane path planning result.
[0015] Optionally, the acquisition of the training data and environment data specifically includes:
[0016] Establish training data collection tasks at edge computing nodes on the construction site, collect and aggregate tower crane operation records of tower crane operators during lifting, operation and lowering processes, and organize them into training data;
[0017] An environmental data acquisition task is established at the edge computing node on the construction site. The radar and camera collect environmental information and obstacle location information near the tower crane to generate environmental perception data, acquire construction site scene information to generate scene data, and acquire tower crane equipment information to generate equipment data.
[0018] At the construction site, edge computing nodes acquire the latitude and longitude of the tower crane lifting point and the target landing point, as well as the size of the suspended object and the length of the lifting rope. Environmental perception data, scene data, equipment data, the latitude and longitude of the tower crane lifting point and the target landing point, the size of the suspended object and the length of the lifting rope are aggregated and organized to generate environmental data.
[0019] Optionally, the construction of the state space specifically includes:
[0020] At the edge computing node, the spatial distance from the current position of the hook to the target landing point is calculated based on the latitude and longitude of the tower crane lifting point and the target landing point, to obtain the first distance state quantity. Then, based on the latitude and longitude of the tower crane lifting point and the position information of the obstacle, the spatial distance from the current position of the hook to the position of the obstacle is calculated, to obtain the second distance state quantity.
[0021] At the edge computing node, the first distance state quantity and the second distance state quantity are used as the state input at the current moment;
[0022] After each movement of the tower crane, the latitude and longitude of the crane's lifting point and the target landing point, as well as the location information of obstacles, are updated at the edge computing node, and the updated state input after the action is executed is generated to form a state space.
[0023] Optionally, the process of obtaining the network model for the tower crane path planning specifically includes:
[0024] An improved deep Q network RDQN is constructed at the edge computing nodes on the construction site. A master Q network and a target Q network are established in the RDQN. The master Q network is configured with a network structure for receiving the state space, and the target Q network is configured with a network structure for providing target value evaluation results during training.
[0025] Initial network parameters are generated for the main Q-network and the target Q-network respectively, and written into the corresponding parameter storage area;
[0026] The training data and the constructed state space are called up. The tower crane operation records corresponding to the training time are read from the training data and aligned with the state space corresponding to the training time to form sample records.
[0027] At the edge computing node, sample records are written into the experience pool, and training batch samples are randomly selected from it. At the same time, the state space in the sample records is sent into the main Q network to obtain the value evaluation results of the main Q network for the corresponding sample operation actions in the sample records. The value evaluation results are then associated with and stored with the sample records.
[0028] The state space corresponding to the sample operation action is obtained and fed into the target Q network to obtain the target value evaluation result of the next time step. The target value evaluation result is used as the target basis for updating the parameters of the main Q network to train and iterate the main Q network to obtain the network model of tower crane path planning.
[0029] Optionally, the step of synchronizing the network parameters of the primary Q-network to the network parameters of the target Q-network, and simultaneously updating the network parameters of the target Q-network using a weighted method, specifically involves:
[0030] During network model training, the state space corresponding to the current training time is obtained, and the sample operation action corresponding to the current training time is obtained and executed to complete one state transition. The state space of the next time after the state transition is obtained, and the values of the first distance state quantity and the second distance state quantity at the next time are read.
[0031] Based on the current and next time values of the first and second distance state variables, the target distance change and obstacle distance change are determined. A base reward value, a target distance reward weight, and an obstacle distance reward weight are set. The obstacle distance change and the obstacle distance reward weight are weighted to obtain the obstacle distance contribution item. The target distance change and the target distance reward weight are weighted to obtain the target distance contribution item. The obstacle distance contribution item and the target distance contribution item are superimposed, or the target distance contribution item is subtracted from the obstacle distance contribution item according to a preset method to obtain a dynamic reward. The dynamic reward is then combined with the base reward value to obtain the dynamic reward value.
[0032] Set a target landing point arrival threshold, compare the current value of the first distance state quantity with the target landing point arrival threshold, and obtain the target point reward;
[0033] Set the minimum collision distance and obstacle range distance parameters, compare the current value of the second distance state quantity with the minimum collision distance and obstacle range distance, and obtain the obstacle reward;
[0034] The reward value for this state transition is obtained by combining the dynamic reward value, the target point reward, and the obstacle reward.
[0035] The network model parameters are updated based on sample records to obtain the state space at the next time step, which is then fed into the target Q-network. The target value evaluation result of the target Q-network is obtained, and the maximum target value evaluation result is selected. A decay factor and learning rate are set, and the reward value is combined with the maximum target value evaluation result according to the decay factor. The error between the reward value and the value evaluation result of the main Q-network is calculated, and a loss is constructed. The loss is backpropagated in the main Q-network, and the network parameters of the main Q-network are updated according to the learning rate using stochastic gradient descent.
[0036] Optionally, the step of synchronizing the network parameters of the primary Q-network to the network parameters of the target Q-network, and simultaneously updating the network parameters of the target Q-network using a weighted method, specifically involves:
[0037] During network model training, the current network parameters of the main Q-network are read as input to the main network parameters, and the current network parameters of the target Q-network are read as input to the target network parameters.
[0038] Weighting coefficients are set at the edge computing nodes, and weighted update processing is performed on the input parameters of the main network and the input parameters of the target network when parameter synchronization is triggered. The input parameters of the main network are weighted according to the weighting coefficients to obtain the weighted parameters of the main network. The input parameters of the target network are weighted according to the complement of the weighting coefficients to obtain the weighted parameters of the target network. The weighted parameters of the main network and the weighted parameters of the target network are added item by item to generate the updated parameters of the target Q network.
[0039] At the edge computing node, the updated parameters of the target Q network are written into the parameter storage area of the target Q network to replace the original network parameters of the target Q network, thus completing the weighted synchronous update of the network parameters of the main Q network to the network parameters of the target Q network.
[0040] Optionally, the output of the corresponding tower crane control action specifically includes:
[0041] The edge computing node at the construction site calls the trained network model, constructs a set of candidate actions based on the control methods of the three main mechanisms of the tower crane, and writes them into the action candidate buffer.
[0042] Online path planning is performed based on the candidate action set. The state space corresponding to the current training time is input into the main Q network, and the main Q network is called to evaluate the value of each candidate action in the candidate action set, so as to obtain the action value evaluation result corresponding to each candidate action.
[0043] The motion value assessment results are compared, and the candidate motion with the highest corresponding motion value assessment result is selected as the tower crane control motion at the current moment.
[0044] Optionally, obtaining the tower crane path planning result specifically includes:
[0045] At the edge computing node on the construction site, the tower crane control actions are parsed into up and down control quantities, forward and backward control quantities, and rotation control quantities, which are mapped into tower crane rotation mechanism control commands, up and down mechanism control commands, and forward and backward mechanism control commands, respectively, and combined to generate the tower crane control command set at the current moment.
[0046] After the tower crane executes the crane control command set, it obtains the updated position information of the hook and calculates the updated distance from the hook to the target landing point. The updated distance is then compared with the target landing point arrival threshold to determine whether the target landing point arrival condition is met.
[0047] If the target landing point is not met, the current tower crane control action is added to the action set.
[0048] When the target landing point is reached, the current tower crane control actions are added to the action set and output as the tower crane path planning result.
[0049] The beneficial effects of this invention are:
[0050] This invention constructs a state space based on environmental data in tower crane path planning, including the distance between the hook and the target landing point and the distance between the hook and obstacles. It also combines the tower crane operator's operation records as training data, enabling the improved deep Q network RDQN to learn decision-making strategies that are more in line with the actual situation on the construction site in continuous state scenarios. This overcomes the shortcomings of traditional mechanized single-line calculation planning, which is prone to getting trapped in local optima and has many inflection points and is not smooth enough, thus improving the feasibility and applicability of tower crane path planning.
[0051] This invention generates a reward value composed of dynamic reward, target point reward, and obstacle reward based on the state space during network model training. The network model parameters are updated using this reward value, so that the model can obtain continuous and effective feedback guidance during the planning process other than reaching the target point and collision. It can simultaneously strengthen the gradual approach of the hook to the target landing point and maintain a safe distance from obstacles, thereby improving the safety and learning efficiency of the planning process.
[0052] This invention employs a weighted approach to update the target Q-network parameters during the parameter synchronization process between the main Q-network and the target Q-network. Compared to the traditional hard synchronization method for the target network, this approach can mitigate value estimation fluctuations and improve training stability. This enables the resulting network model to more stably calculate each tower crane operation step during the online path planning phase and continuously output tower crane control actions to form an action set. When the target landing point arrival condition is met, the output is used as the final path planning result, thereby further improving the stability and operational efficiency of tower crane path planning. Attached Figure Description
[0053] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:
[0054] Figure 1 This is a flowchart illustrating the overall scheme of a tower crane path planning method based on edge computing proposed in this invention.
[0055] Figure 2 This is a flowchart illustrating the online path planning process of a tower crane path planning method based on edge computing proposed in this invention.
[0056] Figure 3 This is a diagram of the neural network structure in the DQN algorithm of the tower crane path planning method based on edge computing proposed in this invention.
[0057] Figure 4 This invention presents the framework and network update structure diagram of a path planning algorithm for tower crane path planning based on edge computing proposed in this invention.
[0058] Figure 5 This is an overview diagram of a prototype tower crane path planning site management system based on edge computing proposed in this invention.
[0059] Figure 6 This is a user management flowchart of a prototype construction site management system for a tower crane path planning method based on edge computing proposed in this invention.
[0060] Figure 7 This is a flowchart illustrating the tower crane equipment management process of a construction site management system based on an edge computing-based tower crane path planning method proposed in this invention.
[0061] Figure 8 This is a site management flowchart illustrating a prototype of a construction site management system based on an edge computing-based tower crane path planning method proposed in this invention.
[0062] Figure 9 This is a personnel management flowchart for a prototype construction site management system based on an edge computing-based tower crane path planning method proposed in this invention. Detailed Implementation
[0063] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.
[0064] refer to Figures 1-3 A tower crane path planning method based on edge computing includes the following steps:
[0065] Training data and environmental data for tower crane path planning are acquired at edge computing nodes on the construction site.
[0066] Construct a state space based on environmental data;
[0067] An improved deep Q-network RDQN is constructed, and the RDQN is trained using training data and state space to obtain a network model for tower crane path planning. The RDQN includes a main Q-network and a target Q-network.
[0068] During the training of the network model, a corresponding reward value is generated based on the state space. The reward value is synthesized from dynamic reward, target point reward and obstacle reward, and the parameters of the network model are updated based on the reward value.
[0069] The network parameters of the main Q network are synchronized to the network parameters of the target Q network, and the network parameters of the target Q network are updated in a weighted manner.
[0070] Based on the trained network model, online path planning is performed, the current state space is input into the main Q network, and the corresponding tower crane control action is output.
[0071] The tower crane control actions are converted into tower crane control commands to drive the tower crane to perform operations. After continuously outputting and executing the tower crane control actions, an action set consisting of tower crane control actions is formed. When the target landing point arrival condition is met, the action set is output as the tower crane path planning result.
[0072] In this embodiment, the acquisition of training data and environmental data specifically includes:
[0073] Training data acquisition tasks are established at edge computing nodes on the construction site to collect and aggregate tower crane operation records of tower crane operators during lifting, operation and lowering processes, and organize them into training data. The edge computing nodes are local computing units deployed on the construction site to process tower crane and environmental data locally and perform path planning reasoning to reduce latency and improve control real-time performance.
[0074] An environmental data acquisition task is established at the edge computing node on the construction site. The environmental information and obstacle location information near the tower crane are collected by radar and camera to generate environmental perception data. The construction site scene information is also acquired to generate scene data, which includes scene data corresponding to the 3D BIM map. At the same time, tower crane equipment information is acquired to generate equipment data, which includes tower crane geometric parameter data and tower crane speed parameter data of the three major mechanisms.
[0075] At the construction site, edge computing nodes acquire the latitude and longitude of the tower crane lifting point and the target landing point, as well as the size of the suspended object and the length of the lifting rope. They then aggregate and organize environmental perception data, scene data, equipment data, the latitude and longitude of the tower crane lifting point and the target landing point, the size of the suspended object, and the length of the lifting rope to generate environmental data. Finally, they write the training data and environmental data into the storage unit of the edge computing nodes.
[0076] In this embodiment, the construction of the state space specifically includes:
[0077] At the edge computing node, based on the latitude and longitude of the tower crane lifting point and the target landing point, the spatial distance from the current position of the hook to the target landing point is calculated to obtain the first distance state quantity. Based on the latitude and longitude of the tower crane lifting point and the position information of the obstacle, the spatial distance from the current position of the hook to the obstacle is calculated to obtain the second distance state quantity. The first distance state quantity is used to characterize the distance between the hook and the target landing point, and the second distance state quantity is used to characterize the distance between the hook and the obstacle.
[0078] At the edge computing node, the first distance state quantity and the second distance state quantity are used as the state input at the current moment;
[0079] After each movement of the tower crane, the latitude and longitude of the crane's lifting point and the target landing point, as well as the location information of obstacles, are updated at the edge computing node, and the updated state input after the action is executed is generated to form a state space.
[0080] In this embodiment, the process of obtaining the network model for tower crane path planning specifically includes:
[0081] An improved deep Q network RDQN is constructed at the edge computing nodes on the construction site. A master Q network and a target Q network are established in the RDQN. The master Q network is configured with a network structure for receiving the state space, and the target Q network is configured with a network structure for providing target value evaluation results during training.
[0082] Generate initial network parameters for the main Q network and the target Q network respectively and write them into the corresponding parameter storage area, and set the initial network parameters of the target Q network to be consistent with the initial network parameters of the main Q network;
[0083] The training data and the constructed state space are called up. The tower crane operation records corresponding to the training time are read from the training data and aligned with the state space corresponding to the training time to form sample records.
[0084] At the edge computing node, sample records are written into the experience pool, and training batch samples are randomly selected from it. At the same time, the state space in the sample records is sent into the main Q network to obtain the value evaluation result of the main Q network for the sample operation action corresponding to the sample record. The value evaluation result is associated with the sample record and stored. The sample operation action refers to the tower crane operation instruction record extracted from the tower crane operator's tower crane operation record and corresponding to the state space at a certain moment.
[0085] The state space corresponding to the sample operation action is obtained and fed into the target Q network to obtain the target value evaluation result of the next time step. The target value evaluation result is used as the target basis for updating the parameters of the main Q network to train and iterate the main Q network to obtain the network model of tower crane path planning.
[0086] This invention constructs an improved deep Q-network (RDQN) containing a master Q-network and a target Q-network on the construction site. It then trains the network using experience pooling data formed by combining tower crane operator operation records with state space. This enables the network to stably learn the decision-making relationships of tower crane operation in a continuous state environment. The target value assessment provided by the target Q-network guides the parameter updates of the master Q-network, effectively improving the stability and convergence efficiency of model training. This enhances the adaptability of path planning results to complex construction site environments, thereby improving the reliability and practical application value of tower crane path planning.
[0087] In this embodiment, the process of synchronizing the network parameters of the primary Q-network to the network parameters of the target Q-network, and simultaneously updating the network parameters of the target Q-network using a weighted approach, specifically involves:
[0088] During network model training, the state space corresponding to the current training time is obtained, and the sample operation action corresponding to the current training time is obtained and executed to complete one state transition. The state space of the next time after the state transition is obtained, and the values of the first distance state quantity and the second distance state quantity at the next time are read.
[0089] Based on the current and next time values of the first and second distance state variables, the target distance change and obstacle distance change are determined. A base reward value, a target distance reward weight, and an obstacle distance reward weight are set. The obstacle distance change and the obstacle distance reward weight are weighted to obtain the obstacle distance contribution item. The target distance change and the target distance reward weight are weighted to obtain the target distance contribution item. When the obstacle distance change indicates that the hook is moving away from the obstacle, the obstacle distance contribution item and the target distance contribution item are superimposed, or the target distance contribution item is subtracted from the obstacle distance contribution item according to a preset method to obtain a dynamic reward. The dynamic reward is then combined with the base reward value to obtain the dynamic reward value.
[0090] Set a target landing point threshold, compare the current value of the first distance state quantity with the target landing point threshold to obtain the target point reward. When the current value of the first distance state quantity is less than or equal to the target landing point threshold, it is set to the first preset positive reward value; otherwise, it is set to the second preset positive reward value.
[0091] Set the minimum collision distance and obstacle range distance parameters. Compare the current value of the second distance state quantity with the minimum collision distance and obstacle range distance to obtain the obstacle reward. When the current value of the second distance state quantity is less than or equal to the minimum collision distance, set the obstacle reward to a first preset negative value. When the current value of the second distance state quantity is greater than the minimum collision distance and less than the obstacle range distance parameter, set the obstacle reward to a second preset negative value. When the current value of the second distance state quantity is equal to the obstacle range distance parameter, set the obstacle reward or penalty to a third preset negative value.
[0092] The reward value for this state transition is obtained by combining the dynamic reward value, the target point reward, and the obstacle reward.
[0093] The network model parameters are updated based on sample records to obtain the state space at the next time step, which is then fed into the target Q-network. The target value evaluation result of the target Q-network is obtained, and the maximum target value evaluation result is selected. A decay factor and learning rate are set, and the reward value is combined with the maximum target value evaluation result according to the decay factor. The error between the reward value and the value evaluation result of the main Q-network is calculated, and a loss is constructed. The loss is backpropagated in the main Q-network, and the network parameters of the main Q-network are updated according to the learning rate using stochastic gradient descent.
[0094] This invention constructs a dynamic reward based on the changes in the distance between the target and the obstacle in the state space, and combines the target point reward and the obstacle reward to form continuous feedback. This allows the model to obtain a more sufficient learning signal during training than with sparse rewards, thereby guiding the hook to gradually approach the target landing point and maintain a safe distance from the obstacle. At the same time, the loss is constructed by using the target value assessment provided by the target Q network and the value assessment error of the main Q network, and backpropagation is used to update the parameters, which improves the network convergence speed and training stability, reduces the risk of collision, and improves the safety and efficiency of path planning.
[0095] In this embodiment, the process of synchronizing the network parameters of the primary Q-network to the network parameters of the target Q-network, and simultaneously updating the network parameters of the target Q-network using a weighted approach, specifically involves:
[0096] During network model training, the current network parameters of the main Q-network are read as input to the main network parameters, and the current network parameters of the target Q-network are read as input to the target network parameters.
[0097] Weighting coefficients are set at edge computing nodes. The weighting coefficients are values greater than zero and less than one. When parameter synchronization is triggered, weighted update processing is performed on the input parameters of the main network and the input parameters of the target network. The input parameters of the main network are weighted according to the weighting coefficients to obtain the weighted parameters of the main network. The input parameters of the target network are weighted according to the complement of the weighting coefficients to obtain the weighted parameters of the target network. The weighted parameters of the main network and the weighted parameters of the target network are added item by item to generate the updated parameters of the target Q network.
[0098] At the edge computing node, the updated parameters of the target Q network are written into the parameter storage area of the target Q network to replace the original network parameters of the target Q network, thus completing the weighted synchronous update of the network parameters of the main Q network to the network parameters of the target Q network.
[0099] This invention introduces a weighted approach for synchronous parameter updates between the main Q-network and the target Q-network. This eliminates the traditional abrupt parameter copying method used by the target Q-network, instead allowing it to gradually integrate the learning results of the main Q-network in a smooth manner. This effectively reduces the instability caused by oscillations in the value function during training, enhances the continuity and robustness of network parameter evolution, and improves the convergence stability and generalization ability of the improved deep Q-network in complex construction site environments. As a result, the tower crane path planning model can output more stable, continuous, and reliable control actions during the online calculation phase, further improving the safety and overall operational efficiency of tower crane operations.
[0100] In this embodiment, the output of the corresponding tower crane control action specifically includes:
[0101] The edge computing node at the construction site calls the trained network model and constructs a candidate action set based on the control methods of the three major mechanisms of the tower crane. The candidate action set includes the control quantities of slewing direction and slewing amplitude, hoisting direction and hoisting amplitude, and luffing direction and luffing amplitude, and writes them into the action candidate buffer for the main Q network to call for action value evaluation.
[0102] Online path planning is performed based on the candidate action set. The state space corresponding to the current training time is input into the main Q network, and the main Q network is called to evaluate the value of each candidate action in the candidate action set, so as to obtain the action value evaluation result corresponding to each candidate action.
[0103] The motion value assessment results are compared, and the candidate motion with the highest corresponding motion value assessment result is selected as the tower crane control motion at the current moment.
[0104] In this embodiment, obtaining the tower crane path planning result specifically includes:
[0105] At the edge computing node on the construction site, the tower crane control actions are parsed into up and down control quantities, forward and backward control quantities, and rotation control quantities, which are mapped into tower crane rotation mechanism control commands, up and down mechanism control commands, and forward and backward mechanism control commands, respectively. These commands are then combined to generate the tower crane control command set for the current moment, and the tower crane control command set is sent to the tower crane controller to drive the tower crane to perform operations.
[0106] After the tower crane executes the crane control command set, it obtains the updated position information of the hook and calculates the updated distance from the hook to the target landing point. The updated distance is then compared with the target landing point arrival threshold to determine whether the target landing point arrival condition is met.
[0107] If the target landing point is not met, the current tower crane control action is added to the action set and the hook update position information is used to update the current state space so that the subsequent online path planning can continue to output the tower crane control action for the next moment.
[0108] When the target landing point is reached, the current tower crane control actions are added to the action set and output as the tower crane path planning result.
[0109] Example 1:
[0110] To verify the feasibility of this invention in practice, it was applied to a construction site tower crane management and path planning scenario involving multiple tower cranes operating in parallel, densely distributed obstacles, and continuous lifting tasks. In this scenario, the tower crane hook needs to make multiple round trips between the lifting point and the target landing point. Obstacles such as scaffolding, material storage areas, temporary facilities, and vehicles and equipment exist simultaneously in the operating area. The positions of obstacles may change locally as the work organization is adjusted. Traditional methods that rely on single-line mechanized calculation or rule-based simulation for path selection are prone to problems such as many path turning points, excessive detours, and unstable constraints on obstacle distances. On the other hand, sparse reward learning methods that only provide feedback upon arrival or collision can easily lead to slow training convergence and large fluctuations in online calculations, making it difficult to form a set of directly executable actions on-site while balancing safety and efficiency.
[0111] This embodiment uses a construction site management system prototype as the platform, unifying data acquisition, device access, access control, training inference, and result output within a closed loop within the same system. Its overall system framework is as follows: Figure 4 As shown, the system includes a user management module, a tower crane equipment management module, a construction site management module, and a personnel management module. The construction site management module is used to configure the organizational structure of region-company-construction site-work team, so that each path planning task, training data archiving, and model version release can be clearly assigned to a specific construction site and work team. The tower crane equipment management module is used to complete equipment access, online operation management, and equipment information maintenance, so that after the tower crane equipment code is bound to the on-site network port, it can continuously transmit hook position, mechanism status, and environmental perception data and support online inference calls. The personnel management module is used to complete personnel information maintenance, permission issuance, and personnel removal, so that tower crane operation and path planning calls are all executed within the authorized scope. The user management module is used to complete user addition, information modification, deletion, and password change, and bind user roles to supervisor roles, construction site roles, or other roles to restrict the scope of system operation.
[0112] Based on the above framework, the typical business process of the user management module can be referenced. Figure 5After successful system login verification, users can proceed to the user information addition or modification stage. Through role and permission scope configuration, hierarchical control is achieved over training task triggering, model release, online inference calls, and result viewing, thereby reducing the interference of unauthorized operations on training data and online control.
[0113] On the equipment side, the access and operation management process of the tower crane equipment management module is as follows: Figure 6 As shown, after the system inputs information such as device name, location, code, manufacturer, model, and network IP port, it completes device binding and continuously monitors the device status on the online management interface. When the device is disassembled or network parameters are adjusted, the binding relationship can be restored by updating the port, thus ensuring the continuity of training data and online inference input. On the site organization side, the configuration process of the site management module can be referred to... Figure 7 The system establishes a hierarchical connection from regional configuration to company level, then to construction site and work team, ensuring that equipment, personnel, and data are isolated across different regions, companies, and construction sites, preventing model bias caused by cross-project data mixing. On the personnel side, the personnel management module process is as follows: Figure 8 As shown, after adding a new person, their site and equipment permissions can be configured, and permissions can be issued when identity elements such as facial images are available. When a person leaves the site, their control permissions are cleared and the records are removed to ensure that "who can trigger online path planning and who can issue control commands" is traceable and auditable at the system level.
[0114] At the algorithm implementation level, this embodiment strictly follows the training data and environmental data organization method of the claims for collection and aggregation. The training data comes from the tower crane operator's tower crane operation records during the lifting, operation and lowering process. The environmental data comes from the surrounding environment information and obstacle location information of the tower crane collected by radar and cameras, scene data corresponding to the 3D BIM map, tower crane geometric parameters and speed parameters of the three major mechanisms, latitude and longitude of the lifting point and the target landing point, size of the lifted object and length of the lifting rope, etc. All data are written into the storage unit of the edge computing node and associated with the equipment code, construction site and work team.
[0115] The state space is constructed with the target distance and obstacle distance as the core. The edge computing nodes calculate the first distance state quantity based on the current position of the hook and the target landing point, and calculate the second distance state quantity based on the current position of the hook and the obstacle. After each movement of the tower crane, the current position of the hook and the obstacle position information are updated to form the state input for the next moment, thus providing a continuous training sample alignment basis for the master Q network and the target Q network.
[0116] In terms of network structure, the main Q-network receives the state space and outputs the value evaluation result, while the target Q-network provides the target value evaluation result for the next time step. Together, they form the improved deep Q-network RDQN. Its training, updates, and data flow closure are as follows:Figure 3 As shown, the experience pool is used to store sample records and randomly select batches of samples to reduce sample correlation. The main Q-network evaluates the current state input, and the target Q-network provides a target evaluation for the next state input. Together, they form the loss and drive the iterative update of the main Q-network. Simultaneously, this invention uses a weighted approach when synchronizing the main Q-network parameters to the target Q-network, allowing the target network parameters to smoothly merge with the main network parameters, reducing the abrupt changes in value estimation caused by traditional hard synchronization, thereby improving training stability. To facilitate understanding of the representation and calculation of the state space within the main Q-network, this embodiment employs the following... Figure 2 The diagram illustrates the neural network computation structure. The state space input interacts with weights, biases, and activation functions to form the network output. The output characterizes the value differences of different candidate control actions in the current state, thus providing a basis for action selection in the online phase. In implementation, the network input receives a state space composed of a first distance state variable and a second distance state variable. The network output corresponds to the value evaluation results of the candidate action set. The activation function is used to enhance the nonlinear fitting ability. The weights and biases are continuously updated and solidified into network model parameters that can be used for online inference during training iterations.
[0117] When used online, construction site users, provided they have the appropriate equipment control permissions, trigger a path planning task through the system interface. Edge computing nodes call the trained network model, input the current state space into the main Q-network to obtain the candidate action value evaluation results, and select the optimal control action. Subsequently, the tower crane control action is converted into a tower crane control command and issued for execution. The system continuously collects the updated position information of the hook and calculates its updated distance to the target landing point. It compares this distance with the target landing point arrival threshold to determine whether the arrival condition is met. If not, it continues to calculate the next step and adds actions to the action set until the arrival condition is met. The action set is then output as the path planning result. The output result is archived in the system along with the task number, equipment code, personnel permissions, and state sequence for supervision and review.
[0118] To verify the beneficial effects, this embodiment sets up a control scheme for comparison under the same system and the same data acquisition link. The control scheme adopts the sparse reward DQN training method and the traditional target network hard synchronization method, and sets up rule planning based on scene boundaries as another type of control. The evaluation task covers different sizes of suspended objects and different rope length configurations. The cumulative evaluation task exceeds 400 times, of which the task in the dense obstacle area accounts for about 40%.
[0119] Statistical results show that the success rate of the proposed solution is consistently above 90%, while the sparse reward solution suffers from repeated corrections and failures in densely obstacle-filled areas, resulting in a success rate of around 80%. Rule-based planning shows a significant decrease in success rate when obstacle layout changes locally. Regarding safety, the proposed solution maintains zero or extremely low collision events across all tasks, with significantly fewer minimum safe distance violations than the sparse reward solution, reducing the number of violations by more than half. This demonstrates that the process guidance resulting from the combination of dynamic rewards, target point rewards, and obstacle rewards can continuously suppress risky actions that approach obstacles. In terms of efficiency, the average number of inflection points in the output action set of the proposed solution is significantly lower than that of rule-based planning, and the fluctuation range is significantly narrower than that of the sparse reward solution. The execution time of the action set is reduced by an average of about 10% under the same hoisting distance and similar obstacle density conditions. The online single-step inference latency is stable at the tens of milliseconds level with minimal fluctuation, meeting the closed-loop control requirements for continuous output and execution of action sets. Based on the comparison results of the closed-loop management capabilities of the above system modules and the training and inference data of the algorithm, it can be seen that the present invention can stably output executable control action sequences and form action sets in complex obstacle environments. While reducing the risk of obstacle contact and collision, it improves path smoothness and operation efficiency. It solves the problems of existing technologies, such as the tendency of mechanized single-line planning to get trapped in local optima, insufficient learning signals and unstable training due to sparse rewards, and difficulty in engineering implementation due to the lack of on-site permissions and equipment management closed loop.
[0120] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.
Claims
1. A tower crane path planning method based on edge computing, characterized in that, Includes the following steps: Training data and environmental data for tower crane path planning are acquired at edge computing nodes on the construction site. Construct a state space based on environmental data; An improved deep Q-network RDQN is constructed, and the RDQN is trained using training data and state space to obtain a network model for tower crane path planning. The RDQN includes a main Q-network and a target Q-network. During the training of the network model, a corresponding reward value is generated based on the state space. The reward value is synthesized from dynamic reward, target point reward and obstacle reward, and the parameters of the network model are updated based on the reward value. The network parameters of the main Q network are synchronized to the network parameters of the target Q network, and the network parameters of the target Q network are updated in a weighted manner. Based on the trained network model, online path planning is performed, the current state space is input into the main Q network, and the corresponding tower crane control action is output. The tower crane control actions are converted into tower crane control commands to drive the tower crane to perform operations. After continuously outputting and executing the tower crane control actions, an action set consisting of tower crane control actions is formed. When the target landing point arrival condition is met, the action set is output as the tower crane path planning result.
2. The tower crane path planning method based on edge computing according to claim 1, characterized in that, The acquisition of the training data and environment data specifically includes: Establish training data collection tasks at edge computing nodes on the construction site, collect and aggregate tower crane operation records of tower crane operators during lifting, operation and lowering processes, and organize them into training data; An environmental data acquisition task is established at the edge computing node on the construction site. The radar and camera collect environmental information and obstacle location information near the tower crane to generate environmental perception data, acquire construction site scene information to generate scene data, and acquire tower crane equipment information to generate equipment data. At the construction site, edge computing nodes acquire the latitude and longitude of the tower crane lifting point and the target landing point, as well as the size of the suspended object and the length of the lifting rope. Environmental perception data, scene data, equipment data, the latitude and longitude of the tower crane lifting point and the target landing point, the size of the suspended object and the length of the lifting rope are aggregated and organized to generate environmental data.
3. The tower crane path planning method based on edge computing according to claim 1, characterized in that, The construction of the state space specifically includes: At the edge computing node, the spatial distance from the current position of the hook to the target landing point is calculated based on the latitude and longitude of the tower crane lifting point and the target landing point, to obtain the first distance state quantity. Then, based on the latitude and longitude of the tower crane lifting point and the position information of the obstacle, the spatial distance from the current position of the hook to the position of the obstacle is calculated, to obtain the second distance state quantity. At the edge computing node, the first distance state quantity and the second distance state quantity are used as the state input at the current moment; After each movement of the tower crane, the latitude and longitude of the crane's lifting point and the target landing point, as well as the location information of obstacles, are updated at the edge computing node, and the updated state input after the action is executed is generated to form a state space.
4. The tower crane path planning method based on edge computing according to claim 1, characterized in that, The process of obtaining the network model for tower crane path planning specifically includes: An improved deep Q network RDQN is constructed at the edge computing nodes on the construction site. A master Q network and a target Q network are established in the RDQN. The master Q network is configured with a network structure for receiving the state space, and the target Q network is configured with a network structure for providing target value evaluation results during training. Initial network parameters are generated for the main Q-network and the target Q-network respectively, and written into the corresponding parameter storage area; The training data and the constructed state space are called up. The tower crane operation records corresponding to the training time are read from the training data and aligned with the state space corresponding to the training time to form sample records. At the edge computing node, sample records are written into the experience pool, and training batch samples are randomly selected from it. At the same time, the state space in the sample records is sent into the main Q network to obtain the value evaluation results of the main Q network for the corresponding sample operation actions in the sample records. The value evaluation results are then associated with and stored with the sample records. The state space corresponding to the sample operation action is obtained and fed into the target Q network to obtain the target value evaluation result of the next time step. The target value evaluation result is used as the target basis for updating the parameters of the main Q network to train and iterate the main Q network to obtain the network model of tower crane path planning.
5. The tower crane path planning method based on edge computing according to claim 1, characterized in that, The process of synchronizing the network parameters of the primary Q-network to the network parameters of the target Q-network, and simultaneously updating the network parameters of the target Q-network using a weighted approach, specifically involves: During network model training, the state space corresponding to the current training time is obtained, and the sample operation action corresponding to the current training time is obtained and executed to complete one state transition. The state space of the next time after the state transition is obtained, and the values of the first distance state quantity and the second distance state quantity at the next time are read. Based on the current and next time values of the first and second distance state variables, the target distance change and obstacle distance change are determined. A base reward value, a target distance reward weight, and an obstacle distance reward weight are set. The obstacle distance change and the obstacle distance reward weight are weighted to obtain the obstacle distance contribution item. The target distance change and the target distance reward weight are weighted to obtain the target distance contribution item. The obstacle distance contribution item and the target distance contribution item are superimposed, or the target distance contribution item is subtracted from the obstacle distance contribution item according to a preset method to obtain a dynamic reward. The dynamic reward is then combined with the base reward value to obtain the dynamic reward value. Set a target landing point arrival threshold, compare the current value of the first distance state quantity with the target landing point arrival threshold, and obtain the target point reward; Set the minimum collision distance and obstacle range distance parameters, compare the current value of the second distance state quantity with the minimum collision distance and obstacle range distance, and obtain the obstacle reward; The reward value for this state transition is obtained by combining the dynamic reward value, the target point reward, and the obstacle reward. The network model parameters are updated based on sample records to obtain the state space at the next time step, which is then fed into the target Q-network. The target value evaluation result of the target Q-network is obtained, and the maximum target value evaluation result is selected. A decay factor and learning rate are set, and the reward value is combined with the maximum target value evaluation result according to the decay factor. The error between the reward value and the value evaluation result of the main Q-network is calculated, and a loss is constructed. The loss is backpropagated in the main Q-network, and the network parameters of the main Q-network are updated according to the learning rate using stochastic gradient descent.
6. The tower crane path planning method based on edge computing according to claim 1, characterized in that, The process of synchronizing the network parameters of the primary Q-network to the network parameters of the target Q-network, and simultaneously updating the network parameters of the target Q-network using a weighted approach, specifically involves: During network model training, the current network parameters of the main Q-network are read as input to the main network parameters, and the current network parameters of the target Q-network are read as input to the target network parameters. Weighting coefficients are set at the edge computing nodes, and weighted update processing is performed on the input parameters of the main network and the input parameters of the target network when parameter synchronization is triggered. The input parameters of the main network are weighted according to the weighting coefficients to obtain the weighted parameters of the main network. The input parameters of the target network are weighted according to the complement of the weighting coefficients to obtain the weighted parameters of the target network. The weighted parameters of the main network and the weighted parameters of the target network are added item by item to generate the updated parameters of the target Q network. At the edge computing node, the updated parameters of the target Q network are written into the parameter storage area of the target Q network to replace the original network parameters of the target Q network, thus completing the weighted synchronous update of the network parameters of the main Q network to the network parameters of the target Q network.
7. The tower crane path planning method based on edge computing according to claim 1, characterized in that, The output of the corresponding tower crane control action specifically includes: The edge computing node at the construction site calls the trained network model, constructs a set of candidate actions based on the control methods of the three main mechanisms of the tower crane, and writes them into the action candidate buffer. Online path planning is performed based on the candidate action set. The state space corresponding to the current training time is input into the main Q network, and the main Q network is called to evaluate the value of each candidate action in the candidate action set, so as to obtain the action value evaluation result corresponding to each candidate action. The motion value assessment results are compared, and the candidate motion with the highest corresponding motion value assessment result is selected as the tower crane control motion at the current moment.
8. The tower crane path planning method based on edge computing according to claim 1, characterized in that, The results of the tower crane path planning are obtained specifically in the following ways: At the edge computing node on the construction site, the tower crane control actions are parsed into up and down control quantities, forward and backward control quantities, and rotation control quantities, which are mapped into tower crane rotation mechanism control commands, up and down mechanism control commands, and forward and backward mechanism control commands, respectively, and combined to generate the tower crane control command set at the current moment. After the tower crane executes the crane control command set, it obtains the updated position information of the hook and calculates the updated distance from the hook to the target landing point. The updated distance is then compared with the target landing point arrival threshold to determine whether the target landing point arrival condition is met. If the target landing point is not met, the current tower crane control action is added to the action set. When the target landing point is reached, the current tower crane control actions are added to the action set and output as the tower crane path planning result.