Multi-oil-cylinder synchronous jacking control method and device for building machine based on reinforcement learning

Through the multi-agent control method based on reinforcement learning, the safety hazards caused by asynchronous work of multi-cylinders of the building machine are solved, and the adaptive synchronous control of multi-cylinders in high-rise buildings is realized, which improves construction safety and efficiency.

CN118938683AActive Publication Date: 2024-11-12HUAZHONG UNIV OF SCI & TECH
View PDF 7 Cites 0 Cited by

Patent Information

Application Number
CN202411146560.8
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-20
Publication Date
2024-11-12
Estimated Expiration
2044-08-20

AI Technical Summary

Technical Problem

In high-rise buildings, the asynchronous operation of the multi-cylinder of the building machine leads to the platform level exceeding the limit and the local deformation of the structure, which poses safety risks, and it is difficult for the prior art to realize real-time perception and adaptive adjustment between multiple cylinders.

Method used

Adopting the multi-cylinder synchronous hoisting control method of building machines based on reinforcement learning, establishing a physical environment through hydraulic simulation tools, combining multi-agent reinforcement learning algorithm and coupled synchronization control technology, intelligent controllers are trained to realize adaptive synchronous control of multi-cylinder systems. .

Benefits of technology

Under uneven load conditions, high-precision coordinated perception and active regulation of multiple oil cylinders are achieved, synchronization errors and tracking errors are reduced, stable structure lifting is ensured, and construction safety and efficiency are improved.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118938683A_ABST
    Figure CN118938683A_ABST
Patent Text Reader

Abstract

The invention belongs to the technical field of hydraulic and synchronous control, discloses a building machine multi-oil-cylinder synchronous jacking control method and device based on reinforcement learning, and aims to solve the problem of asynchronous jacking of hydraulic oil cylinders. A coupling synchronous control technology and a multi-agent reinforcement learning algorithm are combined, and multi-oil-cylinder self-adaptive synchronous control in the building machine jacking process is achieved. According to the method, actual jacking stacking data of the building machine is selected, the displacement mean value coupling control technology is adopted, a high-precision controller is trained through a multi-agent reinforcement learning method, and intelligent sensing and cooperative jacking of a multi-oil-cylinder system are achieved. Research results show that the intelligent control algorithm provided by the invention can adaptively adjust the motion condition in the jacking process, reduces the motion error of multiple oil cylinders, ensures stable jacking of the structure, and improves the intelligent construction level of the building machine.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of hydraulic and synchronous control, and in particular relates to a multi-cylinder synchronous lifting control method and device for a building construction machine based on reinforcement learning. Background Art

[0002] With the rapid development of global urbanization in the past two decades, high-rise buildings have been widely praised and used worldwide for their beautiful design, comfortable environment, and land saving. However, high-rise building construction has a series of problems such as high difficulty, high risk, and complex organization. Traditional climbing formwork construction has operational risks such as formwork collapse, falling objects from high altitude, and collision with objects. Therefore, there is an urgent need for advanced construction equipment to improve the on-site working environment and improve construction efficiency. In this context, the aerial building machine (construction equipment integration platform, hereinafter referred to as the "building machine") has been rapidly put into the construction of many landmark high-rise buildings in China due to its high equipment integration, factory-like operation, fast construction speed, and guaranteed construction safety. It has greatly improved the mechanization and intelligent level of high-rise building construction, and promoted my country's high-rise building construction technology to a new height.

[0003] In the complex environment of 300 to 500 meters above the ground, the building construction machine provides a safe, comfortable and stable construction platform for construction workers. As the building construction machine continues to rise, a vertical engineering assembly line is formed inside the building construction machine, and construction workers can complete tasks such as steel bar binding, formwork erection and concrete pouring, which reduces labor costs and significantly improves construction efficiency. During the lifting process, the building construction machine controls multiple sets of hydraulic cylinders to achieve step-by-step lifting. However, due to uneven loading of the steel platform and differences between the cylinders, asynchronous operation may occur between multiple cylinders, which may cause the platform level to exceed the limit and local deformation of the structure, posing safety hazards to high-rise building construction. In order to improve the safety of high-rise building construction and reduce the probability of accidents, it is necessary to carry out intelligent and synchronous control of the lifting process of the building construction machine. In the context of digitalization of construction equipment, how to use advanced intelligent algorithms to achieve real-time perception and adaptive adjustment between multiple cylinders is a key issue.

[0004] Through the above analysis, the problems and defects of the prior art are as follows:

[0005] Due to uneven loading of the steel platform and differences among the cylinders, asynchronous operation may occur between multiple cylinders, which may cause the platform to exceed the level limit and local deformation of the structure, posing a safety hazard to the construction of high-rise buildings. Summary of the invention

[0006] In view of the problems existing in the prior art, the present invention provides a multi-cylinder synchronous jacking control method and device for a building construction machine based on reinforcement learning.

[0007] The present invention is implemented in this way: a multi-cylinder synchronous lifting control method for a building construction machine based on reinforcement learning includes:

[0008] Step 1: Create a multi-cylinder physical environment;

[0009] Step 2: Multi-agent reinforcement learning modeling;

[0010] Step 3: Intelligent control algorithm debugging.

[0011] Furthermore, the multi-cylinder physical environment is created:

[0012] First, the hydraulic model of the building construction machine was established using the hydraulic simulation tool AMESim. The hydraulic system was simplified to some extent, retaining the core components such as the oil tank, motor, plunger pump, proportional directional valve, pressure relief valve and piston. A vertical load signal was applied to the upper part of the piston to simulate the stacking load on the platform.

[0013] The reference model is defined as the average value of the output displacement of each power point. The trained intelligent agent is used as the controller to receive the cylinder stroke error feedback and adjust the cylinder control valve to maintain multi-cylinder synchronization.

[0014]

[0015]

[0016]

[0017] where x i (t) is the stroke of the i-th cylinder at time t, n is the total number of cylinders in the building construction machine, a(t) is the mean stroke of all cylinders at time t, Represents the input signal, and They represent the tracking error and synchronization error during the lifting process.

[0018] To prove the convergence of the synchronization error, formula (3) can be expressed as follows:

[0019]

[0020] Through the initial transformation, it can be seen that the coefficient matrix A is a full rank matrix, as shown below:

[0021]

[0022] From the above, we can see that when When x i (t) has a unique solution, indicating that the synchronization error can converge to zero under this control loop.

[0023] Furthermore, the multi-agent reinforcement learning modeling:

[0024] The independent learning idea is integrated into SAC, and an independent SAC algorithm is proposed; ISAC is a completely decentralized algorithm that assumes that each agent learns independently in its own environment without considering changes in other agents.

[0025] Further, the intelligent control algorithm is debugged:

[0026] The state of the agent is as follows:

[0027]

[0028] Among them, x i (t) represents the stroke of the i-th cylinder at time t; It represents the difference between the i-th cylinder's stroke and the target stroke at time t, i.e., the tracking error; The difference between the stroke of the i-th cylinder and the mean stroke of all cylinders at time t is the synchronization error;

[0029] The action of the agent is set as the control voltage of the proportional reversing valve. The maximum voltage to control the piston to lift is -40mA, and the voltage to close the valve is 0mA. The action range of agent 1 is set to [-40,0];

[0030] The smaller the error during the lifting process, the greater the reward value given; the reward function and termination condition studied are shown in equations (7) and (8);

[0031]

[0032]

[0033] Among them, ω1 and ω2 represent the weights of the two errors in the reward function; γ represents the additional reward value; δ represents the penalty value; T r and T s represent the simulation time and the stop time respectively; α and β represent the thresholds of the tracking error and the synchronization error respectively.

[0034] Another object of the present invention is to provide a multi-cylinder synchronous lifting control device for a building construction machine based on reinforcement learning, comprising:

[0035] Creation module for creating multi-cylinder physical environment;

[0036] Modeling module, used for multi-agent reinforcement learning modeling;

[0037] Debug module, used for debugging through intelligent control algorithms.

[0038] Another object of the present invention is to provide a computer device, which includes a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the reinforcement learning-based multi-cylinder synchronous jacking control method for a building construction machine.

[0039] Another object of the present invention is to provide a computer-readable storage medium storing a computer program, which, when executed by a processor, enables the processor to execute the steps of the reinforcement learning-based multi-cylinder synchronous jacking control method for a building construction machine.

[0040] Another object of the present invention is to provide an information data processing terminal, which is used to implement the reinforcement learning-based multi-cylinder synchronous jacking control device for the building construction machine.

[0041] In combination with the above technical solutions and the technical problems solved, the advantages and positive effects of the technical solutions to be protected by the present invention are as follows:

[0042] First, the present invention proposes a reinforcement learning-based multi-cylinder synchronous lifting control method and device for a building construction machine. Aiming at the problem of asynchronous lifting of hydraulic cylinders, the present invention combines the coupling synchronous control technology with the multi-agent reinforcement learning algorithm to realize the adaptive synchronous control of multiple cylinders in the lifting process of the building construction machine.

[0043] The present invention selects the actual loading data of the building construction machine, adopts the displacement mean coupling control technology, and trains the high-precision controller with the multi-agent reinforcement learning method to realize the intelligent perception and coordinated lifting of the multi-cylinder system. Taking the construction of a 358-meter high-rise building as an example, the intelligent control algorithm proposed in the present invention obtains the maximum synchronization error of the multi-cylinder joint lifting of 700mm through simulation under the condition of uneven distribution of steel platform loading, which is only 1.060mm, less than the 5mm safety threshold, and the maximum steady-state error after the lifting is only 0.403mm, which is less than the target stroke of 700mm×2%=14mm, meeting the actual use requirements. The research results show that the intelligent control algorithm proposed in the present invention can adaptively adjust the movement during the lifting process, reduce the multi-cylinder movement error, ensure the stable lifting of the structure, and improve the intelligent construction level of the building construction machine.

[0044] Second, the technical solution of the present invention fills the technical gap in the industry at home and abroad:

[0045] Traditional multi-cylinder control methods require setting corresponding control parameters and lack versatility in different external environments. In order to improve the intelligent level of building construction machinery, the present invention combines multi-agent reinforcement learning algorithm with coupled synchronous control technology, and proposes a multi-cylinder synchronous lifting control method and device for building construction machinery based on reinforcement learning, which realizes the collaborative perception and active regulation of multiple cylinders under uncertain conditions of cylinder load, fills the research gap of intelligent synchronous lifting of building construction machinery at home and abroad, and promotes the application and development of multi-agent reinforcement learning in multi-cylinder synchronous control scenarios.

[0046] The technical solution of the present invention solves the technical problems that people have been eager to solve but have never been able to solve successfully:

[0047] When the building machine completes the construction of a standard layer, the construction platform will be lifted upward through the hydraulic system to start the construction of the next standard layer. However, due to the uneven loads such as steel bars, wood, and fabric machines stacked on the construction platform, the multiple groups of hydraulic cylinders at the bottom of the platform are unevenly stressed during lifting, and asynchronous lifting is prone to occur, resulting in deformation of the construction platform, and even the danger of the platform overturning in severe cases. Therefore, lifting work is generally considered to be a key stage in the construction process of the building machine. In order to solve the problem of asynchronous lifting of the building machine, improve the safety of high-rise building construction, and improve the intelligent construction efficiency of the building machine, the present invention uses an advanced multi-agent reinforcement learning algorithm as the controller of the synchronous control system, uses coupled synchronous control technology to form a multi-cylinder perception and control loop, and by establishing a high-fidelity hydraulic system physical environment, trains the collaborative perception and active regulation performance of the intelligent body controller under load uncertainty conditions, realizes the intelligent one-button lifting function of the building machine, avoids the shortcomings of the traditional method that requires manual regulation, solves the asynchronous working problem of the building machine cylinder under uneven load, and provides technical support and solid guarantee for promoting my country's building machine technology to the world.

[0048] Third, the technical solution of the present invention aims at the problem of insufficient precision and stability of the multi-cylinder synchronous jacking control of building construction machines in the prior art, and proposes a multi-cylinder synchronous jacking control method based on reinforcement learning. By introducing an independent SAC algorithm, the learning and control of multiple agents in an independent environment are realized, which effectively reduces the synchronization error and tracking error of the system during the jacking process. The hydraulic simulation model established by AMESim is used to dynamically adjust the voltage of the cylinder control valve through the intelligent control algorithm, so that the system can adapt to the changing load conditions in real time and ensure the accuracy of synchronous jacking.

[0049] Compared with traditional control methods, the present invention has achieved significant technical progress in many aspects: first, through the application of reinforcement learning algorithm, the system has the ability of adaptive adjustment and can maintain high-precision synchronous control in complex environments; second, the design of intelligent control algorithm significantly improves the response speed and stability of the system and reduces the error in the jacking process; finally, the synchronous control method of the present invention has proved its convergence and reliability through verification, which provides higher safety and efficiency for the multi-cylinder jacking system in practical applications.

[0050] Fourth, the mathematical model of the present invention solves the problem of the difficulty in achieving the accuracy and stability of multi-cylinder synchronous control in the prior art by accurately describing the tracking error and synchronization error in the multi-cylinder synchronous jacking system. By establishing the mathematical model of the tracking error and synchronization error, the present invention can accurately calculate the stroke difference of each cylinder during the jacking process, and optimize the control strategy through an independent SAC (Soft Actor-Critic) algorithm.

[0051] Specifically, the synchronization error formula defined in the mathematical model has been proven to be convergent through matrix operations, ensuring that the error can gradually approach zero under the control loop, thereby achieving high-precision synchronization control of multiple cylinders. This mathematical model not only provides a theoretical basis for the intelligent control algorithm, but also enables the system to work stably under complex load conditions.

[0052] Compared with the traditional control method, the present invention has made significant progress in synchronization control accuracy, response speed and system stability through the establishment and verification of mathematical models. The introduction of mathematical models enables the system to better handle complex dynamic relationships, achieve higher control accuracy and reliability, and provide a new solution for the multi-cylinder synchronous jacking control method. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 It is a flow chart of a multi-cylinder synchronous lifting control method for a building construction machine based on reinforcement learning provided by an embodiment of the present invention.

[0054] Figure 2 It is a structural block diagram of a multi-cylinder synchronous lifting control device for a building construction machine based on reinforcement learning provided by an embodiment of the present invention.

[0055] Figure 3 It is a composition diagram of the hydraulic system of a building construction machine provided in an embodiment of the present invention.

[0056] Figure 4 It is a displacement mean coupling control loop diagram provided by an embodiment of the present invention.

[0057] Figure 5 This is a reward value distribution area diagram provided by an embodiment of the present invention.

[0058] Figure 6 Schematic diagram of the hydraulic system of a building construction machine provided by an embodiment of the present invention. (a) The location distribution of the hydraulic system; (b) The load weight of the hydraulic system (unit: tons)

[0059] Figure 7 1 is a time history curve diagram of the lifting displacement of multiple groups of oil cylinders provided by an embodiment of the present invention. (a) Equivalent control; (b) Intelligent control.

[0060] Figure 8 It is a maximum synchronization error diagram of the test case provided by the embodiment of the present invention.

[0061] Fig. 9 It is a maximum steady-state error diagram of the test case provided by the embodiment of the present invention. DETAILED DESCRIPTION

[0062] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0063] like Figure 1 As shown, a reinforcement learning-based multi-cylinder synchronous lifting control method for a building construction machine provided by an embodiment of the present invention includes the following steps:

[0064] S101, creation of multi-cylinder physical environment;

[0065] First, the hydraulic model of the building construction machine was established using the hydraulic simulation tool AMESim. The hydraulic system was simplified to some extent, retaining the core components such as the oil tank, motor, plunger pump, proportional directional valve, pressure relief valve and piston. A vertical load signal was applied to the upper part of the piston to simulate the stacking load on the platform. Figure 3 The figure shows a single set of hydraulics for a building construction machine. To get closer to the actual situation on site, the model also considers the mechanical efficiency and volumetric efficiency of the hydraulic pump. The hydraulic pump model of the building construction machine is 25SCY, and the mechanical efficiency is assumed to be 95%. According to the technical manual, the volumetric efficiency is 93.5%.

[0066] On this basis, in order to realize the synchronous lifting control of multiple cylinders of the building construction machine, a displacement mean coupling control loop is established in Simulink, such as Figure 4 As shown; the displacement mean coupling control method is to compare the hydraulic output in each sub-circuit with the reference model one by one; the present invention defines the reference model as the average value of the output displacement of each power point, uses a trained intelligent agent as a controller, receives the cylinder stroke error feedback, and adjusts the control valve of the cylinder to maintain multi-cylinder synchronization;

[0067]

[0068]

[0069]

[0070] where x i (t) is the stroke of the i-th cylinder at time t, n is the total number of cylinders in the building construction machine, a(t) is the mean stroke of all cylinders at time t, Represents the input signal, and They represent the tracking error and synchronization error during the lifting process.

[0071] In order to prove the convergence of the synchronization error, formula (3) can be expressed as follows:

[0072]

[0073] Through the initial transformation, it can be seen that the coefficient matrix A is a full rank matrix, as shown below:

[0074]

[0075] From the above, we can see that when When x i (t) There is a unique solution, indicating that the synchronization error can converge to zero under this control loop;

[0076] S102, multi-agent reinforcement learning modeling;

[0077] The SAC (Soft Actor-Critic) algorithm is a deep reinforcement learning algorithm based on the maximum entropy theory. It has the following three advantages: (1) It strengthens the exploration of the environment by introducing entropy in the learning objective, making it easier to find better strategies under multimodal rewards; (2) It has better robustness and higher sampling efficiency; (3) It provides a stable learning framework for control tasks in continuous action space; The most important feature of the SAC algorithm is the use of information entropy, which is an indicator of the randomness of the strategy; the larger the entropy, the higher the randomness of the strategy; high randomness is obtained through more exploration, which allows the algorithm to learn more information to calculate the optimal solution; therefore, the algorithm has more output possibilities when facing disturbances, ensuring the robustness of the algorithm; The SAC algorithm strikes a balance between maximizing expected returns and maximizing entropy, making the algorithm robust while ensuring strategy optimization;

[0078] However, the traditional SAC algorithm is mainly suitable for single-agent reinforcement learning. The lifting process of the building construction machine includes the complex coupling and coordinated work of multiple hydraulic cylinders. The traditional SAC algorithm cannot be used for the adaptive control of the lifting of the building construction machine. In order to solve this problem, the present invention integrates the independent learning idea into SAC and proposes an independent SAC (ISAC) algorithm. ISAC is a completely decentralized algorithm, which assumes that each agent learns independently in its own environment without considering the changes of other agents. Specifically, parameter sharing techniques are used in the agent training process, that is, the same set of strategy parameters are used for all agents. This method makes the model training data more and more stable. The premise of using this technique is that the agents are completely homogeneous, that is, their state space and action space are completely consistent, and their optimization goals are also completely consistent. The pseudo code of ISAC is shown in Algorithm 1.

[0079]

[0080] S103, intelligent control algorithm debugging;

[0081] In order to improve the performance of the agent and enable it to fully perceive the environment, it is crucial to have an explicit observation of the state; in this technique, the agent is used as a controller in hydraulics, and the state of the agent is as follows:

[0082]

[0083] Among them, x i (t) represents the stroke of the i-th cylinder at time t; It represents the difference between the i-th cylinder's stroke and the target stroke at time t, i.e., the tracking error; The difference between the stroke of the i-th cylinder and the mean stroke of all cylinders at time t is the synchronization error;

[0084] The action of the agent is set as the control voltage of the proportional reversing valve. The maximum voltage for controlling the piston to lift is -40mA, and the closing voltage is 0mA. Therefore, the action range of agent 1 is set to [-40,0]. In order to judge the performance of the action, the critic network needs to be trained based on the reward value of the action and the terminal state of the simulation. Therefore, the reward function plays a key role in guiding the behavior of the agent and guiding the learning process to achieve the predefined goal. For the lifting process of the building construction machine, multiple cylinders need to take into account the tracking error and the synchronization error to achieve collaborative work. Therefore, the overall design of the reward function must include two errors. In general, the smaller the error during the lifting process, the greater the reward value given. The reward function and termination condition studied are shown in equations (7) and (8).

[0085]

[0086]

[0087] Among them, ω1 and ω2 represent the weights of the two errors in the reward function; γ represents the additional reward value; δ represents the penalty value; T r and T s Represent the simulation time and stop time respectively; α and β represent the thresholds of tracking error and synchronization error respectively;

[0088] Figure 5 The reward value distribution area of ​​the cylinder lifting stroke curve is shown; Figure 5 As shown in Figure 1, the lifting process of the building machine can be divided into two stages: the synchronous working stage and the synchronous stopping stage. In area I, the task of the agent is to lift to the target displacement while reducing the synchronization error with other cylinder pistons. The reward in this stage comes from the tracking error. and synchronization error decrease; in region II, the agent enters the error band of the target displacement and then gradually stops lifting; in order to encourage the agent to stay in this region as long as possible, an additional constant reward term γ is added to the reward function; region III is a penalty region, which means that if the agent exceeds the error band, it will immediately receive a penalty value; in addition, in the simulation process of the agent interacting with the physical environment, the termination condition is defined as: when the simulation time T r >T s , it means that the agent has successfully completed the task and the simulation ends; if the tracking error is or synchronization error If the threshold α or β is exceeded, the simulation will stop immediately. After the training is completed, the corresponding cylinder load is set according to the actual jacking situation of the building construction machine to test the synchronous jacking effect of the intelligent control algorithm.

[0089] like Figure 2 As shown, an embodiment of the present invention provides a multi-cylinder synchronous lifting control device for a building construction machine based on reinforcement learning, comprising:

[0090] Creation module for creating multi-cylinder physical environment;

[0091] Modeling module, used for multi-agent reinforcement learning modeling;

[0092] Debug module, used for debugging through intelligent control algorithms.

[0093] Another object of the present invention is to provide a computer device, which includes a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the reinforcement learning-based multi-cylinder synchronous jacking control method for a building construction machine.

[0094] Another object of the present invention is to provide a computer-readable storage medium storing a computer program, which, when executed by a processor, enables the processor to execute the steps of the reinforcement learning-based multi-cylinder synchronous jacking control method for a building construction machine.

[0095] Another object of the present invention is to provide an information data processing terminal, which is used to implement the reinforcement learning-based multi-cylinder synchronous jacking control device for the building construction machine.

[0096] This implementation case provides a method and device for controlling the synchronous lifting of multiple cylinders of a building construction machine based on reinforcement learning. The case is the application of a building construction machine in the construction of a 358m high-rise building. It mainly includes the following steps:

[0097] Step 1, modeling of the hydraulic system and control loop. The present invention uses AMESim to establish a single hydraulic system for a building construction machine, including a fuel tank, a motor, a plunger pump, a proportional directional valve, a pressure relief valve and a piston. The maximum load capacity of the hydraulic cylinder is 150 tons, the maximum flow is 35L / min, and more modeling parameters are shown in Table 1. The present invention establishes a displacement mean coupling control loop in Simulink, that is, the output of the hydraulic system in each sub-loop is compared with the reference model one by one. The reference model is defined as the average value of the output displacement of each group of cylinders. The controller receives the cylinder stroke error feedback and adjusts the control valve of the cylinder to keep the multi-cylinders synchronized. Use the multi-agent reinforcement learning algorithm in Python to train a high-precision controller, see step 2.

[0098] Table 1. Hydraulic system modeling parameters

[0099]

[0100] Step 2: Multi-agent reinforcement learning modeling. Figure 6As shown in the figure, based on the measured loading data of a certain lifting of the building construction machine, it can be seen that the upper loading distribution of the 10 sets of hydraulic cylinders of the building construction machine is uneven. Therefore, the upper loading is used as a variable parameter, and the range of loading weight is set to [50t, 120t]. Before each training, the upper loading weight is randomly selected for the hydraulic cylinder to enhance the robustness of the algorithm. In the multi-agent training process, the Critic network is a 3-layer neural network composed of 512 neurons, and the Actor network is a 2-layer neural network composed of 512 neurons. The learning rates of the Critic network and the Actor network are set to 0.0001 and 0.0005 respectively. The discount coefficient is set to 0.98, and the soft update coefficient τ is 0.005. During model training, the states received by the agent are normalized to increase the versatility of the agent. Considering the computational cost and time, the training condition is set to 2 sets of hydraulic cylinders lifting 100mm at the same time, the action time interval is 0.2s, and the single training time is 20s. In terms of the reward function, the weight coefficients ω1 and ω2 are set to 0.9 and 0.1, the additional reward value γ is 5; the penalty value δ is -10, the thresholds α and β of the tracking error and the synchronization error are set to 2% and 1mm exceeding the target stroke respectively; a settlement reward value of 20 is given for successfully completing the task.

[0101] Step 3: Intelligent control algorithm debugging. In the test environment, set the following settings for 10 groups of cylinders: Figure 6 The load weight is shown in the figure, and the agent trained in step 2 is used as the synchronous lifting controller. The test conditions are set as follows: 10 groups of hydraulic cylinders lift 700mm at the same time, the action time interval is 0.2s, and the test time is 150s. The test results are shown in Figure 7 shown. Figure 7 (a) is the displacement curve of the cylinder under traditional equivalent control. Each cylinder is controlled individually, and the maximum synchronization error of 700mm lifting is 22.1mm. The reason is that the coordinated work between the cylinders is not considered. Figure 7(b) is the displacement curve of the oil cylinder under the intelligent control algorithm of the present invention. It can be seen that the displacement curves of the 10 groups of oil cylinders maintain good synchronization, and the jacking displacement curve presents an "S" shape, indicating that the adaptive jacking process can be roughly divided into three stages: in the first stage, all oil cylinders sense each other, adjust their own speeds, and ensure coordinated work, so the displacement curve is relatively gentle; in the second stage, all oil cylinders begin to jack up normally, and the slope of the displacement curve becomes larger; in the third stage, after approaching the target stroke, the 10 groups of oil cylinders gradually slow down until they reach the target stroke. During the whole process, the maximum synchronization error of multiple oil cylinders jacking 700mm together is only 1.060mm, which is less than the 5mm safety threshold. The maximum steady-state error after the jacking is 0.403mm, which is less than 700mm×2%=14mm, meeting the basic error requirements. The research results show that the intelligent control algorithm proposed in the present invention can adaptively adjust the movement of the oil cylinders during the jacking process, reduce the movement error of multiple oil cylinders, ensure the stable jacking of the structure, and improve the dynamic safety of the building construction machine.

[0102] In order to further verify the robustness of the control method of the present invention, the Latin hypercube sampling method is used to sample the loads of 10 groups of cylinders. The sampling range is [50t, 120t], and 20 cases are generated. The control method of the present invention is used to perform synchronous jacking control, and the maximum synchronization error and steady-state error of multi-cylinder jacking are observed. Figure 8 The maximum synchronization error of the control method of the present invention in 20 cases is shown. As can be seen from the figure, under different external loads, the maximum synchronization between multiple cylinders maintains a similar change pattern, indicating that the control method of the present invention can adaptively adjust the multi-cylinder movement under uneven loads. In addition, the maximum synchronization error of all cases is about 1.2mm, which meets the safety requirements. Fig. 9 The maximum steady-state error of the control method of the present invention in 20 cases is shown, of which the maximum steady-state error is 3.86 mm, which is less than 700 mm × 2% = 14 mm, indicating that the multi-cylinder meets the basic error requirements in all cases. In summary, the control method of the present invention can be applied to the scene of uneven load distribution on the steel platform, and has a significant control effect on the coordinated work of multiple cylinders, with good generalization and robustness.

[0103] It should be noted that the embodiments of the present invention can be implemented by hardware, software, or a combination of software and hardware. The hardware part can be implemented using dedicated logic; the software part can be stored in a memory and executed by appropriate instructions, such as a microprocessor or dedicated design hardware. It can be understood by a person of ordinary skill in the art that the above-mentioned devices and methods can be implemented using computer executable instructions and / or contained in a processor control code, such as a carrier medium such as a disk, CD or DVD-ROM, a programmable memory such as a read-only memory (firmware), or a data carrier such as an optical or electronic signal carrier. Such code is provided on the carrier medium. The device and its modules of the present invention can be implemented by hardware circuits such as very large-scale integrated circuits or gate arrays, semiconductors such as logic chips, transistors, etc., or programmable hardware devices such as field programmable gate arrays, programmable logic devices, etc., can also be implemented by software executed by various types of processors, and can also be implemented by a combination of the above-mentioned hardware circuits and software, such as firmware.

[0104] The above description is only a specific implementation mode of the present invention, but the protection scope of the present invention is not limited thereto. Any modifications, equivalent substitutions and improvements made by any technician familiar with the technical field within the technical scope disclosed by the present invention and within the spirit and principle of the present invention should be covered by the protection scope of the present invention.

Claims

1. A multi-cylinder synchronous lifting control method for a building construction machine based on reinforcement learning, characterized in that: The multi-cylinder physical environment model was established through the hydraulic simulation tool AMESim, and an independent SAC algorithm was introduced for multi-agent reinforcement learning modeling. After real-time monitoring of the difference between the cylinder stroke and the target stroke and the difference in the mean stroke between the cylinders, the agent dynamically adjusted the voltage of the control valve to ensure the accuracy and stability of the synchronous lifting of the multiple cylinders. By designing a specific reward function and termination conditions, the system can effectively converge the synchronization error and ensure the reliability and safety of the lifting process.

2. The reinforcement learning-based multi-cylinder synchronous lifting control method for a building construction machine as claimed in claim 1, characterized in that: The reinforcement learning-based multi-cylinder synchronous lifting control method for a building construction machine comprises the following steps: Step 1: Create a multi-cylinder physical environment; Step 2: Multi-agent reinforcement learning modeling; Step 3: Intelligent control algorithm debugging.

3. The reinforcement learning-based multi-cylinder synchronous lifting control method for a building construction machine as claimed in claim 2, characterized in that: The multi-cylinder physical environment is created by: First, the hydraulic model of the building construction machine was established using the hydraulic simulation tool AMESim. The hydraulic system was simplified to some extent, retaining the core components such as the oil tank, motor, plunger pump, proportional directional valve, pressure relief valve and piston. A vertical load signal was applied to the upper part of the piston to simulate the stacking load on the platform. The reference model is defined as the average value of the output displacement of each power point. The trained intelligent agent is used as the controller to receive the cylinder stroke error feedback and adjust the cylinder control valve to maintain multi-cylinder synchronization. where x i (t) is the stroke of the i-th cylinder at time t, n is the total number of cylinders in the building construction machine, a(t) is the mean stroke of all cylinders at time t, Represents the input signal, and They represent the tracking error and synchronization error during the lifting process. To prove the convergence of the synchronization error, formula (3) can be expressed as follows: Through the initial transformation, it can be seen that the coefficient matrix A is a full rank matrix, as shown below: From the above, we can see that when When x i (t) has a unique solution, indicating that the synchronization error can converge to zero under this control loop.

4. The reinforcement learning-based multi-cylinder synchronous lifting control method for a building construction machine as claimed in claim 2, characterized in that: The multi-agent reinforcement learning modeling: The independent learning idea is integrated into SAC, and an independent SAC algorithm is proposed; ISAC is a completely decentralized algorithm that assumes that each agent learns independently in its own environment without considering changes in other agents.

5. The reinforcement learning-based multi-cylinder synchronous lifting control method for a building construction machine as claimed in claim 2, characterized in that: The intelligent control algorithm debugging: The state of the agent is as follows: Among them, x i (t) represents the stroke of the i-th cylinder at time t; It represents the difference between the i-th cylinder's stroke and the target stroke at time t, i.e., the tracking error; The difference between the stroke of the i-th cylinder and the mean stroke of all cylinders at time t is the synchronization error; The action of the agent is set as the control voltage of the proportional reversing valve. The maximum voltage to control the piston to lift is -40mA, and the voltage to close the valve is 0mA. The action range of agent 1 is set to [-40,0]; The smaller the error during the lifting process, the greater the reward value given; the reward function and termination condition studied are shown in equations (7) and (8); Among them, ω1 and ω2 represent the weights of the two errors in the reward function; γ represents the additional reward value; δ represents the penalty value; T r and T s represent the simulation time and the stop time respectively; α and β represent the thresholds of the tracking error and the synchronization error respectively.

6. A reinforcement learning-based multi-cylinder synchronous lifting control device for a building construction machine that implements the reinforcement learning-based multi-cylinder synchronous lifting control method for a building construction machine as described in any one of claims 1 to 5, characterized in that: The multi-cylinder synchronous lifting control device for a building construction machine based on reinforcement learning comprises: Creation module for creating multi-cylinder physical environment; Modeling module, used for multi-agent reinforcement learning modeling; Debug module, used for debugging through intelligent control algorithms.

7. A computer device, characterized in that: The computer device includes a memory and a processor, the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the reinforcement learning-based multi-cylinder synchronous jacking control method for a building construction machine as described in any one of claims 1-5.

8. A computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor, the processor executes the steps of the reinforcement learning-based multi-cylinder synchronous jacking control method for a building construction machine as described in any one of claims 1 to 4.

9. An information data processing terminal, characterized in that: The information data processing terminal is used to implement the multi-cylinder synchronous jacking control device for a building construction machine based on reinforcement learning as described in claim 6.

Citation Information

Patent Citations

  • Self-adaptive control method and system for posture of shield tunneling machine under shield construction condition

    CN116220713A

  • Synchronous jacking control method and jacking control method and system based on synchronous jacking control method

    CN117401609A

  • Displacement output self-adaptive jacking hydraulic system and working method

    CN117865001A

  • Multi-stage coordinated synchronous hydraulic oil cylinder and intelligent control method thereof

    CN118482059A

  • Multi-point synchronization jacking device

    CN202208972U