Reinforcement learning-based building machine multi-oil cylinder synchronous jacking control method and device

Abstract

The application belongs to the technical field of hydraulic and synchronization control, discloses a kind of based on reinforcement learning's building machine multiple oil cylinder synchronous jacking control method and device, the application proposes a kind of based on reinforcement learning's building machine multiple oil cylinder synchronous jacking control method and device, for hydraulic oil cylinder asynchronous jacking problem, coupling synchronization control technology and multi-agent reinforcement learning algorithm are combined, the adaptive synchronous control of multiple oil cylinder in building machine jacking process is realized.The application selects building machine jacking actual load data, adopts displacement mean coupling control technology, trains high-precision controller using multi-agent reinforcement learning method, realizes the intelligent perception and collaborative jacking of multiple oil cylinder system.The research results show that the intelligent control algorithm proposed in the application can adaptively adjust the motion in the jacking process, reduce the motion error of multiple oil cylinders, ensure the stable jacking of the structure, and improve the intelligent construction level of building machine.

Description

Technical Field

[0001] This invention belongs to the field of hydraulic and synchronous control technology, and particularly relates to a multi-cylinder synchronous lifting control method and device for building construction machines based on reinforcement learning. Background Technology

[0002] With the rapid urbanization of the past two decades, high-rise buildings have gained widespread popularity and use worldwide due to their aesthetic design, comfortable environment, and land-saving advantages. However, high-rise building construction presents a series of challenges, including high difficulty, high risk, and complex organization. Traditional climbing formwork construction carries operational risks such as formwork collapse, falling objects from heights, and collisions with objects. Therefore, there is an urgent need for advanced construction equipment to improve the on-site working environment and enhance construction efficiency. Against this backdrop, the aerial building machine (construction equipment integration platform, hereinafter referred to as "building machine") has been rapidly deployed in the construction of numerous landmark high-rise buildings in China due to its high degree of equipment integration, factory-like operation, fast construction speed, and guaranteed construction safety. This has greatly improved the mechanization and intelligence level of high-rise building construction, propelling my country's high-rise building construction technology to new heights.

[0003] In the complex environment of heights of 300 to 500 meters, high-altitude construction rigs provide construction workers with a safe, comfortable, and stable working platform. As the rig continuously lifts, a vertical engineering assembly line is formed inside, allowing workers to complete tasks such as rebar tying, formwork erection, and concrete pouring, reducing labor costs and significantly improving construction efficiency. During the lifting process, the rig controls multiple sets of hydraulic cylinders to achieve step-by-step lifting. However, due to uneven loading of the steel platform and differences between the cylinders, asynchronous operation may occur between the cylinders, potentially leading to excessive platform levelness and localized structural deformation, posing safety hazards for high-rise building construction. To improve the safety of high-rise building construction and reduce the probability of accidents, it is necessary to implement intelligent synchronous control of the rig's lifting process. In the context of the digitalization of construction equipment, how to use advanced intelligent algorithms to achieve real-time perception and adaptive adjustment between multiple cylinders is a crucial issue.

[0004] Based on the above analysis, the problems and shortcomings of the existing technology are as follows:

[0005] Due to uneven loading of the steel platform and differences among the hydraulic cylinders, asynchronous operation may occur between multiple hydraulic cylinders, which may lead to excessive leveling of the platform and local deformation of the structure, posing safety hazards to the construction of high-rise buildings. Summary of the Invention

[0006] To address the problems existing in the prior art, this invention provides a method and device for synchronous lifting control of multiple hydraulic cylinders in a building construction machine based on reinforcement learning.

[0007] This invention is implemented as follows: A multi-cylinder synchronous lifting control method for a building construction machine based on reinforcement learning includes:

[0008] Step 1: Creating the physical environment for multiple hydraulic cylinders;

[0009] Step 2, multi-agent reinforcement learning modeling;

[0010] Step 3: Debugging the intelligent control algorithm.

[0011] Furthermore, the physical environment for the multi-cylinder system is created as follows:

[0012] First, a hydraulic model of the building machine was built using the hydraulic simulation tool AMESim. The hydraulic system was simplified to retain core components such as the oil tank, motor, piston pump, proportional directional valve, pressure relief valve, and piston. A vertical load signal was applied to the upper part of the piston to simulate the stacking load on the platform.

[0013] The reference model is defined as the average value of the output displacement of each power point. A trained intelligent agent is used as the controller to receive feedback on the cylinder stroke error and adjust the control valve of the cylinder to maintain synchronization of multiple cylinders.

[0014]

[0015]

[0016]

[0017] Where x i a(t) represents the stroke of the i-th cylinder at time t, n represents the total number of cylinders in the building machine, and a(t) represents the average stroke of all cylinders at time t. Indicates the input signal, and These represent the tracking error and synchronization error during the lifting process, respectively.

[0018] To prove the convergence of the synchronization error, formula (3) can be expressed as follows:

[0019]

[0020] Through the initial transformation, we know that the coefficient matrix A is a full-rank matrix, as shown below:

[0021]

[0022] From the above, it can be seen that when At that time, x i The fact that (t) has a unique solution indicates that the synchronization error of the control loop can converge to zero.

[0023] Furthermore, the multi-agent reinforcement learning modeling:

[0024] By incorporating the concept of independent learning into SAC, the Independent SAC algorithm is proposed. ISAC is a completely decentralized algorithm that assumes that each agent learns independently in its own environment without considering changes made by other agents.

[0025] Furthermore, the intelligent control algorithm is debugged:

[0026] The agent's state is as follows:

[0027]

[0028] Where, x i (t) represents the stroke of the i-th cylinder at time t; This represents the difference between the i-th cylinder's stroke and the target stroke at time t, i.e., the tracking error; The difference between the stroke of the i-th cylinder at time t and the average stroke of all cylinders, i.e., the synchronization error;

[0029] The action of the agent is set to the control voltage of the proportional directional valve. The maximum voltage for controlling the piston to rise is -40mA, and the voltage for closing the valve is 0mA. The action range of agent 1 is set to [-40,0].

[0030] The smaller the error during the jacking process, the greater the reward value is given; the reward function and termination condition studied are shown in equations (7) and (8);

[0031]

[0032]

[0033] Where ω1 and ω2 represent the weights of the two errors in the reward function; γ represents the additional reward; δ represents the penalty; T r and T s These represent the simulation time and the stopping time, respectively; α and β represent the threshold values ​​for tracking error and synchronization error, respectively.

[0034] Another objective of this invention is to provide a multi-cylinder synchronous lifting control device for a building construction machine based on reinforcement learning, comprising:

[0035] Create a module for creating a physical environment for multiple hydraulic cylinders;

[0036] The modeling module is used for multi-agent reinforcement learning modeling.

[0037] The debugging module is used for debugging via intelligent control algorithms.

[0038] Another object of the present invention is to provide a computer device, the computer device including a memory and a processor, the memory storing a computer program, the computer program being executed by the processor causing the processor to perform the steps of the reinforcement learning-based multi-cylinder synchronous lifting control method for building construction machines.

[0039] Another object of the present invention is to provide a computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to perform the steps of the reinforcement learning-based multi-cylinder synchronous lifting control method for building construction machines.

[0040] Another objective of this invention is to provide an information data processing terminal for implementing the reinforcement learning-based multi-cylinder synchronous lifting control device for building construction machines.

[0041] Based on the above technical solutions and the technical problems solved, the advantages and positive effects of the technical solution to be protected by this invention are as follows:

[0042] First, this invention proposes a multi-cylinder synchronous lifting control method and device for a building construction machine based on reinforcement learning. Addressing the problem of asynchronous lifting of hydraulic cylinders, it combines coupled synchronous control technology with multi-agent reinforcement learning algorithm to achieve adaptive synchronous control of multiple hydraulic cylinders during the lifting process of the building construction machine.

[0043] This invention selects actual load data from the jacking operation of a building construction machine, employs displacement mean coupling control technology, and trains a high-precision controller using a multi-agent reinforcement learning method to achieve intelligent perception and coordinated jacking of a multi-cylinder system. Taking the construction of a 358-meter high-rise building as an example, the intelligent control algorithm proposed in this invention, under the condition of uneven load distribution on the steel platform, shows through simulation that the maximum synchronization error of the multi-cylinder joint jacking of 700mm is only 1.060mm, less than the 5mm safety threshold. The maximum steady-state error after the jacking is completed is only 0.403mm, less than the target stroke of 700mm × 2% = 14mm, meeting the requirements for practical use. The research results show that the intelligent control algorithm proposed in this invention can adaptively adjust the motion during the jacking process, reduce the motion error of the multi-cylinder system, ensure stable structural jacking, and improve the intelligent construction level of the building construction machine.

[0044] Second, the technical solution of this invention fills a technological gap in the industry both domestically and internationally:

[0045] Traditional multi-cylinder control methods require setting corresponding control parameters and lack versatility in different external environments. To improve the intelligence level of building construction machinery, this invention combines multi-agent reinforcement learning algorithms with coupled synchronous control technology, proposing a reinforcement learning-based multi-cylinder synchronous lifting control method and device for building construction machinery. This method achieves collaborative perception and active control of multiple cylinders under uncertain cylinder load conditions, filling the research gap in intelligent synchronous lifting of building construction machinery both domestically and internationally, and promoting the application and development of multi-agent reinforcement learning in multi-cylinder synchronous control scenarios.

[0046] The technical solution of this invention solves a technical problem that people have long desired to solve but have been unable to achieve:

[0047] When a building construction machine completes the construction of a standard floor, it uses a hydraulic system to lift the construction platform upwards to begin construction of the next standard floor. However, due to the uneven loads on the construction platform, such as steel bars, timber, and concrete placing booms, the multiple sets of hydraulic cylinders under the platform experience uneven stress during lifting, easily leading to asynchronous lifting, deformation of the construction platform, and in severe cases, even the risk of platform overturning. Therefore, the lifting operation is generally considered a critical stage in the building construction process. To solve the problem of asynchronous lifting in building construction machines, improve the safety of high-rise building construction, and enhance the intelligent construction efficiency of building construction machines, this invention uses an advanced multi-agent reinforcement learning algorithm as the controller of the synchronous control system. It employs coupled synchronous control technology to form a multi-cylinder sensing and control loop. By establishing a high-fidelity hydraulic system physical environment, the intelligent agent controller is trained to perform collaborative sensing and proactive adjustment under uncertain load conditions, realizing the intelligent one-button lifting function of the building construction machine. This avoids the shortcomings of traditional methods that require manual adjustment, solves the problem of asynchronous operation of the building construction machine cylinders under uneven loads, and provides technical support and a solid guarantee for promoting my country's building construction machine technology to the world.

[0048] Third, the technical solution of this invention addresses the insufficient accuracy and stability of multi-cylinder synchronous lifting control in existing building construction machinery technologies by proposing a multi-cylinder synchronous lifting control method based on reinforcement learning. By introducing an independent SAC algorithm, multiple agents can learn and control in an independent environment, effectively reducing synchronization and tracking errors during the lifting process. A hydraulic simulation model established using AMESim is employed, and the voltage of the cylinder control valves is dynamically adjusted through an intelligent control algorithm, enabling the system to adapt to changing load conditions in real time and ensuring the accuracy of synchronous lifting.

[0049] Compared with traditional control methods, this invention has achieved significant technical progress in several aspects: First, through the application of reinforcement learning algorithms, the system has the ability to adaptively adjust and maintain high-precision synchronous control in complex environments; second, the design of intelligent control algorithms significantly improves the system's response speed and stability, and reduces errors during the lifting process; finally, the synchronous control method of this invention has been verified to demonstrate its convergence and reliability, providing higher safety and efficiency for multi-cylinder lifting systems in practical applications.

[0050] Fourth, the mathematical model of this invention solves the problem of achieving high accuracy and stability in multi-cylinder synchronous control by accurately describing the tracking error and synchronization error in a multi-cylinder synchronous lifting system. By establishing a mathematical model of tracking error and synchronization error, this invention can accurately calculate the stroke difference of each cylinder during the lifting process and optimize the control strategy using an independent SAC (SoftActor-Critic) algorithm.

[0051] Specifically, the synchronization error formula defined in the mathematical model was proven to be convergent through matrix operations, ensuring that the error gradually approaches zero under the control loop, thereby achieving high-precision synchronous control of multiple cylinders. This mathematical model not only provides a theoretical basis for intelligent control algorithms but also enables the system to operate stably under complex load conditions.

[0052] Compared with traditional control methods, this invention achieves significant improvements in synchronous control accuracy, response speed, and system stability through the establishment and verification of a mathematical model. The introduction of the mathematical model enables the system to better handle complex dynamic relationships, achieving higher control accuracy and reliability, and providing a new solution for multi-cylinder synchronous lifting control methods. Attached Figure Description

[0053] Figure 1 This is a flowchart of the multi-cylinder synchronous lifting control method for a building construction machine based on reinforcement learning, provided in an embodiment of the present invention.

[0054] Figure 2 This is a structural block diagram of a multi-cylinder synchronous lifting control device for a building construction machine based on reinforcement learning, provided in an embodiment of the present invention.

[0055] Figure 3 This is a diagram of the hydraulic system composition of a building construction machine provided in an embodiment of the present invention.

[0056] Figure 4 This is a displacement mean coupling control loop diagram provided in an embodiment of the present invention.

[0057] Figure 5 This is a reward value distribution area map provided in an embodiment of the present invention.

[0058] Figure 6 This is a schematic diagram of the hydraulic system for a building construction machine provided in an embodiment of the present invention. (a) Location distribution of the hydraulic system; (b) Load weight of the hydraulic system (unit: tons)

[0059] Figure 7 These are time-history curves of the lifting displacement of multiple sets of hydraulic cylinders provided in embodiments of the present invention. (a) Equivalent control; (b) Intelligent control.

[0060] Figure 8 This is the maximum synchronization error diagram of the test case provided in the embodiment of the present invention.

[0061] Figure 9 This is the maximum steady-state error diagram of the test case provided in the embodiment of the present invention. Detailed Implementation

[0062] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0063] like Figure 1 As shown in the figure, the multi-cylinder synchronous lifting control method for a building construction machine based on reinforcement learning provided by this invention includes the following steps:

[0064] S101, Multi-cylinder physical environment creation;

[0065] First, a hydraulic model of the building machine was built using the hydraulic simulation tool AMESim. The hydraulic system was simplified to retain core components such as the oil tank, motor, piston pump, proportional directional valve, pressure relief valve, and piston. A vertical load signal was applied to the upper part of the piston to simulate the stacking load on the platform. Figure 3 The model shown is a single hydraulic system for the building construction machine. To further approximate the actual situation on site, the model also considers the mechanical efficiency and volumetric efficiency of the hydraulic pump. The hydraulic pump model of the building construction machine is 25SCY, and the mechanical efficiency is assumed to be 95%. According to the technical manual, the volumetric efficiency is taken as 93.5%.

[0066] Based on this, in order to achieve synchronous lifting control of multiple hydraulic cylinders in a building construction machine, a displacement averaging coupling control loop was established in Simulink, such as... Figure 4 As shown; the displacement mean coupling control method compares the hydraulic output in each sub-loop with the reference model one by one; the present invention defines the reference model as the average value of the output displacement of each power point, and uses a trained intelligent agent as the controller to receive feedback on the cylinder stroke error and adjust the control valve of the cylinder to maintain multi-cylinder synchronization.

[0067]

[0068]

[0069]

[0070] Where x i a(t) represents the stroke of the i-th cylinder at time t, n represents the total number of cylinders in the building machine, and a(t) represents the average stroke of all cylinders at time t. Indicates the input signal, and These represent the tracking error and synchronization error during the lifting process, respectively.

[0071] To prove the convergence of the synchronization error, equation (3) can be expressed as follows:

[0072]

[0073] Through the initial transformation, we know that the coefficient matrix A is a full-rank matrix, as shown below:

[0074]

[0075] From the above, it can be seen that when At that time, x i (t) has a unique solution, indicating that the synchronization error of the control loop can converge to zero;

[0076] S102, Multi-agent reinforcement learning modeling;

[0077] SAC (SoftActor-Critic) algorithm is a deep reinforcement learning algorithm based on maximum entropy theory. It has the following three advantages: (1) By introducing entropy into the learning target, it strengthens the exploration of the environment, making it easier to find better policies under multimodal rewards; (2) It has better robustness and higher sampling efficiency; (3) It provides a stable learning framework for control tasks in continuous action space. The most important feature of SAC algorithm is the use of information entropy, which is an indicator of policy randomness. The greater the entropy, the higher the randomness of the policy. High randomness is obtained through more exploration, which allows the algorithm to learn more information to calculate the optimal solution. Therefore, the algorithm has more output possibilities when facing disturbances, ensuring the robustness of the algorithm. SAC algorithm balances maximizing expected reward and maximizing entropy, making the algorithm robust while ensuring policy optimization.

[0078] However, traditional SAC algorithms are mainly suitable for single-agent reinforcement learning. The lifting process of a building construction machine involves complex coupling and collaborative work of multiple hydraulic cylinders, making traditional SAC algorithms unsuitable for adaptive control of the lifting process. To address this issue, this invention incorporates the concept of independent learning into SAC, proposing the Independent SAC (ISAC) algorithm. ISAC is a completely decentralized algorithm that assumes each agent learns independently in its own environment, without considering changes from other agents. Specifically, it uses parameter sharing techniques during agent training, applying the same set of policy parameters to all agents. This method results in more training data and greater model stability. The prerequisite for using this technique is that the agents are completely homogeneous, meaning their state spaces and action spaces are identical, and their optimization objectives are also identical. The pseudocode for ISAC is shown in Algorithm 1.

[0079]

[0080] S103, Intelligent control algorithm debugging;

[0081] To improve the performance of the agent and enable it to fully perceive its environment, explicit observation of its state is crucial; in this technology, the agent is used as a controller in hydraulic pressure, and the agent's state is as follows:

[0082]

[0083] Where, x i (t) represents the stroke of the i-th cylinder at time t; This represents the difference between the i-th cylinder's stroke and the target stroke at time t, i.e., the tracking error; The difference between the stroke of the i-th cylinder at time t and the average stroke of all cylinders, i.e., the synchronization error;

[0084] The agent's action is set to the control voltage of the proportional directional valve. The maximum voltage for controlling the piston to rise is -40mA, and the voltage for closing the valve is 0mA. Therefore, the action range of agent 1 is set to [-40,0]. In order to judge the performance of the action, the critique network needs to be trained based on the reward value of the action and the simulated termination state. Therefore, the reward function plays a key role in guiding the behavior of the agent and guiding the learning process to achieve the predefined goal. For the lifting process of the building machine, multiple cylinders need to take into account both tracking error and synchronization error to achieve collaborative work. Therefore, the overall design of the reward function must include both types of error. In general, the smaller the error during the lifting process, the larger the reward value is given. The reward function and termination condition studied are shown in equations (7) and (8).

[0085]

[0086]

[0087] Where ω1 and ω2 represent the weights of the two errors in the reward function; γ represents the additional reward; δ represents the penalty; T r and T s These represent the simulation time and the stopping time, respectively; α and β represent the threshold values ​​for tracking error and synchronization error, respectively.

[0088] Figure 5 The distribution area of ​​the bonus value on the hydraulic cylinder lifting stroke curve is shown; for example... Figure 5 As shown, the lifting process of the building machine can be divided into two stages: the synchronous working stage and the synchronous stopping stage. In region I, the agent's task is to lift to the target displacement while reducing the synchronization error with other hydraulic cylinder pistons. The reward for this stage comes from the tracking error. and synchronization error The reduction of [something]; in region II, the agent enters the error band of the target displacement and then gradually stops rising; to encourage the agent to stay in this region for as long as possible, an additional constant reward term γ is added to the reward function; region III is a penalty region, representing that if the agent goes out of the error band, it will immediately receive a penalty value; furthermore, in the simulation of the agent's interaction with the physical environment, the termination condition is defined as: when the simulation time T [something]... r >T s When the simulation ends, it means the intelligent agent has successfully completed the task; if the tracking error occurs during the cylinder lifting process... or synchronization error If the threshold α or β is exceeded, the simulation will stop immediately. After training, the corresponding cylinder load will be set according to the actual lifting situation of the building machine to test the synchronous lifting effect of the intelligent control algorithm.

[0089] like Figure 2 As shown in the figure, an embodiment of the present invention provides a multi-cylinder synchronous lifting control device for a building construction machine based on reinforcement learning, comprising:

[0090] Create a module for creating a physical environment for multiple hydraulic cylinders;

[0091] The modeling module is used for multi-agent reinforcement learning modeling.

[0092] The debugging module is used for debugging via intelligent control algorithms.

[0093] Another object of the present invention is to provide a computer device, the computer device including a memory and a processor, the memory storing a computer program, the computer program being executed by the processor causing the processor to perform the steps of the reinforcement learning-based multi-cylinder synchronous lifting control method for building construction machines.

[0094] Another object of the present invention is to provide a computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to perform the steps of the reinforcement learning-based multi-cylinder synchronous lifting control method for building construction machines.

[0095] Another objective of this invention is to provide an information data processing terminal for implementing the reinforcement learning-based multi-cylinder synchronous lifting control device for building construction machines.

[0096] This implementation case provides a reinforcement learning-based multi-cylinder synchronous lifting control method and device for a building construction machine. The case study applies this method to the construction of a 358m high-rise building. The main steps include:

[0097] Step 1: Hydraulic System and Control Loop Modeling. This invention uses AMESim to model a single hydraulic system for a building construction machine, including components such as an oil tank, motor, piston pump, proportional directional valve, pressure relief valve, and piston. The maximum load capacity of the hydraulic cylinder is 150 tons, and the maximum flow rate is 35 L / min. More modeling parameters are shown in Table 1. This invention establishes a displacement mean coupled control loop in Simulink, that is, comparing the hydraulic system output in each sub-loop with the reference model one by one. The reference model is defined as the average value of the output displacement of each group of cylinders. The controller receives feedback on the cylinder stroke error and adjusts the control valves of the cylinders to maintain synchronization among multiple cylinders. A high-precision controller is trained in Python using a multi-agent reinforcement learning algorithm, see Step 2.

[0098] Table 1. Hydraulic System Modeling Parameters

[0099]

[0100] Step 2: Multi-agent reinforcement learning modeling. For example... Figure 6As shown, based on the measured load data of a building construction machine during a lifting operation, it can be seen that the load distribution on the upper part of the 10 sets of hydraulic cylinders is uneven. Therefore, the upper load is used as a variable parameter, with the load weight variation range set to [50t, 120t]. Before each training session, the upper load weight is randomly selected for the hydraulic cylinders to enhance the robustness of the algorithm. During the multi-agent training process, the Critic network is a 3-layer neural network composed of 512 neurons, and the Actor network is a 2-layer neural network composed of 512 neurons. The learning rates of the Critic network and the Actor network are set to 0.0001 and 0.0005, respectively. The discount factor is set to 0.98, and the soft update factor τ is 0.005. During model training, the states received by the agents are normalized to increase the generality of the agents. Considering computational cost and time, the training condition is set as two sets of hydraulic cylinders lifting 100mm simultaneously, with an action time interval of 0.2s, and a single training time of 20s. Regarding the reward function, the weight coefficients ω1 and ω2 are set to 0.9 and 0.1 respectively, the additional reward value γ is 5, the penalty value δ is -10, and the thresholds α and β for tracking error and synchronization error are set to 2% and 1mm beyond the target travel distance, respectively; a settlement reward value of 20 is given for successfully completing the task.

[0101] Step 3: Intelligent control algorithm debugging. In the test environment, configure the following settings for 10 sets of hydraulic cylinders: Figure 6 The load weight is shown, and the agent trained in step 2 is used as the synchronous lifting controller. The test conditions are set as follows: 10 sets of hydraulic cylinders simultaneously lift 700mm, with an action time interval of 0.2s, and a test time of 150s. The test results are as follows. Figure 7 As shown. Figure 7 (a) is the cylinder displacement curve under traditional equivalent control. Each cylinder is controlled individually. The maximum synchronization error of lifting 700mm is 22.1mm. This is because the cooperative work between cylinders is not considered. Figure 7(b) shows the cylinder displacement curves under the intelligent control algorithm of this invention. It can be seen that the displacement curves of the 10 sets of cylinders maintained good synchronization, and the lifting displacement curve exhibited an "S" shape, indicating that the adaptive lifting process can be roughly divided into three stages: In the first stage, all cylinders sensed each other and adjusted their own speeds to ensure coordinated work, resulting in a relatively flat displacement curve; in the second stage, all cylinders began normal coordinated lifting, and the slope of the displacement curve increased; in the third stage, after approaching the target stroke, the 10 sets of cylinders gradually decelerated until the target stroke was reached. Throughout the process, the maximum synchronization error of multiple cylinders lifting 700mm was only 1.060mm, less than the 5mm safety threshold. The maximum steady-state error after the lifting was completed was 0.403mm, less than 700mm × 2% = 14mm, meeting the basic error requirements. The research results show that the intelligent control algorithm proposed in this invention can adaptively adjust the cylinder movement during the lifting process, reduce the multi-cylinder movement error, ensure stable structural lifting, and improve the dynamic safety of the building construction machine.

[0102] To further verify the robustness of the control method of the present invention, the load of 10 sets of hydraulic cylinders was sampled using the Latin hypercube sampling method, with a sampling range of [50t, 120t], generating 20 cases. The control method of the present invention was used for synchronous lifting control, and the maximum synchronization error and steady-state error of multi-cylinder lifting were observed. Figure 8 The maximum synchronization error was demonstrated in 20 cases using the control method of this invention. As shown in the figure, under different external loads, the maximum synchronization among the multiple cylinders maintained a similar variation pattern, indicating that the control method of this invention can adaptively adjust the movement of multiple cylinders under uneven loads. Furthermore, the maximum synchronization error in all cases was approximately 1.2 mm, meeting safety requirements. Figure 9 The maximum steady-state error using the control method of this invention was demonstrated in 20 cases, with a maximum steady-state error of 3.86 mm, which is less than 700 mm × 2% = 14 mm, indicating that the multi-cylinder system meets the basic error requirements in all cases. In summary, the control method of this invention is applicable to scenarios with uneven load distribution on steel platforms and has a significant control effect on the coordinated operation of multiple hydraulic cylinders, exhibiting good generalization and robustness.

[0103] It should be noted that embodiments of the present invention can be implemented in hardware, software, or a combination of both. The hardware portion can be implemented using dedicated logic; the software portion can be stored in memory and executed by appropriate instructions, such as a microprocessor or dedicated hardware. Those skilled in the art will understand that the above-described devices and methods can be implemented using computer-executable instructions and / or included in processor control code, for example, such code provided on a carrier medium such as a disk, CD, or DVD-ROM, a programmable memory such as read-only memory (firmware), or a data carrier such as an optical or electronic signal carrier. The devices and modules of the present invention can be implemented by hardware circuitry such as very large-scale integrated circuits or gate arrays, semiconductors such as logic chips, transistors, or programmable hardware devices such as field-programmable gate arrays, programmable logic devices, etc., or by software executed by various types of processors, or by a combination of the above-described hardware circuitry and software, such as firmware.

[0104] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications, equivalent substitutions, and improvements made by those skilled in the art within the scope of the technology disclosed in the present invention, and within the spirit and principles of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A multi-cylinder synchronous lifting control method for a building construction machine based on reinforcement learning, characterized in that, A multi-cylinder physical environment model was established using the hydraulic simulation tool AMESim, and an independent SAC algorithm was introduced for multi-agent reinforcement learning modeling. After the agent monitors the difference between the cylinder stroke and the target stroke and the average difference between the strokes of the cylinders in real time, it dynamically adjusts the voltage of the control valve to ensure the accuracy and stability of the synchronous lifting of the multi-cylinder system. By designing specific reward functions and termination conditions, the system can effectively converge the synchronization error, ensuring the reliability and safety of the lifting process. The agent's state is as follows: (6) in, This represents the stroke of the i-th cylinder at time t; This represents the difference between the i-th cylinder's stroke and the target stroke at time t, i.e., the tracking error; The difference between the stroke of the i-th cylinder at time t and the average stroke of all cylinders, i.e., the synchronization error; The agent's action is set to the control voltage of the proportional directional valve, the maximum current for controlling the piston to rise is -40 mA, and the current for closing the valve is 0 mA. The agent's action range is set to [-40, 0]. The smaller the error during the jacking process, the greater the reward value is given; the reward function and termination condition studied are shown in equations (7) and (8); (7) (8) in, and This represents the weights of the two errors in the reward function; This indicates the additional reward value; Indicates the penalty value; and These represent the simulation time and the stop time, respectively. and These represent the threshold values ​​for tracking error and synchronization error, respectively.

2. The multi-cylinder synchronous lifting control method for a building construction machine based on reinforcement learning as described in claim 1, characterized in that, The creation of the multi-cylinder physical environment: First, a hydraulic model of the building machine was built using the hydraulic simulation tool AMESim. The hydraulic system was simplified to retain the core components, including the oil tank, motor, piston pump, proportional directional valve, pressure relief valve, and piston. A vertical load signal was applied to the upper part of the piston to simulate the stacking load on the platform. The reference model is defined as the average value of the output displacement of each power point. A trained intelligent agent is used as the controller to receive feedback on the cylinder stroke error and adjust the control valve of the cylinder to maintain synchronization of multiple cylinders. (1) (2) (3) in It is the first One oil cylinder in The schedule of the moment This indicates the total number of hydraulic cylinders in the building construction machine. Indicates that all hydraulic cylinders are in The average distance traveled at each time point Indicates the input signal, and These represent the tracking error and synchronization error during the lifting process, respectively. To prove the convergence of the synchronization error, formula (3) can be expressed as follows: (4) Through the initial transformation, we know that the coefficient matrix... The matrix is ​​full rank, as shown below: (5) From the above, it can be seen that when When = 0, The existence of a unique solution indicates that the synchronization error of the control loop can converge to zero.

3. The multi-cylinder synchronous lifting control method for a building construction machine based on reinforcement learning as described in claim 1, characterized in that, The multi-agent reinforcement learning modeling: By incorporating the concept of independent learning into SAC, the Independent SAC algorithm is proposed. ISAC is a completely decentralized algorithm that assumes that each agent learns independently in its own environment without considering changes made by other agents.

4. A reinforcement learning-based multi-cylinder synchronous lifting control device for a building construction machine, implementing the reinforcement learning-based multi-cylinder synchronous lifting control method for a building construction machine as described in any one of claims 1-3, characterized in that, The reinforcement learning-based multi-cylinder synchronous lifting control device for the building construction machine includes: Create a module for creating a physical environment for multiple hydraulic cylinders; The modeling module is used for multi-agent reinforcement learning modeling. The debugging module is used for debugging via intelligent control algorithms.

5. A computer device, characterized in that, The computer device includes a memory and a processor. The memory stores a computer program, which, when executed by the processor, causes the processor to perform the steps of the reinforcement learning-based multi-cylinder synchronous lifting control method for building construction machines as described in any one of claims 1-3.

6. A computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to perform the steps of the reinforcement learning-based multi-cylinder synchronous lifting control method for a building construction machine as described in any one of claims 1-3.

7. An information data processing terminal, characterized in that, The information data processing terminal is used to implement the reinforcement learning-based multi-cylinder synchronous lifting control device for building machines as described in claim 4.

Images