Dynamic balance control method for column unloading construction based on physical constraint reinforcement learning

Abstract

The application discloses a center column unloading construction dynamic balance control method based on physical constraint reinforcement learning. According to the strong dependence progressive logic of "foundation environment construction decision model modeling precise state sensing model error calibration robust safety training real-time closed loop execution", the application constructs a digital twin virtual simulation environment based on a finite element physical engine, realizes global state reconstruction and uncertainty quantification of sparse sensing data by adopting a Bayesian graph neural network, solves the model distortion problem by online correction of the structure stiffness matrix through a meta-learning algorithm, and generates a robust control strategy meeting mechanical hard constraints by combining double-agent adversarial training and differentiable convex optimization physical projection layers. The application realizes 10Hz high-frequency closed-loop autonomous control in the center column unloading process, reduces the maximum stress fluctuation of the structure by 64.5%, and reduces the number of manual interventions by 98.7%, effectively solving the technical problems that the real-time performance, safety and adaptability of the traditional control method are difficult to balance.

Description

Technical Field

[0001] This invention belongs to the field of interdisciplinary technology of large-span spatial structure construction, computational mechanics and artificial intelligence. In particular, it relates to a dynamic balance control method for unloading the central column during the renovation or construction of large-span steel structures (such as coal sheds, stadiums, convention centers, etc.) based on physical constraint reinforcement learning to ensure the dynamic balance and safe unloading of the structure. Background Technology

[0002] Time-varying construction control of large-span spatial structures (such as formwork removal, unloading, and jacking) is an extremely high-risk engineering project. Taking the removal of the central column in an enclosed coal yard as an example, the hundreds of tons of load originally borne by the central column need to be gradually transferred to the newly reinforced surrounding structure using jacks. Because the space frame structure is a nonlinear, complex, and statically indeterminate system, manual control of jack unloading can easily lead to sudden increases in local stress, causing structural instability or even instantaneous collapse.

[0003] Currently, the following technical solutions are mainly used for unloading construction of large-span steel structures: Manual experience control method: Technicians pre-determine unloading plans based on design drawings and construction experience, and unload in stages by manually operating hydraulic jacks. This method has drawbacks such as slow response, poor coordination, and high subjectivity. It is difficult for multiple workers to operate multiple jacks simultaneously to achieve precise coordination, and the formulation and execution of the unloading plan are highly dependent on the personal experience of the technicians.

[0004] Programmed automatic control method: The PLC controller automatically executes the unloading program based on preset displacement or pressure thresholds. Although this method improves execution efficiency, it has poor adaptability. The preset program cannot cope with nonlinear time-varying disturbances at the construction site (such as uneven foundation settlement, sudden changes in wind load, temperature deformation, etc.), and it lacks a feedback adjustment mechanism and sufficient safety redundancy.

[0005] Traditional optimization algorithm control methods employ optimization techniques such as genetic algorithms and particle swarm optimization to solve for the optimal unloading path. These methods suffer from low computational efficiency, and the optimization of large-scale structures is time-consuming, making it difficult to meet real-time control requirements. Furthermore, they are highly model-dependent, with optimization results heavily reliant on the accuracy of the finite element model, and cannot be adaptively corrected based on measured data during construction.

[0006] In summary, existing technologies have failed to effectively solve the technical challenge of simultaneously achieving "real-time performance, safety, and adaptability" in the unloading construction of large-span steel structures. Summary of the Invention

[0007] To address the aforementioned technical problems, this invention provides a dynamic balance control method for unloading construction of central columns based on physical constraint reinforcement learning. Following a strongly dependent progressive logic of "basic environment construction → decision model modeling → precise state perception → model error calibration → robust safety training → real-time closed-loop execution," the method achieves autonomous and refined control of the construction process through digital twin-driven, physical rule-guided decision-making, adversarial robust training, and uncertainty perception closed-loop adjustment.

[0008] To achieve the above objectives, the technical solution of the present invention is implemented as follows: A dynamic equilibrium control method for unloading of the middle column during construction based on physical constraint reinforcement learning includes the following steps: S1. Construct a digital twin virtual simulation environment based on a finite element physical engine, establish a real-time bidirectional data transmission channel between field sensors and simulation models, realize spatiotemporal synchronous mapping between physical entities and digital models, and provide a unified physical benchmark for all subsequent simulations, training and control. S2. Based on the digital twin virtual simulation environment described in step S1, the entire process of unloading the central column is modeled as a Markov decision process in which the state evolves dynamically with the construction progress. The basic elements of state, action, and reward are defined, and the basic decision framework of the physical perception reinforcement learning control system is constructed. S3. To address the global state unobservability problem caused by sparse sensor data in the Markov decision-making process described in step S2, a robust state observer is introduced. A Bayesian graph neural network is used to reconstruct the global structural state of the sparse sensor data transmitted in step S1, and at the same time, the observation uncertainty measure of each state component is output, forming a complete state space containing the reconstructed stress tensor, displacement vector and uncertainty tensor, providing input for subsequent model calibration and decision-making. S4. Based on the global state output in step S3 and the predicted state of the virtual simulation environment in step S1, construct a system identification loop for virtual and real residual feedback, calculate the residual between the two in real time, and when the residual exceeds the preset threshold, trigger the online adaptive correction of the structural stiffness matrix of the finite element model in step S1 to solve the distortion problem between the digital twin model and the actual construction conditions. S5. Based on the decision framework described in step S2, the state space described in step S3, and the calibrated digital twin model described in step S4, a robust control strategy is generated using a dual-agent adversarial training mode based on a red-blue adversarial game mechanism. At the same time, to address the security risks caused by random exploration in reinforcement learning, a differentiable convex optimization layer is embedded in the output layer of the reinforcement learning neural network as a physical consistency mapping architecture. This transforms the mechanical equilibrium equations and stress-deformation constraints into hard constraints for the strategy output, resulting in a safe control strategy that satisfies physical rules. S6. Deploy the safety control strategy trained in step S5 to the field control system. Combine the global state and observation uncertainty assessment results output in real time in step S3, dynamically adjust the unloading command online, and drive the mechanical actuator to complete the whole process of adaptive balance control for unloading the middle column.

[0009] Furthermore, the mechanical actuator includes a hydraulic jack, an electric jack, or an electromagnetic drive mechanism; the unloading command includes the lifting rate, output pressure value, or physical position change of the actuator.

[0010] Furthermore, in step S6, by controlling the amount of motion of the mechanical actuator, the displacement difference between any two adjacent support points is always less than a preset coordination threshold. This ensures that the structure is subjected to uniform stress.

[0011] Furthermore, in step S5, the dual-agent adversarial training adopts minimax game logic. The goal of the master control agent is to minimize the structural instability risk and maximize construction efficiency, while the goal of the adversarial agent is to generate extreme load and disturbance combinations that cause local buckling of the structure. The reward function of the master control agent includes a penalty term for the instability risk generated by the adversarial agent.

[0012] Furthermore, in step S3, the specific implementation of the Bayesian graph neural network is as follows: (1) Graph structure construction: The nodes of the steel structure are used as the vertices of the graph, and the connection relationship of the members is used as the edges of the graph. The weight of the edge is determined by the stiffness coefficient of the corresponding member. (2) Network structure: It consists of 3 layers of spectral domain graph convolutional layers and 1 layer of variational inference layer. The variational inference layer uses reparameterization technique to achieve differentiability. (3) Output: Mean state vector and uncertainty covariance matrix ,in The diagonal elements represent the estimated variance of each state component; (4) Training objective: The lower bound of evidence ELBO is used as the loss function, and the formula is:

[0013] In the formula: The lower bound of the evidence loss function value; The expected value under the variational posterior distribution; It is a variational posterior distribution; A vector of latent variables; For observation data vectors; Let KL divergence be a metric. This represents the prior distribution of the latent variable.

[0014] Furthermore, in step S4, the specific implementation method of online correction of the stiffness matrix is ​​as follows: (1) Triggering condition: When the residual between simulation and actual measurement Correction is initiated when the value exceeds the measured value by 5%. (2) Correction algorithm: The online adaptation method based on Model Independent Meta-Learning (MAML) is adopted, and the specific steps are as follows: a. Inner loop: Under the current stiffness parameters Calculate the virtual-real consistency loss Perform a gradient descent step to obtain temporary parameters. ,in, This represents the loss function value between simulation and actual measurement; express The measured data vector at time point; Represents the observation mapping function; express The structural state vector at time step; This represents the current stiffness parameter matrix. Denotes the square of the L2 norm. Represents the temporary stiffness parameter matrix; This represents the inner loop learning rate, which is dimensionless. Indicates the stiffness parameter The gradient operator; This represents the loss function value between simulation and actual measurement; b. Outer loop: in temporary parameters Calculate the element loss The corrected stiffness matrix is ​​obtained by updating the second-order gradient. ,in, This represents the value of the meta-loss function; express The measured data vector at time point; Represents the observation mapping function; express The structural state vector at time t, This represents the updated stiffness parameter matrix; The outer loop learning rate is dimensionless. (3) Parameter settings: Inner loop learning rate outer loop learning rate Each time it is triggered, it will be updated 3-5 times.

[0015] Furthermore, in step S5, the microconvex optimization layer performs a security projection on the action based on a dynamic trust region, wherein the dynamic trust region... The value ranges from 5% to 20% of the displacement step length of a single actuator, limiting the single-step movement amplitude to ensure structural stability.

[0016] Furthermore, the comprehensive reward function of the main control agent is specifically as follows:

[0017] In the formula, For comprehensive cumulative rewards; , , These are the weighting coefficients; This represents the maximum stress increment of the component. The allowable stress of the material; This is a motion smoothing term used to suppress the dynamic load effect caused by frequent fluctuations of the jack; To counteract the perturbation tensors generated by the agent; To perturb the search space; This is the loss function for the risk of instability.

[0018] Furthermore, in step S5, the microconvex optimization layer uses the primal-dual interior point method to solve the motion projection problem, constructing the mechanically feasible domain as a linearly constrained convex polyhedron. The single-step solution takes less than 5ms, meeting the real-time requirements of 10Hz closed-loop control.

[0019] Furthermore, in step S6, the control strategy is adaptively adjusted based on the observation uncertainty tensor output in step S3: when the uncertainty of the state estimation exceeds the preset threshold, the unloading step size is automatically reduced to 30%-50% of the original step size, and the sensor sampling frequency is increased to above 10Hz to reduce the construction risk under high uncertainty conditions.

[0020] Beneficial Effects: This invention, by embedding physical constraints into a reinforcement learning framework, achieves a technological leap from "experience-driven" to "physically-aware self-evolving control" in the field of large-scale steel structure construction control. Its beneficial effects are mainly reflected in the following dimensions: (1) Algorithm-level construction safety safeguard: By embedding a differentiable physical projection layer that supports gradient backpropagation at the end of the decision-making process, the mechanical equilibrium equation is transformed into a hard constraint in the policy space, ensuring that every unloading command output by the agent is physically verified, and reducing the peak stress fluctuation during the unloading process by more than 60%.

[0021] (2) Significantly improve the control robustness in complex environments: The "red-blue confrontation" game mechanism is introduced. The main agent has undergone millions of extreme working conditions in the virtual environment, and has strong fault tolerance and force flow redistribution capabilities, ensuring that the structure remains stable in environments with no production stoppage and strong disturbances.

[0022] (3) Solve the problem of “distortion” between digital twin model and real working conditions: The system integrates “uncertainty perception” observer and meta-learning calibration loop. The system can identify the confidence level of the sensor in real time and automatically start online correction of stiffness matrix according to residual, reducing the model prediction residual from an average of 15% to less than 3%.

[0023] (4) Significantly reduce the cost and risk of manual intervention: It realizes autonomous and refined lifting control throughout the process, reducing the need for manual intervention by 80%; the 10Hz high-frequency closed-loop adjustment capability makes the unloading process smooth and efficient, greatly shortening the duration of high-risk operations. Attached Figure Description

[0024] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an undue limitation of the invention. In the drawings: Figure 1 This is a flowchart illustrating the steps of the dynamic balance control method for unloading construction of the middle column based on physical constraint reinforcement learning, as described in an embodiment of the present invention. Figure 2 This is a diagram of the overall system architecture of the present invention; Figure 3 A comparison curve of stress fluctuations for the three control methods; Figure 4 A comparison curve of displacement compatibility for the three control methods; Figure 5 A bar chart comparing the performance indicators of the three control methods; Figure 6 Ablation experiment analysis diagram showing the contribution of each technical module of the present invention; Figure 7 This is a comparison of the policy convergence curves of the present invention and conventional reinforcement learning methods; Figure 8 A comparison of the robustness of the three control methods under different noise levels; Figure 9 This is a diagram showing the effect of online correction of the stiffness matrix of the meta-learning method in this invention. Detailed Implementation

[0025] It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other.

[0026] The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0027] Example 1 See Figure 1 A dynamic equilibrium control method for unloading of the middle column during construction based on physical constraint reinforcement learning includes the following steps: S1. Construct a digital twin virtual simulation environment based on a finite element physical engine, establish a real-time bidirectional data transmission channel between field sensors and simulation models, realize spatiotemporal synchronous mapping between physical entities and digital models, and provide a unified physical benchmark for all subsequent simulations, training and control. S2. Based on the digital twin virtual simulation environment described in step S1, the entire process of unloading the central column is modeled as a Markov decision process in which the state evolves dynamically with the construction progress. The basic elements of state, action, and reward are defined, and the basic decision framework of the physical perception reinforcement learning control system is constructed. S3. To address the global state unobservability problem caused by sparse sensor data in the Markov decision-making process described in step S2, a robust state observer is introduced. A Bayesian graph neural network is used to reconstruct the global structural state of the sparse sensor data transmitted in step S1, and at the same time, the observation uncertainty measure of each state component is output, forming a complete state space containing the reconstructed stress tensor, displacement vector and uncertainty tensor, providing input for subsequent model calibration and decision-making. S4. Based on the global state output in step S3 and the predicted state of the virtual simulation environment in step S1, construct a system identification loop for virtual and real residual feedback, calculate the residual between the two in real time, and when the residual exceeds the preset threshold, trigger the online adaptive correction of the structural stiffness matrix of the finite element model in step S1 to solve the distortion problem between the digital twin model and the actual construction conditions. S5. Based on the decision framework described in step S2, the state space described in step S3, and the calibrated digital twin model described in step S4, a robust control strategy is generated using a dual-agent adversarial training mode based on a red-blue adversarial game mechanism. At the same time, to address the security risks caused by random exploration in reinforcement learning, a differentiable convex optimization layer is embedded in the output layer of the reinforcement learning neural network as a physical consistency mapping architecture. This transforms the mechanical equilibrium equations and stress-deformation constraints into hard constraints for the strategy output, resulting in a safe control strategy that satisfies physical rules. S6. Deploy the safety control strategy trained in step S5 to the field control system. Combine the global state and observation uncertainty assessment results output in real time in step S3, dynamically adjust the unloading command online, and drive the mechanical actuator to complete the whole process of adaptive balance control for unloading the middle column.

[0028] This embodiment first establishes a digital twin physical benchmark to provide a unified simulation base for all steps; it transforms the engineering problem into a Markov decision problem solvable by reinforcement learning; it uses Bayesian graph neural networks to solve the global state perception problem of sparse sensing; it uses meta-learning to correct model errors in real time to ensure the accuracy of the control benchmark; it generates robust strategies through red-blue adversarial training and achieves algorithm-level safety fallback through physical projection; and it combines real-time state and uncertainty to achieve adaptive closed-loop control, thereby systematically solving the three major technical problems of poor real-time performance of traditional column unloading, insufficient safety redundancy, and inability to cope with nonlinear disturbances. In addition, it achieves high-frequency autonomous fine control, reduces the maximum stress fluctuation of the structure, reduces the number of manual interventions, shortens the unloading cycle, and greatly reduces the number of stress exceedances throughout the construction process.

[0029] In a specific example, the mechanical actuator includes a hydraulic jack, an electric jack, or an electromagnetic drive mechanism; the unloading command includes the lifting rate, output pressure value, or physical position change of the actuator.

[0030] This embodiment clarifies that the physical execution carrier of this method is the mainstream hydraulic / electric / electromagnetic lifting mechanism in the engineering field, and the control commands cover the three core dimensions of displacement, pressure, and rate, thus concretizing the abstract "control action" into a physical quantity that can be realized in engineering.

[0031] In a specific example, in step S6, the amount of motion of the mechanical actuator is controlled so that the displacement difference between any two adjacent support points is always less than a preset coordination threshold. This ensures that the structure is subjected to uniform stress.

[0032] This embodiment transforms the abstract goal of "structural dynamic equilibrium" into a quantifiable hard indicator: by adjusting the action of each jack in real time, the displacement difference between any adjacent support points is forced to always be lower than the preset safety threshold, thus avoiding uneven load distribution from the root. This embodiment ensures uniform stress on the structure from a mechanical perspective, preventing local stress surges that could lead to instability. It provides clear and detectable judgment criteria for real-time safety monitoring during construction and forms a dual safety guarantee with the displacement coordination constraints of the physical projection layer.

[0033] In a specific example, in step S5, the dual-agent adversarial training adopts minimax game logic. The goal of the master control agent is to minimize the structural instability risk and maximize construction efficiency, while the goal of the adversarial agent is to generate extreme load and disturbance combinations that cause local buckling of the structure. The reward function of the master control agent includes a penalty term for the instability risk generated by the adversarial agent.

[0034] This embodiment introduces a red-blue adversarial game mechanism, employing minimax game logic: The red side (the main controlling agent) aims to minimize instability risk and maximize construction efficiency. Blue side (adversarial agent): The goal is to generate extreme loads / disturbances (such as foundation settlement, sudden wind load changes) that can lead to local buckling. The reward function of the main agent includes the instability risk penalty term generated by the blue side, and the agent learns to resist disturbances through game theory.

[0035] This embodiment allows the main intelligent agent to undergo millions of extreme working condition simulations in a virtual environment, actively learning the ability to redistribute force flow. The system's robustness to sudden disturbances is significantly improved, and the unloading of the central column can be carried out without interrupting production.

[0036] In a specific example, in step S3, the Bayesian graph neural network is implemented as follows: (1) Graph structure construction: The nodes of the steel structure are used as the vertices of the graph, and the connection relationship of the members is used as the edges of the graph. The weight of the edge is determined by the stiffness coefficient of the corresponding member. (2) Network structure: It consists of 3 layers of spectral domain graph convolutional layers and 1 layer of variational inference layer. The variational inference layer uses reparameterization technique to achieve differentiability. (3) Output: Mean state vector and uncertainty covariance matrix ,in The diagonal elements represent the estimated variance of each state component; (4) Training objective: The lower bound of evidence ELBO is used as the loss function, and the formula is:

[0037] In the formula: The lower bound of the evidence loss function value; The expected value under the variational posterior distribution; It is a variational posterior distribution; A vector of latent variables; For observation data vectors; Let KL divergence be a metric. This represents the prior distribution of the latent variable.

[0038] This embodiment addresses the problem of unobservable global state caused by sensor sparsity: 1. The steel structure topology is transformed into a graph data structure (nodes = vertices, rods = edges); 2. Structural features are extracted through 3 layers of spectral domain graph convolution, and probabilistic output is achieved through 1 layer of variational inference; 3. The global state mean vector and uncertainty covariance matrix are output simultaneously to quantify the perceived confidence level.

[0039] This embodiment only requires sparse sensors to reconstruct the stress and displacement distribution across the entire field, reducing sensor deployment costs and solving the key bottleneck of traditional methods that "can only observe the local and cannot perceive the global situation." The output uncertainty metric provides the core decision-making basis for subsequent adaptive control.

[0040] In a specific example, the online correction of the stiffness matrix in step S4 is implemented as follows: (1) Triggering condition: When the residual between simulation and actual measurement Correction is initiated when the value exceeds the measured value by 5%. (2) Correction algorithm: The online adaptation method based on Model Independent Meta-Learning (MAML) is adopted, and the specific steps are as follows: a. Inner loop: Under the current stiffness parameters Calculate the virtual-real consistency loss Perform a gradient descent step to obtain temporary parameters. ,in, This represents the loss function value between simulation and actual measurement; express The measured data vector at time point; Represents the observation mapping function; express The structural state vector at time step; This represents the current stiffness parameter matrix. Denotes the square of the L2 norm. Represents the temporary stiffness parameter matrix; This represents the inner loop learning rate, which is dimensionless. Indicates the stiffness parameter The gradient operator; This represents the loss function value between simulation and actual measurement; b. Outer loop: in temporary parameters Calculate the element loss The corrected stiffness matrix is ​​obtained by updating the second-order gradient. ,in, This represents the value of the meta-loss function; express The measured data vector at time point; Represents the observation mapping function; express The structural state vector at time t, This represents the updated stiffness parameter matrix; The outer loop learning rate is dimensionless. (3) Parameter settings: Inner loop learning rate outer loop learning rate Each time it is triggered, it will be updated 3-5 times.

[0041] This embodiment constructs a virtual-real residual feedback loop: 1. Real-time calculation of the residual between the simulation prediction value and the measured value, triggering correction when the residual > 5%; 2. Using the MAML meta-learning algorithm, the structural stiffness matrix is ​​corrected online through second-order gradient updates of "inner loop task adaptation + outer loop meta-optimization"; 3. Fixed learning rate parameters to ensure the real-time performance and stability of the correction.

[0042] This embodiment solves the problem of "distortion" between digital twin models and actual construction conditions. The model prediction residual is reduced from 15% to less than 3%. It automatically adapts to construction uncertainties such as material parameter deviations and loose connectors. The single correction takes about 50ms, which meets the requirements of real-time control.

[0043] In a specific example, in step S5, the microconvex optimization layer performs a security projection on the action based on a dynamic trust region, wherein the dynamic trust region... The value ranges from 5% to 20% of the displacement step length of a single actuator, limiting the single-step movement amplitude to ensure structural stability.

[0044] This embodiment introduces dynamic trust region constraints in the differentiable convex optimization layer, limiting the single-step lifting amount to 5%-20% of the single displacement step size, to prevent the agent from outputting overly aggressive actions.

[0045] This embodiment balances the flexibility and safety of control, ensuring construction efficiency while avoiding instantaneous structural instability caused by excessive movement. It provides clear action boundary constraints for the physical projection layer and complements the adjacent displacement difference constraints, covering both single-step action and multi-step coordination safety dimensions.

[0046] In a specific example, the comprehensive reward function of the main control agent is as follows:

[0047] In the formula, For comprehensive cumulative rewards; , , These are the weighting coefficients; This represents the maximum stress increment of the component. The allowable stress of the material; This is a motion smoothing term used to suppress the dynamic load effect caused by frequent fluctuations of the jack; To counteract the perturbation tensors generated by the agent; To perturb the search space; This is the loss function for the risk of instability.

[0048] In this embodiment, the main intelligent agent adopts a multi-objective weighted comprehensive reward function: stress safety term: ensures that the stress of the component does not exceed the allowable value; action smoothing term: suppresses frequent fluctuations of the jack; risk resistance term: forces consideration of the worst working condition generated by the blue team, and achieves multi-objective optimization of "safety first, efficiency in mind, and robustness enhancement". The action smoothing term avoids the dynamic load effect from aggravating structural vibration. The risk resistance term further enhances the survivability of the system under extreme working conditions.

[0049] In a specific example, in step S5, the microconvex optimization layer uses the primal-dual interior point method to solve the motion projection problem, constructing the mechanically feasible region as a linearly constrained convex polyhedron. The single-step solution takes less than 5ms, meeting the real-time requirements of 10Hz closed-loop control.

[0050] In this embodiment, the microconvex optimization layer uses the primal-dual interior point method to solve the action projection problem, constructing the mechanically feasible region as a linearly constrained convex polyhedron to ensure the uniqueness and global optimality of the solution.

[0051] This embodiment solves the technical problem of "slow physical constraint calculation and inability to meet real-time requirements". The single-step solution time is <5ms, and the computational complexity for supporting 10Hz high-frequency closed-loop control is O(n^2). 2 It can be extended to large-scale engineering scenarios involving dozens of actuators.

[0052] In a specific example, in step S6, the control strategy is adaptively adjusted based on the observation uncertainty tensor output in step S3: when the uncertainty of the state estimation exceeds the preset threshold, the unloading step size is automatically reduced to 30%-50% of the original step size, and the sensor sampling frequency is increased to above 10Hz to reduce the construction risk under high uncertainty conditions.

[0053] This embodiment establishes a mapping relationship between "perception confidence and control strategy": when the uncertainty of state estimation exceeds the threshold, the unloading step size is automatically reduced and the sensor sampling frequency is increased, switching to a conservative control mode, realizing a closed-loop feedback of "perception uncertainty and adjustment of control strategy", enabling the system to have "self-protection" capability. Under high-risk working conditions with high sensor noise and strong environmental disturbance, it further reduces construction risks and forms a triple safety protection system with physical projection layer and displacement difference constraints.

[0054] Example 2: Central pillar unloading in complex foundation coal yards under non-stop production conditions This example is applied to a renovation project of a dry coal shed that had been in service for 15 years. The project required the removal of the original 12 central pillars to expand the working space. The core challenge was the uneven bearing capacity of the foundation and the requirement for the coal yard to operate normally. During the unloading process, heavy trucks continued to pass and material reclaiming operations were carried out, resulting in significant environmental disturbance.

[0055] The core technical solution of this invention is as follows: Figure 1 As shown, the system executes according to a strongly dependent, progressive logic: "Basic Environment Construction → Decision Model Modeling → Precise State Awareness → Model Error Calibration → Robust Safety Training → Real-time Closed-Loop Execution." The output of each preceding step serves as the sole input for the subsequent step. The overall system architecture is as follows: Figure 2 As shown, it consists of three parts: physical entities (sensors / jacks), digital twins (finite element simulation), and PIRL agents (actors-critic + physical projectors).

[0056] Step 1: Building a Digital Twin Virtual Simulation Environment A finite element digital twin containing 12 sets of hydraulic jacks and 2,400 rods was established. Data from 36 displacement sensors and 50 stress sensors were accessed in real time. A real-time bidirectional data transmission channel was established between the field sensors and the simulation model to achieve spatiotemporal synchronous mapping between the physical entity and the digital model, providing a unified physical benchmark for all subsequent simulations, training, and control.

[0057] Step 2: Construction of the Physical Perception Reinforcement Learning Decision Framework Based on the aforementioned digital twin virtual simulation environment, the entire process of unloading the central column is modeled as a Markov decision process in which the state dynamically evolves with the construction progress, defining the basic elements of state, action, and reward: State space: The reconstructed stress tensor, displacement vector, and uncertainty tensor output by the Bayesian graph neural network in subsequent step 3; Action space: Displacement increment of 12 sets of hydraulic jacks, in meters; Reward function: A comprehensive reward function that includes safety, smoothness, and risk mitigation, as defined in subsequent step 5.

[0058] Step 3: Robust State Awareness Based on Bayesian Graph Neural Network To address the problem of global state unobservability caused by sparse sensor data in Markov decision-making, a robust state observer is introduced. A Bayesian graph neural network is used to reconstruct the global structural state of the sparse sensor data transmitted in step 1, and at the same time, the observation uncertainty measure of each state component is output.

[0059] Specific implementation of Bayesian graphical neural networks: Graph structure construction: The nodes of the steel structure are used as vertices of the graph, and the connections between the members are used as edges of the graph. The weight of the edge is determined by the stiffness coefficient of the corresponding member. Network structure: It consists of 3 layers of spectral domain graph convolutional layers (input dimension 36→128→64→32) and 1 layer of variational inference layer. The variational inference layer uses reparameterization techniques to achieve differentiability. Output: Mean state vector and uncertainty covariance matrix ,in The diagonal elements represent the estimated variance of each state component; Training parameters: Adam optimizer, learning rate 0.001, batch size 32, number of training epochs 100, KL divergence weight 0.1, and evidence lower bound (ELBO) as the loss function.

[0060] When the sensors near column 5 are disturbed by environmental vibrations, the high uncertainty signal output by the Bayesian graph neural network will guide the AI ​​to take more conservative actions.

[0061] Step 4: Online correction of stiffness matrix based on meta-learning Based on the global state output in step 3 and the predicted state of the virtual simulation environment in step 1, a system identification loop for virtual-real residual feedback is constructed to calculate the residuals of the two in real time. . When the residual When the value exceeds 5% of the measured value, online adaptive correction of the structural stiffness matrix based on model-independent meta-learning (MAML) is initiated to solve the distortion problem between the digital twin model and the actual construction conditions.

[0062] Specific parameters for MAML stiffness correction: Inner loop learning rate outer loop learning rate ; Each trigger performs a 5-dimensional update iteration, with each correction taking approximately 50ms, meeting real-time requirements.

[0063] The effect of online correction of meta-learning stiffness matrix is ​​as follows Figure 9 As shown, a total of 12 corrections were triggered during the construction process. After the corrections, the model prediction residuals decreased from an average of 15% to less than 3%, which significantly improved the prediction accuracy of the digital twin model.

[0064] Step 5: Adversarial Robust Training and Physical Consistency Constraints Based on the decision-making framework constructed in step 2, the state space output in step 3, and the calibrated digital twin model in step 4, a robust control strategy is generated using a dual-agent adversarial training mode. The PIRL decision-making workflow is as follows: first, the AI ​​predicts the original action; then, after physical balance checks and trust region constraints, the final safe action is obtained through convex optimization projection.

[0065] The dual-agent adversarial training employs a minimax game logic: the master agent aims to minimize structural instability risk and maximize construction efficiency, while the adversarial agent aims to generate extreme load and disturbance combinations that cause local buckling of the structure (such as ±15% local load abrupt changes, uneven foundation settlement, etc.). The master agent's overall reward function is:

[0066] To address the security risks associated with random exploration in reinforcement learning, a differentiable convex optimization layer is embedded in the output layer of the reinforcement learning neural network as a physics consistency mapping architecture. This transforms the mechanical equilibrium equations and stress-deformation constraints into hard constraints on the policy output. The differentiable convex optimization layer projects the actions for safety based on a dynamic trust region. The value ranges from 5% to 20% of the displacement step of a single actuator, and is set to 10% in this embodiment.

[0067] The microconvex optimization layer uses the primal-dual interior point method to solve the motion projection problem, constructing the mechanical feasible region as a linearly constrained convex polyhedron. The single-step solution takes less than 5ms, meeting the real-time requirements of 10Hz closed-loop control.

[0068] In a virtual simulation environment, after 2 million steps of adversarial training, the master control agent learns to maintain overall steady state by actively adjusting the force flow distribution of the jack group in a game against the adversary agent. The policy convergence curve is as follows: Figure 7 As shown, the method of this invention reaches the convergence threshold (cumulative reward > 95) after about 800 episodes, while conventional RL methods require about 1500 episodes to reach the same level of convergence. This indicates that the introduction of the physical constraint layer effectively reduces the policy search space and accelerates the learning process.

[0069] Step 6: Real-time closed-loop adaptive control The safety control strategy trained in step 5 is deployed to the field control system. Combined with the global state and observation uncertainty assessment results output in real time in step 3, the unloading command is dynamically adjusted online to drive 12 sets of hydraulic jacks to complete the entire process of unloading the central column using adaptive balance control.

[0070] During the control process, the displacement difference between any two adjacent support points is kept less than a preset coordination threshold by controlling the amount of motion of the hydraulic jack. =0.5mm, ensuring uniform stress on the structure. At the same time, the control strategy is adaptively adjusted according to the observation uncertainty tensor: when the uncertainty of the state estimation exceeds the preset threshold, the unloading step size is automatically reduced to 50% of the original step size, and the sensor sampling frequency is increased to above 10Hz to reduce the construction risk under high uncertainty conditions.

[0071] On the day of construction, the intelligent agent output unloading commands at a frequency of 10Hz. In the third phase, the system detected a 0.8mm instantaneous elastic settlement in the foundation of the area near column 5 due to the passage of a heavy-duty truck. The main intelligent agent immediately identified the risk, and the physical projection architecture calculated the optimal compensation action based on the real-time calibrated stiffness matrix. It automatically reduced the unloading step size to 50% of the original size and ensured that the displacement difference between adjacent jacks was less than 0.5mm by controlling the support force of columns 4 and 6, thus successfully offsetting the secondary stress caused by nonlinear settlement.

[0072] Experimental Results and Comparative Analysis Under the same engineering conditions, the method of the present invention (method A) is compared with the human experience control method (method B) and the conventional reinforcement learning method (method C). The experimental conditions are as follows: Structural scale: A space frame structure with 2400 members. Actuating mechanism: 12 sets of hydraulic jacks Sensor configuration: 36 displacement sensors + 50 stress sensors External disturbances: Simulated environmental disturbances including heavy-duty truck traffic (approximately 20 tons axle load) and wind speed of 8 m / s. The stress fluctuation comparison curves of the three control methods are as follows: Figure 3 As shown, the method of the present invention controls the maximum stress fluctuation of the structure within 15.2 MPa throughout the unloading process, which is 64.5% lower than the manual control method (42.8 MPa). In the third stage of unloading, when the foundation settlement of 0.8 mm occurs in the area of ​​column No. 5, the method of the present invention automatically adjusts the force distribution of adjacent support points within 0.1 seconds, avoiding a sudden increase in stress.

[0073] The displacement compatibility comparison curves of the three control methods are as follows: Figure 4 As shown, the method of this invention controls the displacement difference between adjacent support points to within 0.42 mm, far below the safety threshold of 0.5 mm. In contrast, the manual control method, due to response lag, results in a peak displacement difference of 1.85 mm, posing a risk of local instability.

[0074] A bar chart comparing the performance indicators of the three control methods is shown below. Figure 5 As shown, the specific data is as follows:

[0075] To verify the contribution of each technical module, ablation experiments were conducted, and the results are as follows: Figure 6 As shown, the specific data is as follows:

[0076] Ablation experiments show that the physical projection layer is the most critical technical module, and its removal results in the most significant performance degradation (134%). The adversarial training, meta-learning correction, and uncertainty estimation modules contributed 50%, 28%, and 19% of the performance improvement, respectively.

[0077] To verify the robustness of the system under sensor noise conditions, control success rate tests were conducted at different noise levels. The results are as follows: Figure 8 As shown, the specific data is as follows:

[0078] The results show that the method of the present invention still maintains a high control success rate in high-noise environments, which is due to the uncertainty estimation and online adaptability of the Bayesian graph neural network and meta-learning.

[0079] This embodiment successfully achieved the unloading of the central column in a complex foundation coal yard under non-stop operation conditions, with zero accidents throughout the process, verifying the effectiveness and reliability of the method of the present invention. This method is also applicable to the support removal of large-span stadiums, the hydraulic jacking control of super high-rise buildings, and the closure construction control of large bridges; only the finite element model and boundary conditions need to be replaced, and the core algorithm architecture remains unchanged.

[0080] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A dynamic equilibrium control method for unloading construction of a central column based on physical constraint reinforcement learning, characterized in that, Includes the following steps: S1. Construct a digital twin virtual simulation environment based on a finite element physical engine, establish a real-time bidirectional data transmission channel between field sensors and simulation models, realize spatiotemporal synchronous mapping between physical entities and digital models, and provide a unified physical benchmark for all subsequent simulations, training and control. S2. Based on the digital twin virtual simulation environment described in step S1, the entire process of unloading the central column is modeled as a Markov decision process in which the state evolves dynamically with the construction progress. The basic elements of state, action, and reward are defined, and the basic decision framework of the physical perception reinforcement learning control system is constructed. S3. To address the global state unobservability problem caused by sparse sensor data in the Markov decision-making process described in step S2, a robust state observer is introduced. A Bayesian graph neural network is used to reconstruct the global structural state of the sparse sensor data transmitted in step S1, and at the same time, the observation uncertainty measure of each state component is output, forming a complete state space containing the reconstructed stress tensor, displacement vector and uncertainty tensor, providing input for subsequent model calibration and decision-making. S4. Based on the global state output in step S3 and the predicted state of the virtual simulation environment in step S1, construct a system identification loop for virtual and real residual feedback, calculate the residual between the two in real time, and when the residual exceeds the preset threshold, trigger the online adaptive correction of the structural stiffness matrix of the finite element model in step S1 to solve the distortion problem between the digital twin model and the actual construction conditions. S5. Based on the decision framework described in step S2, the state space described in step S3, and the calibrated digital twin model described in step S4, a robust control strategy is generated using a dual-agent adversarial training mode based on a red-blue adversarial game mechanism. At the same time, to address the security risks caused by random exploration in reinforcement learning, a differentiable convex optimization layer is embedded in the output layer of the reinforcement learning neural network as a physical consistency mapping architecture. This transforms the mechanical equilibrium equations and stress-deformation constraints into hard constraints for the strategy output, resulting in a safe control strategy that satisfies physical rules. S6. Deploy the safety control strategy trained in step S5 to the field control system. Combine the global state and observation uncertainty assessment results output in real time in step S3, dynamically adjust the unloading command online, and drive the mechanical actuator to complete the whole process of adaptive balance control for unloading the middle column.

2. The dynamic equilibrium control method for unloading construction of the middle column based on physical constraint reinforcement learning according to claim 1, characterized in that, The mechanical actuator includes a hydraulic jack, an electric jack, or an electromagnetic drive mechanism; the unloading command includes the lifting rate, output pressure value, or physical position change of the actuator.

3. The dynamic equilibrium control method for unloading construction of the middle column based on physical constraint reinforcement learning according to claim 1, characterized in that, In step S6, the amount of motion of the mechanical actuator is controlled so that the displacement difference between any two adjacent support points is always less than a preset coordination threshold. This ensures that the structure is subjected to uniform stress.

4. The dynamic equilibrium control method for unloading construction of the central column based on physical constraint reinforcement learning according to claim 1, characterized in that, In step S5, the dual-agent adversarial training adopts minimax game logic. The goal of the master control agent is to minimize the structural instability risk and maximize construction efficiency, while the goal of the adversarial agent is to generate extreme load and disturbance combinations that cause local buckling of the structure. The reward function of the master control agent includes a penalty term for the instability risk generated by the adversarial agent.

5. The dynamic equilibrium control method for unloading construction of the middle column based on physical constraint reinforcement learning according to claim 1, characterized in that, In step S3, the specific implementation of the Bayesian graph neural network is as follows: (1) Graph structure construction: The nodes of the steel structure are used as the vertices of the graph, and the connection relationship of the members is used as the edges of the graph. The weight of the edge is determined by the stiffness coefficient of the corresponding member. (2) Network structure: It consists of 3 layers of spectral domain graph convolutional layers and 1 layer of variational inference layer. The variational inference layer uses reparameterization technique to achieve differentiability. (3) Output: Mean state vector and uncertainty covariance matrix ,in The diagonal elements represent the estimated variance of each state component; (4) Training objective: The lower bound of evidence ELBO is used as the loss function, and the formula is: In the formula: The lower bound of the evidence loss function value; The expected value under the variational posterior distribution; It is a variational posterior distribution; A vector of latent variables; For observation data vectors; Let KL divergence be a metric. This represents the prior distribution of the latent variable.

6. The dynamic equilibrium control method for unloading construction of the middle column based on physical constraint reinforcement learning according to claim 1, characterized in that, In step S4, the specific implementation method of online stiffness matrix correction is as follows: (1) Triggering condition: When the residual between simulation and actual measurement Correction is initiated when the value exceeds the measured value by 5%. (2) Correction algorithm: The online adaptation method based on Model Independent Meta-Learning (MAML) is adopted, and the specific steps are as follows: a. Inner loop: Under the current stiffness parameters Calculate the virtual-real consistency loss Perform a gradient descent step to obtain temporary parameters. ,in, This represents the loss function value between simulation and actual measurement; express The measured data vector at time point; Represents the observation mapping function; express The structural state vector at time step; This represents the current stiffness parameter matrix. Denotes the square of the L2 norm. Represents the temporary stiffness parameter matrix; This represents the inner loop learning rate, which is dimensionless. Indicates the stiffness parameter The gradient operator; This represents the loss function value between simulation and actual measurement; b. Outer loop: in temporary parameters Calculate the element loss The corrected stiffness matrix is ​​obtained by updating the second-order gradient. ,in, This represents the value of the meta-loss function; express The measured data vector at time point; Represents the observation mapping function; express The structural state vector at time t, This represents the updated stiffness parameter matrix; The outer loop learning rate is dimensionless. (3) Parameter settings: Inner loop learning rate outer loop learning rate Each time it is triggered, it will be updated 3-5 times.

7. The dynamic equilibrium control method for unloading construction of the middle column based on physical constraint reinforcement learning according to claim 1, characterized in that, In step S5, the microconvex optimization layer performs a security projection on the action based on the dynamic trust region, wherein the dynamic trust region... The value ranges from 5% to 20% of the displacement step length of a single actuator, limiting the single-step movement amplitude to ensure structural stability.

8. The dynamic equilibrium control method for unloading construction of the middle column based on physical constraint reinforcement learning according to claim 4, characterized in that, The comprehensive reward function of the main control agent is as follows: In the formula, For comprehensive cumulative rewards; , , These are the weighting coefficients; This represents the maximum stress increment of the component. The allowable stress of the material; This is a motion smoothing term used to suppress the dynamic load effect caused by frequent fluctuations of the jack; To counteract the perturbation tensors generated by the agent; To perturb the search space; This is the loss function for the risk of instability.

9. The method for dynamic equilibrium control of mid-column unloading construction based on physical constraint reinforcement learning according to claim 1, characterized in that, In step S5, the microconvex optimization layer uses the primal-dual interior point method to solve the motion projection problem, constructing the mechanically feasible domain as a linearly constrained convex polyhedron. The single-step solution takes less than 5ms, meeting the real-time requirements of 10Hz closed-loop control.

10. The dynamic equilibrium control method for unloading construction of the middle column based on physical constraint reinforcement learning according to claim 1, characterized in that, In step S6, the control strategy is adaptively adjusted based on the observation uncertainty tensor output in step S3: when the uncertainty of the state estimation exceeds the preset threshold, the unloading step size is automatically reduced to 30%-50% of the original step size, and the sensor sampling frequency is increased to above 10Hz to reduce the construction risk under high uncertainty conditions.

Images