A deep foundation pit support dynamic decision method and system based on reinforcement learning

Abstract

The application discloses a deep foundation pit support dynamic decision method and system based on reinforcement learning, and relates to the technical field of reinforcement learning; the technical points are as follows: building information model data of a deep foundation pit project is acquired; a space-time observation feature matrix corresponding to a current construction stage is constructed; the comprehensive uncertainty scores of each space-time node are determined, and a weighted space-time observation feature matrix with enhanced credibility is obtained; the weighted space-time observation feature matrix with enhanced credibility is input into a decision network based on a deep deterministic policy gradient which is pre-trained, and a continuous initial support adjustment action vector is output by an executor network in the decision network; a reference yield strength limit matrix of a field support component is acquired, and a target support structure internal force adjustment vector is obtained; based on the target support structure internal force adjustment vector, corresponding field support parameter adjustment instructions are generated; and the application improves the efficiency of deep foundation pit support regulation and control under complex working conditions.

Description

Technical Field

[0001] This invention relates to the field of reinforcement learning technology, specifically to a dynamic decision-making method and system for deep foundation pit support based on reinforcement learning. Background Technology

[0002] In the field of construction engineering, especially deep foundation pit excavation and support engineering, with the increasing depth of urban underground space development and the growing complexity of the surrounding environment, the safety control and cost management of deep foundation pits face enormous challenges. One existing foundation pit support control method involves designing a static support scheme based on limited geological survey reports in the early stages of the project, and using IoT sensors to acquire stress and displacement data during excavation. When the monitoring data exceeds the set alarm threshold, remedial measures are calculated based on the experience of on-site engineers, such as increasing the axial force of the supports and densifying the anchor cables, and then performing delayed reinforcement operations. This existing control method based on static design and human experience has technical defects such as delayed decision feedback, single control dimension, and lack of adaptive evolution capability. Because deep foundation pit projects are located in complex geological environments, the mechanical properties of the soil exhibit strong nonlinear and time-varying characteristics during excavation and unloading. The initial static design cannot cover geological abrupt changes such as water inrush and soil softening during excavation. When monitoring data is abnormal, human experience cannot synthesize the states of hundreds or thousands of sensors in a very short time to perform multi-objective optimal matching solutions for global support parameters.

[0003] In recent years, some studies have attempted to introduce machine learning models to predict foundation pit deformation. However, such methods are essentially passive response modeling, only providing state warnings or trend predictions, and cannot generate support action strategies with active control capabilities. Furthermore, although reinforcement learning has shown great potential in fields such as robotics and autonomous driving due to its advantages in sequential decision-making and adaptive control, its application in high-risk civil engineering scenarios such as deep foundation pits still faces severe challenges: on the one hand, the real foundation pit environment does not allow for trial and error exploration, making it difficult to meet the needs of reinforcement learning for large amounts of interactive data; on the other hand, on-site monitoring data is often interfered with by factors such as sensor failures and geological changes, resulting in unreliable state observations, and traditional reinforcement learning algorithms are highly sensitive to such uncertainties; in addition, support actions must strictly meet structural strength and stability constraints, while the continuous actions output by standard reinforcement learning strategies may violate physical safety boundaries, posing significant risks when directly applied to engineering projects. Summary of the Invention

[0004] To achieve the above objectives, the present invention provides the following technical solution: A dynamic decision-making method for deep foundation pit support based on reinforcement learning includes: Acquire building information model data for deep foundation pit projects, as well as real-time environmental monitoring data sequences collected by IoT sensor networks deployed on-site; The spatiotemporal features of building information model data and real-time environmental monitoring data sequences are aligned to construct a spatiotemporal observation feature matrix corresponding to the current construction stage. Based on the geological stratification information, current excavation depth, and sensor fluctuation characteristics of real-time environmental monitoring data sequences in the building information model data, the comprehensive uncertainty score of each spatiotemporal node is determined, and node credibility weights are generated based on the comprehensive uncertainty scores. The spatiotemporal observation feature matrix is ​​then weighted using the node credibility weights to obtain a credibility-enhanced weighted spatiotemporal observation feature matrix. The confidence-enhanced weighted spatiotemporal observation feature matrix is ​​input into a pre-trained decision network based on deep deterministic policy gradients, and the actuator network in the decision network outputs a continuous initial support and adjustment action vector. Obtain the reference yield strength limit matrix of the on-site support components, and use the reference yield strength limit matrix to perform safety mask truncation on the initial support adjustment action vector to obtain the target support structure internal force adjustment vector; Based on the target support structure internal force adjustment vector, the corresponding on-site support parameter adjustment command is generated and sent to the servo control system on the excavation site.

[0005] Furthermore, a spatiotemporal observation feature matrix corresponding to the current construction stage is constructed, including: Obtain the three-dimensional spatial coordinate system of the foundation pit and the relative topological position coordinates of each support component from the building information model data; Extract the groundwater level monitoring values, horizontal displacement values ​​of the support piles, and axial force values ​​of the anchor cables at each time step in the real-time environmental monitoring data sequence; By using a time synchronization window, groundwater level monitoring values, horizontal displacement values ​​of support piles, and axial force values ​​of anchor cables are mapped to the corresponding relative topological coordinates, and spliced ​​together to generate a spatiotemporal observation feature matrix with a time length axis and a spatial node axis.

[0006] Furthermore, the initial support adjustment action vector is truncated using a safety mask based on the benchmark yield strength constraint matrix, including: Obtain the predicted internal force adjustment values ​​for each anchor cable and steel support in the initial support adjustment action vector; The predicted internal force adjustment values ​​are compared element by element with the corresponding safety boundary values ​​in the benchmark yield strength constraint matrix. When predicting that the internal force adjustment value is not less than the safety boundary value, the piecewise cutoff function is used to forcibly correct the value that is not less than the safety boundary value to the safety boundary value. The remaining predicted internal force adjustment values ​​are retained and recombined to generate the target support structure internal force adjustment vector.

[0007] Furthermore, the pre-trained decision network based on deep deterministic policy gradients is obtained through offline reinforcement learning methods, specifically including: Construct a digital twin agent model for deep foundation pit support and use it as an interactive environment for reinforcement learning; During the offline training phase, the actuator network is used to output exploration action vectors based on the current observation state matrix; The exploration action vector is input into the digital twin agent model of deep foundation pit support to obtain the environmental feedback reward value and the virtual next moment state matrix; The current observation state matrix, exploration action vector, environmental feedback reward value and virtual next time state matrix are combined to form a Markov experience tuple and stored in the experience replay pool. Batch data is extracted from the experience replay pool, and the node weight parameters of the executor network and the corresponding evaluator network are updated until the algorithm converges.

[0008] Furthermore, a digital twin agent model for deep foundation pit support is constructed, including: Acquire historical multi-dimensional survey data, historical monitoring evolution sequences, and corresponding historical construction action records of historical deep foundation pit projects; Historical multi-dimensional survey data and historical monitoring evolution sequences are used as input features, and the actual deformation results after the action of historical construction actions are used as supervision labels. The long short-term memory neural network model is used to perform supervised fitting training on historical data. The backpropagation algorithm is used to update the weights of the network layers, and the trained digital twin agent model for deep foundation pit support is output.

[0009] Furthermore, the determination of environmental feedback reward values ​​includes a multi-objective joint assessment of costs and safety: Obtain the expected maximum pile displacement value in the virtual next-time state matrix; Determine the difference between the expected maximum pile displacement and the design alarm limit, and construct a safety over-limit penalty item; Obtain the absolute values ​​of each numerical element in the exploration action vector, perform weighted aggregation, and construct a cost penalty term; The environmental feedback reward value is obtained by linearly combining the safety violation penalty term and the cost penalty term using a weighted coefficient matrix.

[0010] Furthermore, the node weight parameters of the actuator network and the corresponding evaluator network are updated, including: Multiple Markov experience tuples are randomly selected from the experience replay pool to construct training batches; The target Q value of this batch of data is determined using a target actuator network and a target evaluator network. The mean square error between the current Q value output by the current evaluator network and the target Q value is determined as the time difference error value; Gradient descent processing is performed based on the temporal difference error value to update the current evaluator network parameters; Determine the gradient of the current evaluator network with respect to the action, and update the parameters of the current executor network using the chain rule, in conjunction with the output gradient of the current executor network.

[0011] Furthermore, after updating the current actuator network parameters, the following also applies: Get the set soft update ratio factor; The current actuator network parameters are weighted by a soft update scaling factor, and then superimposed with the target actuator network parameters and the weight value obtained based on the difference between 1 and the soft update scaling factor to obtain new target actuator network parameters. The target evaluator network parameters are updated synchronously using an isomorphic formula to maintain the convergence stability of the reinforcement learning training process.

[0012] Furthermore, after generating the corresponding on-site support parameter adjustment instructions, it also includes: After the servo control system executes the on-site support parameter adjustment command, the actual engineering observation state matrix after a time delay is collected. The digital twin proxy model for deep foundation pit support is invoked to output the corresponding predicted state matrix; Determine the nodal residual matrix between the actual engineering observed state matrix and the predicted state matrix; The digital twin surrogate model for deep foundation pit support is fine-tuned online using the node residual matrix to eliminate environmental simulation errors caused by time-varying geological parameters.

[0013] An adaptive decision-making system for deep foundation pit support based on reinforcement learning, comprising: The multi-source data acquisition module acquires building information model data for deep foundation pit projects, as well as real-time environmental monitoring data sequences collected by the IoT sensor network deployed on site. The spatiotemporal feature alignment module aligns the building information model data with the real-time environmental monitoring data sequence to construct a spatiotemporal observation feature matrix corresponding to the current construction stage. The state credibility weighting module determines the comprehensive uncertainty score of each spatiotemporal node based on the geological stratification information, current excavation depth, and sensor fluctuation characteristics of real-time environmental monitoring data sequences in the building information model data. It then generates node credibility weights based on the comprehensive uncertainty scores and uses these node credibility weights to weight the spatiotemporal observation feature matrix, resulting in a credibility-enhanced weighted spatiotemporal observation feature matrix. The reinforcement learning decision module inputs the confidence-enhanced weighted spatiotemporal observation feature matrix into a pre-trained decision network based on deep deterministic policy gradients, and outputs continuous initial support and adjustment action vectors through the actuator network in the decision network. The safety constraint truncation module obtains the reference yield strength constraint matrix of the on-site support components, and uses the reference yield strength constraint matrix to perform safety mask truncation on the initial support adjustment action vector to obtain the target support structure internal force adjustment vector. The support command execution module generates corresponding on-site support parameter adjustment commands based on the target support structure internal force adjustment vector, and sends them to the servo control system on the excavation site.

[0014] This invention provides a dynamic decision-making method and system for deep foundation pit support based on reinforcement learning, which has the following beneficial effects: 1. To address the issues of easily interfered field monitoring data and highly uncertain geological environment, this invention integrates BIM spatial information with sensor time-series data to construct a spatiotemporally aligned observation matrix. It also introduces a node credibility weighting mechanism based on geological stratification and sensor fluctuation characteristics to effectively suppress the misleading influence of abnormal observations on decision-making. Furthermore, after the reinforcement learning strategy output, a safety mask truncation module based on the component yield strength is set up to forcibly constrain support actions to not exceed the structural bearing capacity limit, thus providing dual protection for the safe and reliable operation of the support system from both the perception and execution ends. 2. This invention relies on an offline-trained deep deterministic policy gradient decision network and a high-fidelity digital twin agent model. This invention can intelligently explore massive support strategies without the need for trial and error in a real environment. By designing a multi-objective reward function that integrates displacement limit violation penalty and adjustment cost penalty, the system is guided to automatically seek the adjustment scheme with the minimum resource consumption while meeting the safety red line, avoiding over-hardening caused by manual intervention, and taking into account both engineering safety and economy. 3. This invention establishes a data link between Building Information Modeling (BIM), IoT monitoring, and servo control equipment, enabling seamless integration from digital space to physical execution. Furthermore, by collecting the residual between the actual response after command execution and the digital twin prediction, the proxy model is fine-tuned online, continuously correcting simulation deviations caused by soil time-varying properties. This gives the system self-learning and self-adaptive capabilities during long-term operation, enabling dynamic intelligent control of deep foundation pit support. Attached Figure Description

[0015] Figure 1 This is a flowchart illustrating the steps of the present invention. Figure 2 This is a system architecture diagram of the present invention. Detailed Implementation

[0016] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0017] Example 1: Please see Figure 1 This embodiment provides a dynamic decision-making method for deep foundation pit support based on reinforcement learning, including the following steps: Step 101: Obtain the Building Information Model (BIM) data for the deep foundation pit project, as well as the real-time environmental monitoring data sequence collected by the IoT sensor network deployed on-site. Specifically, the BIM data is a three-dimensional parametric model data specifically for foundation pit construction, including the site topography, soil and rock parameters, spatial arrangement of support piles, anchor cable placement points, steel support placement elevation, and a complete set of structured modeling parameters for the foundation pit zoning excavation boundaries. The data format is the industry-standard RVT format. The on-site IoT sensor network consists of a water level transmitter, pile lateral displacement gauges, anchor cable axial force sensors, soil pore water pressure sensors, and on-site tilt sensors. All sensors are uniformly networked and connected to the edge gateway, with a 10-second single-step sampling frequency. The rate and time-series output are real-time environmental monitoring data sequences with timestamps and spatial point numbers. The real-time environmental monitoring data sequence referred to here is a multi-dimensional foundation pit working condition sensor data stream that is stored according to a fixed sampling time sequence, bound to the spatial ID of the sensor point, and has temporal continuity. For example, for a 12m deep foundation pit of a commercial and residential complex in an urban area, the RVT foundation pit model completed by the site survey can be directly imported. Displacement gauges are deployed at the corner points of the foundation pit, and water level transmitters are deployed at the underground confined aquifer. Time-series sensor data is collected throughout the entire area at a frequency of 10 seconds. It can be understood that all kinds of sensor data collected from the support points and soil and water monitoring points throughout the foundation pit can be collected into the monitoring sequence of this step. The number and location of the sensors are not limited here. Step 102: Align the building information model (BIM) data with the real-time environmental monitoring data sequence in terms of spatiotemporal features to construct a spatiotemporal observation feature matrix corresponding to the current construction stage. Specifically, using the global geodetic coordinate system of the foundation pit built into the BIM as a unified spatial reference, calibrate the spatial coordinates of all IoT sensor deployment points to eliminate offset errors between model coordinates and sensor measured coordinates. Define a single construction stage as a fixed-duration spatiotemporal window, eliminate invalid sensor data spanning excavation and support modification processes, and match the static model geotechnical support parameters and dynamic time-series monitoring data bidirectionally according to spatial nodes and time steps, arranging them in a row-column structured manner to generate a deep learning-adaptive matrix. The input format is a spatiotemporal observation feature matrix. This spatiotemporal observation feature matrix is ​​a two-dimensional structured input matrix where rows correspond to sensing space nodes, columns correspond to time-series sampling steps, and matrix elements correspond to working condition feature values. For example, the second layer of excavation in the foundation pit is considered an independent construction stage. Invalid noise data from the first layer excavation completion and the second layer anchor cable tensioning modification periods are removed. The geotechnical parameters of the pit's inner corner and central support points are matched with time-series monitoring data to generate a matrix. This can be understood as any independent excavation or support construction process in the foundation pit being able to complete matrix construction by separately defining a spatiotemporal window; the duration of a single construction stage window is not limited here. Step 103: Based on the geological stratification information, current excavation depth, and sensor fluctuation characteristics of real-time environmental monitoring data sequences in the Building Information Model (BIM) data, determine the comprehensive uncertainty score for each spatiotemporal node, and generate node credibility weights based on the comprehensive uncertainty scores. Use these node credibility weights to weight the spatiotemporal observation feature matrix to obtain a credibility-enhanced weighted spatiotemporal observation feature matrix. Specifically, retrieve the model's inherent parameters such as geological stratification thickness, soil-rock cohesion, internal friction angle, and groundwater permeability coefficient, and combine them with the real-time excavation depth to determine the soil disturbance level; use time-series indexes... The quasi-error algorithm calculates the fluctuation amplitude of single-node sensor data to characterize the data uncertainty caused by sensor drift, on-site electromagnetic interference, and transient soil disturbance. It couples the inherent geological uncertainty and sensor observation uncertainty to calculate a weighted comprehensive uncertainty score in the 0-1 interval. The higher the score, the lower the reliability of the node data. The reliability weight of the node in the 0-1 interval is obtained by normalization. The original spatiotemporal observation feature matrix is ​​weighted node by node using the matrix Hadamard product method. The feature weight of distorted data nodes is weakened, and the feature weight of stable and effective data nodes is strengthened. The output is a noise-reduced and enhanced weighted spatiotemporal observation feature matrix. The calculation process for sensor observation uncertainty is as follows: Multiple sets of time-series monitoring values ​​continuously collected from the same spatiotemporal node are selected. First, the arithmetic mean of all monitoring values ​​in the set is calculated. Then, the degree of dispersion fluctuation of all monitoring values ​​relative to the arithmetic mean is calculated to obtain the standard deviation of the time-series fluctuation. Finally, the standard deviation value is normalized and scaled to converge the value to the interval between 0 and 1, obtaining the sensor observation uncertainty score for that node. A higher score indicates greater interference in the sensor data. The calculation process for inherent geological uncertainty is as follows: Multiple sets of geotechnical test parameters from the same stratum are retrieved. First, the mean and standard deviation of the geotechnical parameters are calculated. The standard deviation is divided by the mean to obtain the coefficient of variation of the geotechnical data. The soil disturbance coefficient is matched to the real-time excavation depth of the foundation pit. The soil disturbance coefficient is then multiplied by the coefficient of variation of the geotechnical data. The inherent geological uncertainty score in the 0-1 interval is obtained. The calculation process of the comprehensive uncertainty score is as follows: the fixed coupling weight of geological uncertainty is preset to 0.6 and the fixed coupling weight of sensing uncertainty is 0.4. The two types of uncertainty scores are multiplied by their corresponding weights and then summed to obtain the global normalized comprehensive uncertainty score in the 0-1 interval. The node credibility weight conversion process is as follows: the comprehensive uncertainty score of the corresponding node is subtracted from the value 1 to obtain the node credibility weight in the 0-1 interval. The higher the comprehensive uncertainty score, the smaller the corresponding node credibility weight. The matrix weighting operation process is as follows: the generated credibility weight matrix is ​​multiplied one by one by the elements at the same position in the original spatiotemporal observation feature matrix to complete the differential weighting of working condition features and finally generate the denoised weighted spatiotemporal observation feature matrix. The comprehensive uncertainty score mentioned here is a dimensionless score that quantifies the coupling degree of three types of interference: the dispersion of stratum parameters, the randomness of excavation disturbance, and the noise of sensor acquisition. For example, the soil parameters of the silty clay interlayer in the site have large dispersion, and the sensors at the roadside points of the foundation pit are strongly affected by construction electromagnetic interference. The comprehensive uncertainty score of such points is high, and the reliability weight is reduced. When the matrix is ​​weighted, the working condition characteristics of such points are weakened. It can be understood that various interference conditions such as karst strata, construction near the street, and sudden changes in groundwater can all have their uncertainty scores adjusted differently to adapt to the changing construction environment on site. Step 104: Input the confidence-enhanced weighted spatiotemporal observation feature matrix into a pre-trained decision network based on deep deterministic policy gradient. The actuator network in the decision network outputs continuous initial support adjustment action vectors. Specifically, the deep deterministic policy gradient decision network has a dual-branch architecture, namely an actuator network and an evaluator network. The input layer size of the network matches the dimension of the weighted spatiotemporal observation feature matrix. The actuator network is equipped with three fully connected encoding layers, a ReLU activation function, and a normalized output layer. Relying on the network weights after offline convergence, after inputting the working condition feature matrix, it directly outputs continuous floating-point multidimensional adjustment values, which are combined to form the initial support adjustment vectors. The support adjustment action vector corresponds one-to-one with all adjustable anchor cable pretension, steel support axial jacking force, and support diagonal brace angle adjustment parameters in the foundation pit. The initial support adjustment action vector mentioned here is a set of continuously adjustable support internal forces output by the optimal control logic without safety constraints. For example, a rectangular foundation pit is equipped with 24 prestressed anchor cables and 12 horizontal steel supports. The decision network synchronously outputs 24 sets of anchor cable pretension and 12 sets of steel support jacking force multidimensional adjustment values, forming an integrated initial action vector. It can be understood that irregular foundation pits and foundation pits with diagonal brace combination support can be matched with the corresponding number of support components and adaptively adjust the action vector dimension. Step 105: Obtain the benchmark yield strength limit matrix of the on-site support components. Use the benchmark yield strength limit matrix to perform safety mask truncation on the initial support adjustment action vector to obtain the target support structure internal force adjustment vector. Specifically, based on structural mechanics verification standards, pre-determine the yield bearing capacity, maximum allowable internal force under working conditions, and minimum pre-tightening support lower limit value of each anchor cable and each steel support. Generate a benchmark yield strength limit matrix with upper and lower limits for the corresponding row and column layout points. Introduce a piecewise safety mask truncation operator, traverse all elements of the initial action vector, forcibly truncate and correct out-of-limit values ​​to within the safe allowable range of the components, and directly truncate compliant values. This approach retains and mitigates two risks: yielding due to overload of support internal forces and instability due to insufficient support internal forces. It outputs a target support structure internal force adjustment vector that meets structural safety specifications. The reference yield strength limit matrix mentioned here is a two-dimensional safety constraint matrix storing the upper limit of allowable internal forces and the lower limit of support pre-tightening internal forces for each support component. For example, for HRB400 material Φ15.2 prestressed anchor cables, the preset upper limit of single-strand safe internal force is 320kN, and the lower limit of stability is 80kN. Initial adjustment internal forces exceeding this range are directly truncated and corrected. This can be understood as allowing support components of different specifications and materials to independently input bearing capacity thresholds and generate a dedicated point-based safety constraint matrix. Step 106: Based on the target support structure internal force adjustment vector, generate corresponding on-site support parameter adjustment instructions and send them to the servo control system at the foundation pit site. Specifically, convert the standardized internal force adjustment values ​​within the vector into servo hydraulic pump station oil pressure control parameters, anchor cable servo tensioning stroke parameters, and steel support servo jacking displacement parameters, and encapsulate them into Modbus industrial communication instructions with point addresses, adjustment amplitudes, and execution durations. Send these instructions to the foundation pit distributed servo tensioning and jacking servo control system via the on-site industrial Ethernet to drive the support components to complete adaptive internal force control. For example, if the anchor cable on the north side of the foundation pit needs to be increased by 60kN pretension, this corresponds to an increase of 0.4MPa in servo pump station oil pressure and an increase of 12mm in tensioning stroke. Encapsulate the point-specific communication instructions and send them for execution. This can be understood as single-point fine-tuning and full-domain batch support internal force control, both of which can complete parameter conversion and instruction issuance, adapting to different control operation requirements.

[0018] In one implementation, under the condition of sudden increase in groundwater during rainstorms, the uncertainty score weight of groundwater nodes can be increased, while the feature weight of water level sensor data can be adaptively reduced to avoid decision bias caused by transient noise of water pressure. For example, under the condition of continuous rainfall during the plum rain season, the short-term fluctuation of the confined water level in the foundation pit exceeds 0.8m. The uncertainty weight of the water level measuring point can be directly increased to weaken the interference of abnormal water level data on the control of internal forces of the support. It can be understood that the weight of water level measuring points can be adjusted to optimize the decision accuracy for various sudden water level conditions such as flood season rise, pipeline leakage, and artificial precipitation.

[0019] By integrating the above technical solutions—including BIM static modeling of foundation pits, IoT dynamic sensing, node credibility weighted noise reduction, deep reinforcement learning intelligent decision-making, and structural safety mask constraints—the entire process is transformed to address the problems of traditional foundation pit support relying on experience for control and decision-making lag. This enables continuous, safe, and adaptive intelligent control of the internal forces of foundation pit support across the entire domain. For example, traditional soft soil foundation pits rely on construction technicians to manually tension anchor cables during daily inspections, resulting in a lag of 4 to 6 hours per adjustment. This solution can complete a round of adaptive fine-tuning of the entire support domain in 10 minutes, while avoiding the problem of anchor cable overload and breakage. This entire control process can be adapted to foundation pits in densely built urban areas and foundation pits near water with high soil and water pressure, improving the safety of construction management.

[0020] In some embodiments, a spatiotemporal observation feature matrix corresponding to the current construction stage is constructed, including: Step 201: Obtain the three-dimensional spatial coordinate system of the foundation pit and the relative topological position coordinates of each support component from the building information model data. Specifically, unify the independent coordinate system of the entire foundation pit project, extract the three-dimensional coordinates of the support pile center coordinates, the anchor cable anchorage endpoint coordinates, and the steel support bearing endpoint coordinates, establish the spatial topological adjacency relationship of the support components, divide the foundation pit into five types of topological partitions: north side, south side, external corner, internal corner, and central area, and bind the support topology number to which each sensor monitoring point belongs. The relative topological position coordinates mentioned here are local three-dimensional coordinates where the relative positions between support components remain unchanged after removing the global translation deviation of the foundation pit site. For example, an L-shaped irregular foundation pit is divided into two external corners, one internal corner, and two sidewall central areas. The external corner displacement sensor is bound to the external corner support pile topology number to achieve point partition binding. It can be understood that various irregular foundation pits such as polygons, circles, and combinations can be divided into topological partitions as needed to complete the sensor point affiliation binding. Step 202: Extract the groundwater level monitoring values, horizontal displacement values ​​of the support piles, and axial force values ​​of the anchor cables from each time step in the real-time environmental monitoring data sequence. Specifically, slice the original sensor data stream into 10-second fixed time steps, and filter out null values ​​and extreme noise data caused by sensor power failures and signal jumps. Retain three core working condition characteristic values ​​for each time step: compliant groundwater depth, horizontal lateral displacement value of the piles along the excavation direction of the foundation pit, and real-time axial tension value of the anchor cables. Eliminate redundant sensor parameters irrelevant to soil temperature and environmental humidity to simplify the dimensions of the time-series data. For example, if a lightning strike causes the anchor cable axial force sensor to jump to an extreme value of 900kN, the system will automatically remove the noise data in that frame, retaining only the three valid data types: compliant axial force, displacement, and water level. This can be understood as automatically filtering out various abnormal sensor data caused by construction dust, equipment electromagnetic interference, and temporary power failures, thus simplifying the valid working condition data. Step 203: Using a time synchronization window, map the groundwater level monitoring values, the horizontal displacement values ​​of the support piles, and the axial force values ​​of the anchor cables to the corresponding relative topological coordinates, and splice them to generate a spatiotemporal observation feature matrix with a time length axis and a spatial node axis. Specifically, set a five-minute sliding time synchronization window to unify the sampling time sequence deviation of different sensor models and achieve time sequence alignment of multiple types of sensor data at the same spatial node. With the spatial support node as the matrix row and the time sequence step as the matrix column, fill the matrix cells with the corresponding working condition monitoring values, and splice them to generate a spatiotemporal observation feature matrix with row and column rules and complete spatiotemporal dimensions. The matrix dimensions can be dynamically expanded as support nodes are added during foundation pit excavation. For example, the water level sensor sampling delay is 2 seconds and the displacement sensor sampling delay is 1 second. Relying on the 5-minute sliding window to unify the time sequence, after a row of support piles is added during foundation pit excavation, the matrix automatically adds the corresponding spatial node row. This can be understood as the working conditions of different brands and different response rates of sensors can all achieve time sequence alignment through the sliding window, adapting to dynamic excavation construction scenarios.

[0021] In one implementation, when the foundation pit is excavated in sections, independent time synchronization windows can be set for each section to avoid the mixing of sensor data time sequences between the new and old excavation areas. For example, if the eastern section of the foundation pit is excavated to 6m and the western section is excavated to 3m while maintaining the original soil, the two sections have independent time synchronization windows and do not mix sensor time sequence data. This can be understood as the construction conditions of the foundation pit being divided into sections or phased construction in the north and south, where time synchronization windows can be configured for each section to ensure the independence of data in each section.

[0022] Through the above technical solution, the BIM spatial topology and IoT time-series data are accurately aligned in both directions. The spatiotemporal matrix construction logic is standardized, eliminating the feature mixing problems caused by asynchronous sensor timing and mismatched point spaces, and ensuring the regularity and effectiveness of the working condition characteristics input to the decision network. For example, before alignment, the misalignment rate of the pit sidewall point data can reach 12%, while after alignment, the misalignment rate of the point time-series data is less than 2%, which greatly improves the quality of network input data. This alignment process can be adapted to scenarios such as multi-brand mixed sensor networking and the addition of sensor equipment to existing pits for renovation and expansion, thereby improving data regularity.

[0023] In some embodiments, the initial support adjustment action vector is truncated using a safety mask based on the reference yield strength constraint matrix, including: Step 301: Obtain the predicted internal force adjustment values ​​of each anchor cable and steel support in the initial support adjustment action vector; Specifically, traverse the one-dimensional initial support adjustment action vector, split and extract the predicted internal force of tension of each anchor cable and the predicted internal force of axial jacking of each steel support according to the point number, bind the unique code of the support component one by one, and establish a one-to-one mapping relationship between the action value and the support component. Step 302: Compare the predicted internal force adjustment value with the corresponding safety boundary value in the benchmark yield strength limit matrix element by element; specifically, the benchmark yield strength limit matrix has built-in double boundary values ​​of the upper limit of the maximum yield internal force and the lower limit of the minimum support stabilization internal force at the corresponding point; according to the same support point index, compare the predicted adjustment value, the upper limit value of the internal force, and the lower limit value of the internal force element by element, and divide the value into three categories: high exceeding limit, compliant range, and low exceeding limit. Step 303: When the predicted internal force adjustment value is not less than the safety boundary value, a piecewise truncation function is used to forcibly correct the value not less than the safety boundary value to the safety boundary value. Specifically, a piecewise step truncation function is used to execute mask constraints, and the correction rules are determined stepwise using pure text: the upper limit value of the preset safety internal force of a single support component and the lower limit value of the support stability internal force are retrieved, and the predicted adjustment internal force of a single point output by the network is compared. In the first case, if the predicted adjustment internal force is greater than or equal to the upper limit of the component's yield safety, the internal force at that point is directly corrected to the upper limit safety value. In the second case, if the predicted adjustment internal force is between the lower limit value and the upper limit value, the original predicted adjustment internal force is retained without modification. In the third case, if the predicted adjustment internal force is less than or equal to the lower limit value of the support stability, the internal force at that point is directly corrected to the lower limit safety value. Through piecewise determination and correction, dangerous actions exceeding limits are prevented from being output during the reinforcement learning exploration phase. Step 304: Retain the remaining predicted internal force adjustment values ​​and recombine them to generate the target support structure internal force adjustment vector; specifically, retain the compliant original adjustment values ​​within the interval, integrate the truncated and corrected boundary values, reconstruct the one-dimensional vector according to the original support point positions, and output the target support structure internal force adjustment vector that meets the structural bearing capacity specifications across the entire domain.

[0024] In one implementation, the lower limit of internal force safety can be temporarily increased under heavy rain conditions in the foundation pit to enhance the pre-tightening bearing capacity of the support and adapt to high water and soil pressure conditions.

[0025] Through the above technical solution, relying on the point-by-point matrix comparison + segmented truncation masking mechanism, a hard safety net is provided for the unconstrained exploration actions of reinforcement learning, avoiding the risks of yielding and fracture of support components and instability of foundation pit sidewalls from the source of instructions, and taking into account both the optimality of intelligent decision-making and the structural safety of the construction site.

[0026] In some embodiments, the pre-trained decision network based on deep deterministic policy gradients is obtained through offline reinforcement learning methods, specifically including: Step 401: Construct a digital twin agent model for deep foundation pit support and use it as an interactive environment for reinforcement learning. Specifically, replicate the support structure and the mechanical response mechanism of soil-water coupling at the construction site, and build a digital twin simulation environment with a one-to-one parameter mapping with the site. This environment can simulate the linkage evolution process of support internal force adjustment, soil deformation, and water level fluctuation, providing an offline interactive simulation carrier for reinforcement learning. Step 402: During the offline training phase, the actuator network is used to output exploration action vectors based on the current observation state matrix. Specifically, during the offline training phase, a Gaussian noise exploration mechanism is enabled to add controllable random noise to the output actions of the actuator network, thereby increasing the breadth of action exploration, avoiding local optimal convergence of the network, and outputting exploration action vectors with both optimal control and random exploration attributes. Step 403: Input the exploration action vector into the deep foundation pit support digital twin agent model to obtain the environmental feedback reward value and the virtual next moment state matrix; specifically, the digital twin agent model simulates and extrapolates the foundation pit soil displacement, water pressure evolution, and internal force distribution results after the support action is applied, quantifies and calculates the reward and penalty score of this action, and simultaneously extrapolates the global virtual working condition state matrix at the next sampling moment to complete the single-step environmental interaction feedback; Step 404: The current observation state matrix, exploration action vector, environmental feedback reward value, and virtual next-time state matrix are combined to form a Markov experience tuple and stored in the experience replay pool. Specifically, following the Markov temporal decision characteristics, the four data elements are bound to temporal labels and packaged into standardized experience tuples. The experience replay pool is set with a storage limit, and old and invalid samples are removed using a first-in-first-out method, leaving nearly 15,000 valid training experience tuples. Step 405: Extract batch data from the experience replay pool, update the node weight parameters of the executor network and the corresponding evaluator network until the algorithm converges; specifically, in each iteration, randomly select 64 batch experience tuples to carry out gradient iteration, optimize the weights of the two networks in both directions, and if the fluctuation amplitude of the reward value is less than 0.02 for 20 consecutive iterations, it is determined that the training of the deep deterministic strategy gradient algorithm has converged, and the network weights are sealed and put into online use in the field.

[0027] In one implementation, a priority experience replay mechanism can be added to prioritize the extraction of high-reward / penalty high-quality tuples, thereby accelerating network convergence and reducing the offline training cycle.

[0028] Through the above technical solution, relying on digital twin offline simulation training, the entire process does not require occupying the construction site for excavation and support testing, avoiding the construction safety risks brought about by on-site trial and adjustment of support, and completing the iterative convergence of reinforcement learning decision network at low cost, adapting to the decision training of foundation pits in various strata.

[0029] In some embodiments, constructing a digital twin agent model for deep foundation pit support includes: Step 501: Obtain historical multi-dimensional survey data, historical monitoring evolution sequences, and corresponding historical construction action records for historical deep foundation pit projects; specifically, the historical multi-dimensional survey data includes site geotechnical test parameters, stratum thickness, annual groundwater level amplitude, and foundation pit support design parameters; the historical monitoring evolution sequences are the time-series evolution data of pile displacement and anchor cable axial force throughout the entire construction cycle; the historical construction action records are the logbook of past manual control actions for anchor cable tensioning and steel support jacking, and the time sequences of the three types of data are matched and archived one by one; Step 502: Using historical multi-dimensional survey data and historical monitoring evolution sequences as input features, and the actual deformation results after the action of historical construction actions as supervision labels; specifically, integrating multi-dimensional parameters of soil and rock, water level, support, and monitoring to construct a high-dimensional input feature set; using the maximum deformation value of the pit sidewall and the steady-state value of internal forces 72 hours after the support action is executed as model supervision regression labels, and establishing a fitting mapping relationship between input features and engineering deformation labels; Step 503: Supervised fitting training of historical data is performed using a long short-term memory neural network model. The backpropagation algorithm is used to update the network layer weights, and the trained digital twin surrogate model of deep foundation pit support is output. Specifically, the long short-term memory neural network is equipped with a double-layer memory gating unit to adapt to the long-term water and soil deformation lag characteristics of the foundation pit. An adaptive learning rate of 0.001 is set, and the mean square error is used as the loss function. Backpropagation is used to correct the gating weights and bias parameters layer by layer. When the deformation prediction error of the verification set is less than 3%, the model is considered to have completed training and can accurately replicate the dynamic response law of the soil and support in the field.

[0030] In one implementation, geological attention mechanism modules can be superimposed on special strata such as karst and soft soil to enhance the fitting accuracy of special strata parameters and improve the accuracy of twin simulation.

[0031] Through the above technical solution, relying on the temporal neural network to replicate the mechanical response mechanism of the real foundation pit, a high-precision digital twin environment is built by data-driven approach. Compared with traditional finite element simulation, it has dynamic adaptive iteration capability, higher accuracy of temporal working condition inference, and adaptability to the high-frequency interactive iteration requirements of reinforcement learning.

[0032] In some embodiments, the determination of the environmental feedback reward value includes a joint assessment of multi-objective costs and safety: Step 601: Obtain the expected maximum pile displacement value in the virtual next moment state matrix; specifically, analyze the virtual state matrix output by the digital twin simulation, extract the extreme values ​​of lateral displacement of piles at the corner of the foundation pit and the high-risk points in the middle of the excavation, and use them as the core evaluation index of sidewall deformation after this support action. Step 602: Determine the difference between the expected maximum pile displacement value and the design alarm red line, and construct a safety limit violation penalty item; specifically, retrieve the pile displacement alarm threshold of the foundation pit construction special design. The closer the displacement is to the red line, the smaller the difference, and the larger the safety penalty item value; if the displacement is below the safe range, a positive safety reward score is generated, and if the displacement exceeds the red line, a negative penalty score is generated, constraining the network to bias safety decisions; Step 603: Obtain the absolute values ​​of each numerical element in the exploration action vector and perform weighted aggregation to construct a cost penalty term; specifically, the larger the internal force adjustment amplitude, the higher the anchor tensioning energy consumption, support component loss, and operation and maintenance cost; the weighted summation of the internal force amplitude of the action vector, the larger the internal force control amplitude, the higher the cost penalty term score, and the constraint network outputs a small energy-saving optimization control action; Step 604: Linearly combine the safety limit violation penalty item and the cost penalty item using a weighted coefficient matrix to obtain the environmental feedback reward value. Specifically, the preset safety weight coefficient is 0.75 and the cost weight coefficient is 0.25, prioritizing the safety of the foundation pit while taking into account the support, control, and maintenance costs. The pure text-based step-by-step reward and penalty fusion calculation steps are as follows: First, quantify and generate the safety limit violation reward and penalty score. The closer the foundation pit displacement is to the alarm red line, the lower the safety score. Exceeding the alarm red line is directly judged as a negative penalty score. Second, quantify and generate the cost control reward and penalty score. The greater the adjustment range of the support internal force, the higher the energy consumption and component loss, and the lower the cost reward and penalty score. Third, weightedly fuse the final reward and penalty score. Multiply the safety reward and penalty score by a weight of 0.75 and the cost reward and penalty score by a weight of 0.25. Summing the two types of scores yields the final environmental feedback reward value. A positive value is judged as an action reward, and a negative value is judged as an action penalty.

[0033] In one implementation, during the high-risk stage of the main excavation of the foundation pit, the safety weight can be increased to 0.9 to maximize the priority of ensuring the safety of the foundation pit structure construction.

[0034] Through the above technical solutions, a multi-objective reward and punishment mechanism that prioritizes safety and balances cost is constructed, changing the traditional single safety reward and punishment logic. This allows reinforcement learning decision-making to simultaneously meet the dual control objectives of foundation pit deformation safety and low-cost support and maintenance, making the decision-making engineering more practical.

[0035] In some embodiments, updating the node weight parameters of the actuator network and the corresponding evaluator network includes: Step 701: Randomly select multiple Markov experience tuples from the experience replay pool to construct training batches; specifically, select 64 time-series independent experience tuples in a single batch to ensure that the batch samples cover water level rise, soil creep, and conventional steady-state multi-dimensional foundation pit conditions, and avoid the network biased fitting due to single samples. Step 702: Determine the target Q value of this batch of data using the target actuator network and the target evaluator network; specifically, the target dual network is an offline fixed weight network that couples the optimal action at the next moment with the optimal target Q value of the environmental reward deduction time sequence as the benchmark value for network iterative fitting. Step 703: Determine the mean square error between the current Q value output by the current evaluator network and the target Q value, as the time-series difference error value; specifically, the pure text step-by-step error calculation steps are as follows: extract the network real-time estimate and network benchmark estimate corresponding to all samples in this batch, calculate the difference between the two types of estimates for each group of samples, square all the differences and then calculate the overall average value, which is the time-series difference error; the larger the error value, the lower the accuracy of the evaluator network in judging the value of the foundation pit working condition; Step 704: Perform gradient descent processing based on the temporal difference error value to update the current evaluator network parameters; specifically, an adaptive moment estimation optimizer is used to reduce the weights of the fully connected layer nodes of the evaluator along the reverse gradient direction of the error, thereby reducing the estimation bias and improving the evaluator's ability to distinguish the working condition value. Step 705: Determine the gradient of the current evaluator network with respect to the action, and update the parameters of the current actuator network by combining the output gradient of the current actuator network with the chain rule; specifically, relying on the chain rule of deep learning, link the bidirectional transmission of value gradient and action gradient, iteratively optimize the actuator action output logic, and improve the rationality of the support adaptive action output.

[0036] In one implementation, a gradient clipping mechanism is added to the iterative process to limit gradient extrema and prevent gradient explosion and divergence during network training. Through the above technical solution, following the deep deterministic strategy gradient standard temporal differential iteration logic, dual-network division of labor iterative optimization, the evaluator judges the value of the working condition and the actuator optimizes the support action, and steadily improves the network's adaptive decision-making ability.

[0037] In some embodiments, after updating the current actuator network parameters, the method further includes: Step 801: Obtain the set soft update ratio factor; specifically, the soft update ratio factor is fixed at 0.005, which is the optimal steady-state coefficient for industry-specific reinforcement learning training. A smaller value ensures smooth iteration of the target network and no parameter mutations. Step 802: Weight the current actuator network parameters using the soft update scaling factor, and then superimpose them with the target actuator network parameters and the weight value obtained based on the difference between 1 and the soft update scaling factor to obtain new target actuator network parameters; Specific textual step-by-step update steps: The preset fixed soft update coefficient is 0.005. First, multiply all the parameters of the actuator network completed in the current iteration by 0.005, and then multiply the original stock target actuator network parameters by 0.995. The two sets of weighted network parameters are superimposed and integrated one by one to generate the target actuator network parameters after the new iteration, so as to achieve small and stable iteration of network parameters and avoid parameter mutations; Step 803: Use isomorphic formulas to synchronously update the target evaluator network parameters to maintain the convergence stability of the reinforcement learning training process; specifically, reuse the same soft update coefficients and the same weighted iteration logic to synchronously complete the iteration of the evaluator target network parameters, ensuring that the update rhythm of the execution and evaluation dual target networks is coordinated, and eliminating training oscillations and convergence problems caused by asynchronous updates of network parameters.

[0038] In one implementation, during the training phase for complex and irregularly shaped foundation pits, the soft update factor can be lowered to 0.002 to further slow down the update speed of the target network and improve training stability.

[0039] By adopting the above technical solution, a soft update mechanism is used to replace the hard assignment update, which avoids sudden oscillations in the target network parameters, ensures the smooth convergence of long-term foundation pit reinforcement learning training, and improves the stability of offline training and online implementation decision-making.

[0040] In some embodiments, after generating the corresponding on-site support parameter adjustment instruction, the method further includes: Step 901: After the servo control system executes the on-site support parameter adjustment command, the actual engineering observation state matrix after time delay is collected; specifically, after the support servo is completed, steady-state sensor data is collected after a 30-minute delay to avoid the interference of instantaneous fluctuations in the support internal force, and steady-state data of soil deformation, water level, and internal force are collected to construct the on-site real post-engineering observation state matrix. Step 902: Call the deep foundation pit support digital twin agent model to output the corresponding prediction state matrix; specifically, input the support control action into the twin model, deduce the foundation pit simulation steady-state working condition under synchronous duration, and output the prediction state matrix with the same sequence and dimension. Step 903: Determine the node residual matrix between the actual engineering observation state matrix and the predicted state matrix; specifically, the pure text residual calculation steps are as follows: compare the field measured working condition matrix and the twin simulation predicted working condition matrix, perform numerical difference calculations one by one according to the same spatial sensing points and the same time sequence nodes, and take the absolute value of the difference. After the global calculation is completed, the node residual matrix is ​​generated; the larger the cell value in the matrix, the greater the twin simulation deviation caused by soil weathering and water disturbance. Step 904: Utilize the node residual matrix to perform online fine-tuning training on the digital twin surrogate model for deep foundation pit support, in order to eliminate environmental simulation errors caused by time-varying geological parameters. Specifically, the calculation and iteration steps for the fine-tuning loss of the pure text model are as follows: First, statistically analyze the values ​​of all cells in the residual matrix and calculate the arithmetic mean of the entire domain, setting this average value as the online fine-tuning loss value of the model; Second, fix the fine-tuning learning step size and use the loss value to reverse-correct the internal weights and bias parameters of the long short-term memory network; Third, iteratively reduce the deviation between the simulated values ​​and the measured values ​​on site, adapt to the time-series deterioration characteristics of soil and rock parameters, and effectively optimize the simulation accuracy of the twin model in the long term.

[0041] In one implementation, the seasonal rise and fall cycle of groundwater can be adjusted to increase the frequency of online twin fine-tuning, thus adapting to the seasonal time-varying characteristics of the water and soil environment.

[0042] By using the above technical solutions, a closed-loop link is established for on-site execution, residual verification, and twin fine-tuning, which solves the industry pain point of static twin model simulation distortion caused by the evolution of soil and rock parameters in foundation pits with excavation and weathering time, and effectively ensures the accuracy of online decision-making in the long term.

[0043] Example 2: Based on Example 1, this embodiment also provides an adaptive decision-making system for deep foundation pit support based on reinforcement learning, including: a multi-source data acquisition module, which acquires building information model data of deep foundation pit projects and real-time environmental monitoring data sequences collected by an IoT sensor network deployed on site; specifically, the module has a built-in BIM data parsing subunit and an IoT edge data access subunit, supports RVT model format parsing and multi-protocol sensor data pass-through, and has functions for preliminary noise filtering and data time-series archiving, and collects two types of data sources for foundation pit static modeling and dynamic perception in the whole domain; The spatiotemporal feature alignment module aligns the building information model data with the real-time environmental monitoring data sequence to construct a spatiotemporal observation feature matrix corresponding to the current construction stage. Specifically, the module has built-in coordinate registration operators and time-series synchronous sliding window operators to unify the spatiotemporal benchmark, complete the two-way binding of support points and sensor data, and output a standardized deep learning compliant two-dimensional spatiotemporal observation feature matrix. The state credibility weighting module determines the comprehensive uncertainty score of each spatiotemporal node based on geological stratification information, current excavation depth, and sensor fluctuation characteristics of real-time environmental monitoring data sequences in the building information model data. It then generates node credibility weights based on these comprehensive uncertainty scores and uses these node credibility weights to weight the spatiotemporal observation feature matrix, resulting in a credibility-enhanced weighted spatiotemporal observation feature matrix. Specifically, the module incorporates an uncertainty quantification algorithm and a matrix weighting operation unit to quantify geological and sensor interference, differentiate and weaken distorted node features, and optimize the quality of input decision network working condition features. The reinforcement learning decision-making module inputs the weighted spatiotemporal observation feature matrix with enhanced credibility into a pre-trained decision network based on deep deterministic policy gradients. The actuator network in the decision network outputs continuous initial support adjustment action vectors. Specifically, the module is equipped with an offline convergent dual-branch decision network, which supports forward inference of working condition features and can output continuous internal force adjustment actions of anchor cables and steel supports in multiple dimensions simultaneously to meet the needs of full-domain support linkage control. The safety constraint truncation module obtains the benchmark yield strength limit matrix of the on-site support components, and uses the benchmark yield strength limit matrix to perform safety mask truncation on the initial support adjustment action vector to obtain the target support structure internal force adjustment vector. Specifically, the module has a built-in structural safety database, a segmented mask truncation operator, pre-stores the yield bearing capacity parameters of all support points, and hard-truncates over-limit control actions to ensure the structural safety of the construction site. The support command execution module generates corresponding on-site support parameter adjustment commands based on the internal force adjustment vector of the target support structure, and sends them to the servo control system on the excavation site. Specifically, the module has a built-in industrial protocol conversion unit to complete the conversion of decision vectors into servo hydraulic and tension stroke electronic control commands. It performs on-site execution under wired and wireless dual links, and collects post-execution sensor data to feed back to the twin model to complete closed-loop fine-tuning. In one implementation, the system is equipped with a local power failure cache unit, which retains network weights, operating condition matrix, and historical decision logs after a power failure, and quickly resumes decision-making operations after power is restored, adapting to unstable power supply conditions at construction sites.

[0044] Through the above technical solutions, multi-source data collection of the foundation pit is achieved through BIM modeling and IoT sensing. A weighted mechanism for uncertainty and reliability of geological sensing coupling is added to weaken the negative impact of on-site interference data on decision-making. The convergence of the support decision-making network is completed by relying on digital twin offline reinforcement learning to avoid the risks of on-site commissioning and construction. Combined with the yield strength safety mask truncation of support components, the optimality of intelligent decision-making and the safety of the foundation pit structure are both taken into account. At the same time, a closed loop of on-site execution residuals is built to fine-tune the twin model online to adapt to the temporal deterioration characteristics of soil and rock. In summary, the present invention differs from the current situation of deep foundation pit support relying on manual experience and decision lag, improves the intelligence and adaptability of deep foundation pit support control, takes into account the safety of foundation pit construction and the cost of support operation and maintenance management, and is suitable for long-term construction support decision-making operations in various complex sites.

[0045] The above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any other combination thereof. When implemented in software, the above embodiments can be implemented, in whole or in part, as a computer program product. Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution.

[0046] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0047] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

Claims

1. A dynamic decision-making method for deep foundation pit support based on reinforcement learning, characterized in that, The method includes: Acquire building information model data for deep foundation pit projects, as well as real-time environmental monitoring data sequences collected by IoT sensor networks deployed on-site; The spatiotemporal features of building information model data and real-time environmental monitoring data sequences are aligned to construct a spatiotemporal observation feature matrix corresponding to the current construction stage. Based on the geological stratification information, current excavation depth, and sensor fluctuation characteristics of real-time environmental monitoring data sequences in the building information model data, the comprehensive uncertainty score of each spatiotemporal node is determined, and node credibility weights are generated based on the comprehensive uncertainty scores. The spatiotemporal observation feature matrix is ​​then weighted using the node credibility weights to obtain a credibility-enhanced weighted spatiotemporal observation feature matrix. The confidence-enhanced weighted spatiotemporal observation feature matrix is ​​input into a pre-trained decision network based on deep deterministic policy gradients, and the actuator network in the decision network outputs a continuous initial support and adjustment action vector. Obtain the reference yield strength limit matrix of the on-site support components, and use the reference yield strength limit matrix to perform safety mask truncation on the initial support adjustment action vector to obtain the target support structure internal force adjustment vector; Based on the target support structure internal force adjustment vector, the corresponding on-site support parameter adjustment command is generated and sent to the servo control system on the excavation site.

2. The dynamic decision-making method for deep foundation pit support based on reinforcement learning according to claim 1, characterized in that, Construct a spatiotemporal observation feature matrix corresponding to the current construction stage, including: Obtain the three-dimensional spatial coordinate system of the foundation pit and the relative topological position coordinates of each support component from the building information model data; Extract the groundwater level monitoring values, horizontal displacement values ​​of the support piles, and axial force values ​​of the anchor cables at each time step in the real-time environmental monitoring data sequence; By using a time synchronization window, groundwater level monitoring values, horizontal displacement values ​​of support piles, and axial force values ​​of anchor cables are mapped to the corresponding relative topological coordinates, and spliced ​​together to generate a spatiotemporal observation feature matrix with a time length axis and a spatial node axis.

3. The dynamic decision-making method for deep foundation pit support based on reinforcement learning according to claim 1, characterized in that, The initial support adjustment action vector is truncated using a safety mask based on the benchmark yield strength constraint matrix, including: Obtain the predicted internal force adjustment values ​​for each anchor cable and steel support in the initial support adjustment action vector; The predicted internal force adjustment values ​​are compared element by element with the corresponding safety boundary values ​​in the benchmark yield strength constraint matrix. When predicting that the internal force adjustment value is not less than the safety boundary value, the piecewise cutoff function is used to forcibly correct the value that is not less than the safety boundary value to the safety boundary value. The remaining predicted internal force adjustment values ​​are retained and recombined to generate the target support structure internal force adjustment vector.

4. The dynamic decision-making method for deep foundation pit support based on reinforcement learning according to claim 1, characterized in that, The pre-trained decision network based on deep deterministic policy gradients is obtained through offline reinforcement learning methods, specifically including: Construct a digital twin agent model for deep foundation pit support and use it as an interactive environment for reinforcement learning; During the offline training phase, the actuator network is used to output exploration action vectors based on the current observation state matrix; The exploration action vector is input into the digital twin agent model of deep foundation pit support to obtain the environmental feedback reward value and the virtual next moment state matrix; The current observation state matrix, exploration action vector, environmental feedback reward value and virtual next time state matrix are combined to form a Markov experience tuple and stored in the experience replay pool. Batch data is extracted from the experience replay pool, and the node weight parameters of the executor network and the corresponding evaluator network are updated until the algorithm converges.

5. The dynamic decision-making method for deep foundation pit support based on reinforcement learning according to claim 4, characterized in that, Constructing a digital twin agent model for deep foundation pit support, including: Acquire historical multi-dimensional survey data, historical monitoring evolution sequences, and corresponding historical construction action records of historical deep foundation pit projects; Historical multi-dimensional survey data and historical monitoring evolution sequences are used as input features, and the actual deformation results after the action of historical construction actions are used as supervision labels. The long short-term memory neural network model is used to perform supervised fitting training on historical data. The backpropagation algorithm is used to update the weights of the network layers, and the trained digital twin agent model for deep foundation pit support is output.

6. The dynamic decision-making method for deep foundation pit support based on reinforcement learning according to claim 4, characterized in that, The determination of environmental feedback reward values ​​includes a multi-objective joint assessment of costs and safety: Obtain the expected maximum pile displacement value in the virtual next-time state matrix; Determine the difference between the expected maximum pile displacement and the design alarm limit, and construct a safety over-limit penalty item; Obtain the absolute values ​​of each numerical element in the exploration action vector, perform weighted aggregation, and construct a cost penalty term; The environmental feedback reward value is obtained by linearly combining the safety violation penalty term and the cost penalty term using a weighted coefficient matrix.

7. The dynamic decision-making method for deep foundation pit support based on reinforcement learning according to claim 4, characterized in that, Update the node weight parameters of the actuator network and the corresponding evaluator network, including: Multiple Markov experience tuples are randomly selected from the experience replay pool to construct training batches; The target Q value of this batch of data is determined using a target actuator network and a target evaluator network. The mean square error between the current Q value output by the current evaluator network and the target Q value is determined as the time difference error value; Gradient descent processing is performed based on the temporal difference error value to update the current evaluator network parameters; Determine the gradient of the current evaluator network with respect to the action, and update the parameters of the current executor network using the chain rule, in conjunction with the output gradient of the current executor network.

8. The dynamic decision-making method for deep foundation pit support based on reinforcement learning according to claim 7, characterized in that, After updating the current executor network parameters, the following also applies: Get the set soft update ratio factor; The current actuator network parameters are weighted by a soft update scaling factor, and then superimposed with the target actuator network parameters and the weight value obtained based on the difference between 1 and the soft update scaling factor to obtain new target actuator network parameters. The target evaluator network parameters are updated synchronously using an isomorphic formula to maintain the convergence stability of the reinforcement learning training process.

9. A dynamic decision-making method for deep foundation pit support based on reinforcement learning according to claim 1, characterized in that, After generating the corresponding on-site support parameter adjustment instructions, the following is also included: After the servo control system executes the on-site support parameter adjustment command, the actual engineering observation state matrix after a time delay is collected. The digital twin proxy model for deep foundation pit support is invoked to output the corresponding predicted state matrix; Determine the nodal residual matrix between the actual engineering observed state matrix and the predicted state matrix; The digital twin surrogate model for deep foundation pit support is fine-tuned online using the node residual matrix to eliminate environmental simulation errors caused by time-varying geological parameters.

10. An adaptive decision-making system for deep foundation pit support based on reinforcement learning, characterized in that, For implementing the reinforcement learning-based dynamic decision-making method for deep foundation pit support according to any one of claims 1-9, the system comprises: The multi-source data acquisition module acquires building information model data for deep foundation pit projects, as well as real-time environmental monitoring data sequences collected by the IoT sensor network deployed on site. The spatiotemporal feature alignment module aligns the building information model data with the real-time environmental monitoring data sequence to construct a spatiotemporal observation feature matrix corresponding to the current construction stage. The state credibility weighting module determines the comprehensive uncertainty score of each spatiotemporal node based on the geological stratification information, current excavation depth, and sensor fluctuation characteristics of real-time environmental monitoring data sequences in the building information model data. It then generates node credibility weights based on the comprehensive uncertainty scores and uses these node credibility weights to weight the spatiotemporal observation feature matrix, resulting in a credibility-enhanced weighted spatiotemporal observation feature matrix. The reinforcement learning decision module inputs the confidence-enhanced weighted spatiotemporal observation feature matrix into a pre-trained decision network based on deep deterministic policy gradients, and outputs continuous initial support and adjustment action vectors through the actuator network in the decision network. The safety constraint truncation module obtains the reference yield strength constraint matrix of the on-site support components, and uses the reference yield strength constraint matrix to perform safety mask truncation on the initial support adjustment action vector to obtain the target support structure internal force adjustment vector. The support command execution module generates corresponding on-site support parameter adjustment commands based on the target support structure internal force adjustment vector, and sends them to the servo control system on the excavation site.

Images