A real-time deviation correction control system for automatic pile placement on soft foundation

Abstract

The present invention relates to the field of civil engineering technology and discloses a real-time correction control system for pile positions in soft foundation automatic pile layout. The system integrates three modules: pile position status monitoring, correction model construction, and control execution, enabling real-time monitoring of pile position, inclination angle, and soil parameters. By constructing a correction model for dynamic optimization problems and combining it with a reinforcement learning algorithm, the system can automatically select the optimal correction action, achieving intelligent decision-making and dynamic adjustment. The implementation of the present invention greatly reduces manual intervention and the uncertainty of relying on operator experience, providing an efficient and accurate correction solution for soft foundation pile layout construction.

Description

Technical Field

[0001] The invention relates to the technical field of civil engineering, in particular to a soft foundation automatic pile arrangement and pile position real-time deviation correction control system. Background Art

[0002] In the field of civil engineering, pile construction on soft soil foundations is a complex and critical task. Due to the low bearing capacity, high compressibility, and volatility of soft soil, pile positions are prone to shifting during construction, making it difficult to ensure the verticality and position accuracy of the piles. Existing pile position correction technologies mostly use post-detection and adjustment methods. That is, after construction is completed, the offset of the pile position is detected by measuring equipment, and then manual or mechanical adjustments are made based on the detection results. This method has many shortcomings: First, post-detection cannot promptly detect and correct pile position offsets during construction, which may lead to a decrease in the bearing capacity of the piles and even cause project quality problems; second, manual adjustments rely on the operator's experience, making it difficult to ensure the accuracy and efficiency of the adjustments; finally, traditional correction methods lack intelligent decision-making support and cannot dynamically adjust the correction plan based on the pile position status and soil conditions.

[0003] Furthermore, changes in soil parameters significantly influence pile misalignment, but existing correction technologies often neglect the real-time monitoring and utilization of key parameters such as soil hardness, moisture, and bearing capacity. This lack of accurate knowledge of these parameters makes the formulation of correction strategies lacking a scientific basis, making it difficult to achieve ideal correction results. Summary of the Invention

[0004] The object of the present invention is to provide a real-time deviation correction control system for automatic pile placement on soft foundation, so as to solve the problems raised in the above-mentioned background technology.

[0005] To achieve the above object, the present invention provides the following technical solution: a real-time deviation correction control system for automatic pile placement on soft foundation, the system comprising:

[0006] Pile status monitoring module, used to monitor pile position, tilt angle and soil parameters in real time;

[0007] A correction model construction module is used to construct a correction strategy model. The correction strategy model models the pile placement process as a dynamic optimization problem and dynamically adjusts the pile position correction plan through an algorithm. This module further includes:

[0008] State definition unit, used to describe the state space of the pile position at the current moment;

[0009] Action definition unit, used to define the action space for correction;

[0010] The feedback evaluation unit is used to define the evaluation function and quantitatively evaluate the results of the correction action based on the accuracy, efficiency, resource consumption and safety performance of the pile position correction;

[0011] The strategy optimization unit uses a reinforcement learning algorithm to select the optimal corrective action under a given state. Through continuous iterative learning, it optimizes the correction strategy to maximize the long-term correction effect.

[0012] The control system execution module is used to input the data obtained by the pile position status monitoring module into the correction strategy model, receive the correction strategy output by the model, and control the pile driver and other auxiliary equipment to perform corresponding correction actions.

[0013] Preferably, the soil parameters include soil hardness, moisture and bearing capacity.

[0014] Preferably, the state space includes pile position, pile inclination angle, soil parameters and current correction action.

[0015] Preferably, the action space includes adjusting the position of the pile driver, changing the pile driving force, using an auxiliary correction device and notifying the operator to intervene.

[0016] Preferably, the feedback evaluation unit is implemented as follows:

[0017] An evaluation function F(s,a) is defined, where s represents the state space of the pile position at the current moment; a represents the corrective action taken, which is selected from the corrective action space and includes adjusting the position of the pile driver, changing the driving force, using auxiliary corrective devices, and notifying the operator to intervene. The evaluation function F(s,a) performs a weighted summation based on the pile position correction accuracy A(s,a), correction efficiency E(s,a), resource consumption R(s,a), and safety performance S(s,a) to obtain a quantitative evaluation value of the corrective action result. The specific algorithm formula is:

[0018] F(s,a)=w1*A(s,a)+w2*E(s,a)-w3*R(s,a)+w4*S(s,a)

[0019] Among them, w1, w2, w3, and w4 are the weight coefficients of accuracy, correction efficiency, resource consumption, and safety performance, respectively, and w1+w2+w3+w4=1; A(s,a) represents the pile position correction accuracy after taking action a in state s, which is obtained by calculating the deviation between the correction and the expected position; E(s,a) represents the correction efficiency, which is obtained by calculating the time required for correction; R(s,a) represents resource consumption, including energy consumption, material consumption, and labor cost; S(s,a) represents safety performance, which is obtained by evaluating the safety risks generated during the correction process.

[0020] Preferably, the strategy optimization unit uses the Q-learning algorithm in the reinforcement learning algorithm to train the correction strategy model.

[0021] Preferably, the step of the strategy optimization unit training the correction strategy model includes:

[0022] Step 1: Initialize the Q-table. The Q-table is a two-dimensional array whose rows represent the pile state s and columns represent the corrective action a. Each element Q(s,a) is initialized to 0, indicating that the expected reward value of taking action a in state s is 0.

[0023] Step 2: Set the learning rate α, discount factor γ, and exploration rate ε, where α controls the learning speed, γ controls the discounted value of future returns, and ε controls the balance between exploration and exploitation.

[0024] Step 3: For each training round:

[0025] Step 301: Initialize the pile position state s to the current real-time monitored pile position state;

[0026] Step 302: Select a corrective action a based on the current state s and the ε-greedy strategy. Randomly select an action with a probability of ε, and select the action with the largest current Q value with a probability of 1-ε.

[0027] Step 303: Execute action a, observe the next state s' and the immediate reward r, where the immediate reward r is calculated based on the evaluation function F(s,a) of the feedback evaluation unit;

[0028] Step 304: Update the Q-table value according to the update formula of the Q-learning algorithm. The update formula is:

[0029] Q(s,a)=Q(s,a)+α*[r+γ*max_a'Q(s',a')-Q(s,a)]

[0030] Among them, max_a'Q(s',a') represents the maximum Q value of all possible actions in the next state s';

[0031] Step 4: Repeat step 3 until the preset number of training rounds is reached or the value of the Q table converges;

[0032] Step 5: After the training is completed, the obtained Q table is used as the correction strategy model. For any given pile position state s, the action a with the largest Q value is selected as the optimal correction action by searching the Q table.

[0033] Preferably, a simulated data set is generated as training data for the correction strategy model, specifically by:

[0034] For each simulated construction scenario, the changes in pile position status and the execution of corrective actions are simulated. The state s, the action a, and the immediate reward r given according to the preset evaluation criteria are recorded at each step to form a state-action-reward triple (s, a, r).

[0035] Summarize the state-action-reward triples generated by all simulation scenarios to form a simulation dataset;

[0036] The data in the simulated dataset is further encoded and standardized, and the processed simulated dataset is used as training data for the correction strategy model.

[0037] Preferably, the method of encoding the data in the simulated data set includes: performing one-hot encoding conversion on discrete features.

[0038] Preferably, the method of standardizing the data in the simulated data set includes: using the Z-score standardization method to standardize the continuous features, specifically: subtracting the mean of each numerical data and dividing it by its standard deviation so that the processed data conforms to the standard normal distribution. The standardization formula is: Z = (X-μ) / σ, where X is the original data, μ is the mean of the original data, σ is the standard deviation of the original data, and Z is the standardized data.

[0039] Compared with the prior art, the present invention has the following beneficial effects:

[0040] The correction model construction module models the pile placement process as a dynamic optimization problem and, combined with a reinforcement learning algorithm, continuously optimizes the correction strategy within the strategy optimization unit. This invention automatically selects the optimal correction action based on the current pile position, soil conditions, and preset pile placement requirements. This enables intelligent decision-making and dynamic adjustment of correction strategies, improving the accuracy and efficiency of correction while significantly reducing the uncertainty associated with manual intervention and reliance on operator experience.

[0041] The present invention provides a more scientific basis for the formulation of correction strategies by real-time monitoring of key parameters such as soil hardness, moisture and bearing capacity, making the correction action more accurate and effective, and better able to adapt to the pile layout requirements under different soil conditions. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 This is a working principle diagram of a real-time deviation correction control system for automatic pile placement on soft foundations according to the present invention;

[0043] Figure 2 This is a schematic diagram of the design of the evaluation function in the feedback evaluation unit;

[0044] Figure 3 This is the training flowchart of the correction strategy model. DETAILED DESCRIPTION

[0045] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0046] See also Figure 1-3 The present invention provides a technical solution: a real-time deviation correction control system for automatic pile placement on soft foundation, the system comprising:

[0047] The pile status monitoring module is responsible for real-time monitoring of pile position, inclination, and soil parameters. By integrating high-precision sensors and testing equipment, such as GPS locators, inclination sensors, and soil parameter measuring instruments, this module collects real-time data related to pile positions. This data includes the precise coordinates of the pile position, the degree of inclination, and soil parameters such as hardness, moisture, and bearing capacity. The monitoring module transmits this data in real time to other parts of the control system, providing basic information for the formulation of subsequent correction strategies.

[0048] The correction model construction module is responsible for building the correction strategy model and dynamically adjusting the pile position correction plan based on the monitored pile position status and preset pile layout requirements. This module contains the following units:

[0049] State definition unit: This unit is used to describe the state space of the pile position at the current moment. The state space includes pile position, pile tilt angle, soil parameters, and the corrective action currently being performed. These state variables are quantified into specific values ​​or categories.

[0050] Action Definition Unit: This unit defines the corrective action space, that is, the corrective measures that the system can take. The action space includes adjusting the pile driver position, changing the pile driving force, using auxiliary correction devices, and notifying the operator to intervene.

[0051] Feedback Evaluation Unit: This unit incorporates a set of evaluation functions to quantitatively assess the results of correction actions. These evaluation functions comprehensively consider multiple aspects, including pile position correction accuracy, efficiency, resource consumption, and safety performance, providing the system with a standard for evaluating correction effectiveness.

[0052] Policy Optimization Unit: This unit uses a reinforcement learning algorithm to select the optimal corrective action for a given state based on the evaluation results provided by the Feedback Evaluation Unit. Through continuous iterative learning, the Policy Optimization Unit gradually optimizes the corrective strategy to maximize long-term corrective effectiveness. The reinforcement learning algorithm enables the system to gradually master the optimal corrective strategy through continuous trial and error.

[0053] The control system execution module is responsible for transmitting data acquired by the pile position status monitoring module to the correction strategy model and receiving the correction strategy output by the model. This module controls the pile driver and other auxiliary equipment to perform the corresponding corrective actions based on the correction strategy. For example, when the correction strategy requires adjusting the pile driver's position, the control system execution module issues a command to the pile driver to move to the specified position; when the pile driving force needs to be changed, the control system adjusts the pile driver's impact force parameters. By precisely controlling the actions of the pile driver and auxiliary equipment, the system can achieve accurate pile position correction.

[0054] The present invention will be further described below in conjunction with Examples 1 to 3:

[0055] Example 1:

[0056] In implementing the feedback evaluation unit, a comprehensive evaluation function F(s,a) was designed to quantitatively evaluate the corrective action a taken under a specific state s. This function comprehensively considers the accuracy, efficiency, resource consumption, and safety of pile position correction to ensure the comprehensiveness and optimization of the correction strategy.

[0057] Definition of the evaluation function F(s,a):

[0058] The evaluation function F(s,a) uses a weighted summation method to combine the four evaluation indicators of pile position correction accuracy A(s,a), correction efficiency E(s,a), resource consumption R(s,a) and safety performance S(s,a) to form a comprehensive quantitative evaluation value. The specific algorithm formula is:

[0059] F(s,a)=w1*A(s,a)+w2*E(s,a)-w3*R(s,a)+w4*S(s,a)

[0060] Where w1, w2, w3, and w4 are the weight coefficients of the four evaluation indicators, respectively, and they satisfy w1 + w2 + w3 + w4 = 1. These weight coefficients are set based on the actual application scenario and importance to reflect the relative importance of different indicators in the correction decision.

[0061] Calculation of evaluation indicators:

[0062] (1) Accuracy A(s,a): After taking action a in state s, the accuracy of the correction is evaluated by calculating the deviation between the corrected pile position and the expected position. The smaller the deviation, the higher the accuracy, and the larger the value of A(s,a). For example, metrics such as Euclidean distance or Manhattan distance can be used to calculate the deviation.

[0063] (2) Correction efficiency E(s,a): Correction efficiency is evaluated by calculating the time required for correction. The shorter the time, the higher the efficiency, and the larger the value of E(s,a). In practical applications, the time required for correction can be calculated by recording the timestamps of the start and end of the correction action.

[0064] (3) Resource consumption R(s,a): Resource consumption includes energy consumption, material consumption, and labor costs. After taking action a in state s, resource consumption is evaluated by calculating the actual amount of resources consumed. The lower the resource consumption, the smaller the value of R(s,a). (Note that the negative sign before R(s,a) in the formula indicates that resource consumption is a deduction from the evaluation value.)

[0065] (4) Safety performance S(s,a): Safety performance is quantified by evaluating the safety risks generated during the correction process. The lower the safety risk, the larger the value of S(s,a). In practical applications, safety risks can be assessed based on the risks of equipment damage, casualties, and other risks that may result from the correction action.

[0066] For example, consider the corrective action decisions during a pile placement operation. Assuming the current pile position s is (position deviation 5 cm, tilt angle 2 degrees, medium soil hardness, moderate moisture), the possible corrective actions a include adjusting the pile driver position, changing the driving force, and using an auxiliary corrective device. Based on historical data and experience, the weight coefficients are set to w1 = 0.4, w2 = 0.3, w3 = 0.2, and w4 = 0.1.

[0067] For each corrective action, we calculate its accuracy, correction efficiency, resource consumption, and safety performance, and then substitute these values ​​into the evaluation function F(s, a). Ultimately, we select the corrective action with the largest F(s, a) value as the optimal decision.

[0068] For example, suppose that after taking the corrective action of adjusting the pile driver's position, the calculation results are A(s,a) = 0.8 (high accuracy), E(s,a) = 0.7 (moderate efficiency), R(s,a) = 0.5 (low resource consumption), and S(s,a) = 0.9 (good safety performance). Then, according to the evaluation function F(s,a), the quantitative evaluation value of this action is:

[0069] F(s,a)=0.4*0.8+0.3*0.7-0.2*0.5+0.1*0.9=0.67

[0070] By comparing the quantitative evaluation values ​​of different correction actions, the system can select the optimal correction strategy to achieve intelligent decision-making and dynamic adjustment of pile position correction.

[0071] Example 2:

[0072] The strategy optimization unit uses the Q-learning algorithm in the reinforcement learning algorithm to train the correction strategy model. The specific implementation steps of the model training include:

[0073] Step 1: Initialize a two-dimensional array as a Q-table, with rows representing the pile position state s and columns representing the corrective action a. Each element Q(s,a) is initialized to 0, indicating that the expected reward of taking action a in state s is 0. The pile position state s can be discretized based on actual conditions. For example, continuous variables such as position deviation, tilt angle, and soil parameters can be divided into several discrete intervals, each corresponding to a row in the Q-table. The corrective action a directly corresponds to a column in the Q-table.

[0074] Step 2: Set the learning rate α, discount factor γ, and exploration rate ε. The learning rate α controls the learning speed of the model and is typically a small value, such as 0.1 or 0.01. The discount factor γ controls the discounted value of future rewards and is typically a value between 0 and 1, such as 0.9 or 0.95. The exploration rate ε is used to balance exploration (randomly selecting an action) and exploitation (selecting the action with the highest current Q value). It is typically gradually reduced as training progresses, allowing the model to explore more different actions in the early stages of training and exploit the known optimal actions more in the later stages of training.

[0075] Step 3: For each training round, perform the following steps:

[0076] Step 301: Initialize the pile position state s to the current real-time monitored pile position state. In actual application, this can be obtained through the pile position state monitoring module.

[0077] Step 302: Select a corrective action a based on the current state s and the ε-greedy strategy. An action is randomly selected with probability ε, and the action with the highest current Q value is selected with probability 1-ε. This ensures that the model can explore new actions while also leveraging known optimal actions during training.

[0078] Step 303: Execute action a and observe the next state s' and the immediate reward r. The immediate reward r is calculated based on the evaluation function F(s, a) of the feedback evaluation unit, which comprehensively considers the accuracy, efficiency, resource consumption, and safety performance of the pile position correction.

[0079] Step 304: Update the Q-table value according to the Q-learning algorithm update formula. The update formula is:

[0080] Q(s,a)=Q(s,a)+α*[r+γ*max_a'Q(s',a')-Q(s,a)]

[0081] Here, max_a'Q(s',a') represents the maximum Q value of all possible actions in the next state s'. This update formula enables the model to update the Q value based on the immediate reward of the current action and the maximum possible future reward, thereby gradually learning the optimal correction strategy.

[0082] Step 4: Repeat step 3 until the preset number of training rounds is reached or the Q-table value converges. The number of training rounds can be set based on actual conditions, for example, 10,000 rounds or more. Convergence of the Q-table value can be determined by observing the changes in the Q-table value. If the Q-table value changes very little over multiple consecutive rounds, it can be considered converged.

[0083] Step 5: After training, the resulting Q-table is used as the correction policy model. For any given position state s, the action a with the highest Q value is selected as the optimal correction action by searching the Q-table. In practice, this can be implemented by writing a simple lookup function that accepts the position state s as input and returns the action a with the highest Q value as output.

[0084] Suppose that in a training round, the current pile position state s is (position deviation 5 cm, tilt angle 2 degrees, medium soil hardness). Possible corrective actions a include adjusting the pile driver position, changing the driving force, and using an auxiliary corrective device. Based on the current Q-table and the ε-greedy strategy, the model selects a corrective action (e.g., adjusting the pile driver position). After executing this action, the next state s' is observed to be (position deviation 3 cm, tilt angle 1 degree, medium soil hardness). The immediate reward r is calculated as 0.8 based on the evaluation function of the feedback evaluation unit (assuming that this action performs well in terms of accuracy, efficiency, resource consumption, and safety). The Q-table is then updated according to the Q-learning algorithm's update formula. By repeatedly repeating this process, the model gradually learns which corrective actions, under different states, yield the greatest long-term reward. Ultimately, the resulting Q-table, after training, can serve as a corrective strategy model to guide actual pile position correction efforts.

[0085] Example 3:

[0086] To train the correction strategy model, we generate a simulated dataset to provide training data. The detailed steps for generating and preprocessing the simulated dataset include:

[0087] For each simulated construction scenario, the state-action-reward triple (s, a, r) ​​is generated by following the steps below:

[0088] S1: Simulate changes in pile position status. Based on actual construction conditions and possible changes in pile position status, set initial values ​​for the pile position status and simulate changes in pile position status (such as position deviation, tilt angle, and soil parameters) during construction. For example, you could set the initial position deviation to 5 cm, the tilt angle to 2 degrees, and the soil hardness to medium, and simulate how these status parameters change as construction progresses.

[0089] S2: Simulate the execution of corrective actions. Based on the simulated pile position, possible corrective actions are selected for execution. These can include adjusting the pile driver position, changing the driving force, or using auxiliary corrective devices. In the simulation, corrective actions are selected randomly or according to a specific rule, and their execution effects are simulated.

[0090] S3: Calculate the immediate reward r. The corrective action is evaluated based on pre-set evaluation criteria, and an immediate reward r is assigned. This evaluation criteria can comprehensively consider aspects such as the pile position correction accuracy, efficiency, resource consumption, and safety performance. For example, if the corrective action brings the pile position closer to the expected value while consuming fewer resources and reducing safety risks, a higher reward will be awarded.

[0091] S4: Form a state-action-reward triple (s, a, r). Combine the simulated pile state s, the corrective action a, and the calculated immediate reward r into a triple and record it.

[0092] S5: Repeat steps S1 to S4 until a sufficient number of simulation scenarios and corresponding state-action-reward triplets are generated.

[0093] After aggregating the generated state-action-reward triples into a simulated dataset, the following preprocessing steps are performed:

[0094] Encoding: Discrete features in the simulation dataset are converted to one-hot encoding. For example, soil hardness in pile position status may be a discrete feature with multiple values ​​such as "soft," "medium," and "hard." Using one-hot encoding, these values ​​are converted into binary vectors for easier model processing.

[0095] Standardization: Standardize the continuous features in the simulation data set. Using the Z-score standardization method, subtract the mean of each numerical data and divide it by its standard deviation so that the processed data conforms to the standard normal distribution. The formula for standardization is: Z = (X-μ) / σ, where X is the original data, μ is the mean of the original data, σ is the standard deviation of the original data, and Z is the standardized data. For example, for the continuous feature of position deviation in pile position status, its mean and standard deviation in all simulation scenarios can be calculated and standardized using the above formula.

[0096] Consider a simulation scenario where the initial pile position states are 5cm deviation, 2 degrees tilt, and medium soil hardness. After the simulation executes a corrective action to adjust the pile driver's position, the deviation is reduced to 3cm, the tilt angle remains unchanged, and the soil hardness remains unchanged. Based on the pre-set evaluation criteria, this corrective action receives an immediate reward of 0.8. This results in a state-action-reward triple: (s = [5cm, 2 degrees, medium], a = adjust pile driver position, r = 0.8).

[0097] After generating multiple simulation scenarios and corresponding triples using the above method, they are aggregated into a simulated dataset. Discrete features in the dataset (such as soil hardness) are then one-hot encoded, and continuous features (such as position deviation) are z-score normalized. This processed simulated dataset serves as training data for the correction strategy model.

[0098] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that includes a list of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus.

[0099] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. A real-time deviation correction control system for automatic pile placement on soft foundation, characterized in that: The system comprises: Pile status monitoring module, used to monitor pile position, tilt angle and soil parameters in real time; A correction model construction module is used to construct a correction strategy model. The correction strategy model models the pile placement process as a dynamic optimization problem and dynamically adjusts the pile position correction plan through an algorithm. This module further includes: State definition unit, used to describe the state space of the pile position at the current moment; Action definition unit, used to define the action space for correction; The feedback evaluation unit is used to define the evaluation function and quantitatively evaluate the results of the correction action based on the accuracy, efficiency, resource consumption and safety performance of the pile position correction; The strategy optimization unit uses a reinforcement learning algorithm to select the optimal corrective action under a given state. Through continuous iterative learning, it optimizes the correction strategy to maximize the long-term correction effect. The control system execution module is used to input the data obtained by the pile position status monitoring module into the correction strategy model, receive the correction strategy output by the model, and control the pile driver and other auxiliary equipment to perform corresponding correction actions; The strategy optimization unit uses the Q-learning algorithm in the reinforcement learning algorithm to train the correction strategy model; The step of the strategy optimization unit training the deviation correction strategy model includes: Step 1: Initialize the Q-table. The Q-table is a two-dimensional array whose rows represent the pile state s and columns represent the corrective action a. Each element Q(s, a) is initialized to 0, indicating that the expected reward value of taking action a in state s is 0. Step 2: Set the learning rate α, discount factor γ, and exploration rate ε, where α controls the learning speed, γ controls the discounted value of future returns, and ε controls the balance between exploration and exploitation. Step 3: For each training round: Step 301: Initialize the pile position state s to the current real-time monitored pile position state; Step 302: Select a corrective action a based on the current state s and the ε-greedy strategy. Randomly select an action with a probability of ε, and select the action with the largest current Q value with a probability of 1-ε. Step 303: Execute action a and observe the next state s' and the immediate reward r, where the immediate reward r is calculated based on the evaluation function F(s, a) of the feedback evaluation unit; Step 304: Update the Q-table value according to the update formula of the Q-learning algorithm. The update formula is: Q(s, a) = Q(s, a) + α * [r + γ * max_a' Q(s', a') - Q(s, a)] Among them, max_a' Q(s', a') represents the maximum Q value of all possible actions in the next state s'; Step 4: Repeat step 3 until the preset number of training rounds is reached or the value of the Q table converges; Step 5: After training, the obtained Q table is used as the correction strategy model. For any given pile position state s, the action a with the largest Q value is selected as the optimal correction action by searching the Q table. The state space includes pile position, pile tilt angle, soil parameters and current correction action; The action space includes adjusting the position of the pile driver, changing the pile driving force, using auxiliary correction devices and notifying the operator to intervene.

2. A real-time deviation correction control system for automatic pile placement on soft foundation according to claim 1, characterized in that: The soil parameters include soil hardness, moisture and bearing capacity.

3. A real-time deviation correction control system for automatic pile placement on soft foundation according to claim 1, characterized in that: The implementation of the feedback evaluation unit includes: An evaluation function F(s, a) is defined, where s represents the state space of the pile position at the current moment; a represents the corrective action taken, which is selected from the corrective action space and includes adjusting the position of the pile driver, changing the driving force, using auxiliary corrective devices, and notifying the operator to intervene. The evaluation function F(s, a) performs a weighted summation based on the pile position correction accuracy A(s, a), correction efficiency E(s, a), resource consumption R(s, a), and safety performance S(s, a) to obtain a quantitative evaluation value of the corrective action result. The specific algorithm formula is: F(s, a) = w1 * A(s, a) + w2 * E(s, a) - w3 * R(s, a) + w4 * S(s, a) Among them, w1, w2, w3, and w4 are the weight coefficients of accuracy, correction efficiency, resource consumption, and safety performance, respectively, and w1 + w2 + w3 + w4 = 1. A(s, a) represents the pile position correction accuracy after taking action a in state s, which is obtained by calculating the deviation between the correction and the expected position. E(s, a) represents the correction efficiency, which is obtained by calculating the time required for correction. R(s, a) represents resource consumption, including energy consumption, material consumption, and labor cost. S(s, a) represents safety performance, which is obtained by evaluating the safety risks generated during the correction process.

4. A real-time deviation correction control system for automatic pile placement on soft foundation according to claim 1, characterized in that: The training data for the correction strategy model is obtained by generating a simulated data set. The specific method is as follows: For each simulated construction scenario, the changes in pile position status and the execution of corrective actions are simulated. The state s, the action a, and the immediate reward r given according to the preset evaluation criteria are recorded at each step to form a state-action-reward triple (s, a, r). Summarize the state-action-reward triples generated by all simulation scenarios to form a simulation dataset; The data in the simulated dataset is further encoded and standardized, and the processed simulated dataset is used as training data for the correction strategy model.

5. A real-time deviation correction control system for automatic pile placement on soft foundation according to claim 4, characterized in that: The method of encoding the data in the simulated data set includes: performing one-hot encoding conversion on discrete features.

6. A real-time deviation correction control system for automatic pile placement on soft foundation according to claim 4, characterized in that: The method of standardizing the data in the simulated data set includes: using the Z-score standardization method to standardize the continuous features, specifically: subtracting the mean of each numerical data and dividing it by its standard deviation so that the processed data conforms to the standard normal distribution. The standardization formula is: ,in is the original data, is the mean of the original data, is the standard deviation of the original data, The data are standardized.

Images