Soft foundation automatic pile arrangement position real-time deviation rectification control system

Abstract

The invention relates to the technical field of civil engineering, and discloses a soft foundation automatic pile arrangement real-time deviation rectification control system which integrates a pile position state monitoring module, a deviation rectification model building module and a control execution module, and real-time monitoring of the pile position, the inclination angle and soil parameters is achieved. The system can automatically select the optimal deviation correction action by constructing a deviation correction model of a dynamic optimization problem and combining a reinforcement learning algorithm, and intelligent decision and dynamic adjustment are realized. According to the implementation of the method, manual intervention is greatly reduced, the uncertainty of relying on experience of operators is reduced, and an efficient and accurate deviation correction solution is provided for soft foundation pile arrangement construction.

Description

Technical Field

[0001] The invention relates to the technical field of civil engineering, in particular to a real-time deviation correction control system for automatic pile arrangement on soft foundation. Background Art

[0002] In the field of civil engineering, pile construction on soft soil foundation is a complex and critical task. Due to the low bearing capacity, high compressibility and volatility of soft soil, the pile position is prone to deviation during the construction process, making it difficult to ensure the verticality and position accuracy of the pile. Most of the existing pile position correction technologies adopt the method of post-detection and adjustment, that is, after the construction is completed, the deviation of the pile position is detected by measuring equipment, and then manual or mechanical adjustments are made based on the detection results. This method has many shortcomings: first, post-detection cannot timely detect and correct the pile position deviation during the construction process, which may lead to a decrease in the bearing capacity of the pile and even cause engineering quality problems; second, manual adjustment relies on the experience of the operator, and it is difficult to ensure the accuracy and efficiency of the adjustment; finally, the traditional correction method lacks intelligent decision-making support and cannot dynamically adjust the correction plan according to the pile position status and soil conditions.

[0003] In addition, the change of soil parameters has an important impact on the deviation of pile position, but the existing correction technology often ignores the real-time monitoring and utilization of key parameters such as soil hardness, moisture and bearing capacity. The lack of accurate grasp of these parameters makes the formulation of correction strategies lack scientific basis and it is difficult to achieve the ideal correction effect. Summary of the invention

[0004] The object of the present invention is to provide a real-time deviation correction control system for automatic pile arrangement on soft foundation to solve the problems raised in the above-mentioned background technology.

[0005] To achieve the above object, the present invention provides the following technical solution: a real-time deviation correction control system for automatic pile placement on soft foundation, the system comprising:

[0006] Pile status monitoring module, used to monitor the position, inclination angle and soil parameters of the pile in real time;

[0007] A correction model construction module is used to construct a correction strategy model. The correction strategy model models the pile arrangement process as a dynamic optimization problem and dynamically adjusts the pile position correction scheme through an algorithm. The module further includes:

[0008] The state definition unit is used to describe the state space of the pile position at the current moment;

[0009] An action definition unit, used to define the action space for correction;

[0010] The feedback evaluation unit is used to define the evaluation function and to quantitatively evaluate the results of the correction action according to the accuracy, efficiency, resource consumption and safety performance of the pile position correction;

[0011] The strategy optimization unit uses a reinforcement learning algorithm to select the optimal correction action under a given state, and optimizes the correction strategy through continuous iterative learning to maximize the long-term correction effect;

[0012] The control system execution module is used to transfer the data obtained by the pile position status monitoring module to the correction strategy model, receive the correction strategy output by the model, and control the pile driver and other auxiliary equipment to perform corresponding correction actions.

[0013] Preferably, the soil parameters include soil hardness, moisture and bearing capacity.

[0014] Preferably, the state space includes pile position, pile inclination angle, soil parameters and current correction action.

[0015] Preferably, the action space includes adjusting the position of the pile driver, changing the pile driving force, using an auxiliary deviation correction device and notifying the operator to intervene.

[0016] Preferably, the feedback evaluation unit is implemented as follows:

[0017] An evaluation function F(s,a) is defined, where s represents the state space of the pile position at the current moment; a represents the corrective action taken, which is selected from the corrective action space, including adjusting the position of the pile driver, changing the pile driving force, using auxiliary corrective devices, and notifying the operator to intervene; the evaluation function F(s,a) performs weighted summation according to the accuracy of pile position correction A(s,a), correction efficiency E(s,a), resource consumption R(s,a) and safety performance S(s,a) to obtain a quantitative evaluation value of the result of the corrective action; the specific algorithm formula is:

[0018] F(s,a)=w1*A(s,a)+w2*E(s,a)-w3*R(s,a)+w4*S(s,a)

[0019] Among them, w1, w2, w3, and w4 are the weight coefficients of accuracy, correction efficiency, resource consumption, and safety performance, respectively, and w1+w2+w3+w4=1; A(s,a) represents the correction accuracy of the pile position after taking action a in state s, which is obtained by calculating the deviation between the correction and the expected position; E(s,a) represents the correction efficiency, which is obtained by calculating the time required for correction; R(s,a) represents resource consumption, including energy consumption, material consumption, and labor cost; S(s,a) represents safety performance, which is obtained by evaluating the safety risks generated during the correction process.

[0020] Preferably, the strategy optimization unit uses a Q-learning algorithm in a reinforcement learning algorithm to train the correction strategy model.

[0021] Preferably, the step of the strategy optimization unit training the deviation correction strategy model includes:

[0022] Step 1: Initialize the Q table. The Q table is a two-dimensional array, whose rows represent the pile state s and columns represent the correction action a. Each element Q(s,a) is initialized to 0, indicating that the expected reward value of taking action a in state s is 0;

[0023] Step 2: Set the learning rate α, discount factor γ and exploration rate ε, where α controls the learning speed, γ controls the discounted value of future returns, and ε controls the balance between exploration and exploitation;

[0024] Step 3: For each training round:

[0025] Step 301: Initialize the pile position state s to the pile position state currently monitored in real time;

[0026] Step 302: Select a corrective action a according to the current state s and the ε-greedy strategy, randomly select an action with a probability of ε, and select the action with the largest current Q value with a probability of 1-ε;

[0027] Step 303: Execute action a, observe the next state s' and the immediate reward r, where the immediate reward r is calculated according to the evaluation function F(s,a) of the feedback evaluation unit;

[0028] Step 304: Update the value of the Q table according to the update formula of the Q-learning algorithm. The update formula is:

[0029] Q(s,a)=Q(s,a)+α*[r+γ*max_a'Q(s',a')-Q(s,a)]

[0030] Among them, max_a'Q(s',a') represents the maximum Q value of all possible actions in the next state s';

[0031] Step 4: Repeat step 3 until the preset number of training rounds is reached or the value of the Q table converges;

[0032] Step 5: After the training is completed, the obtained Q table is used as the correction strategy model. For any given pile position state s, the action a with the largest Q value is selected as the optimal correction action by searching the Q table.

[0033] Preferably, a simulated data set is generated as training data for the correction strategy model, and the specific method is as follows:

[0034] For each simulated construction scenario, simulate the change of pile position status and the execution of the correction action, record the state s, the action a, and the immediate reward r given according to the preset evaluation criteria at each step, and form a state-action-reward triple (s, a, r);

[0035] Aggregate the state-action-reward triplets generated by all simulation scenarios to form a simulation dataset;

[0036] The data in the simulated data set is further encoded and standardized, and the processed simulated data set is used as training data for the correction strategy model.

[0037] Preferably, the method of encoding the data in the simulated data set includes: performing one-hot encoding conversion on discrete features.

[0038] Preferably, the method of standardizing the data in the simulated data set includes: using the Z-score standardization method to standardize the continuous features, specifically: subtracting the mean of each numerical data and dividing it by its standard deviation so that the processed data conforms to the standard normal distribution, and the standardization formula is: Z = (X-μ) / σ, wherein X is the original data, μ is the mean of the original data, σ is the standard deviation of the original data, and Z is the standardized data.

[0039] Compared with the prior art, the present invention has the following beneficial effects:

[0040] The correction model building module models the pile arrangement process as a dynamic optimization problem, and combines the reinforcement learning algorithm to continuously optimize the correction strategy in the strategy optimization unit. The present invention can automatically select the optimal correction action according to the current pile position status, soil conditions and preset pile arrangement requirements, realize intelligent decision-making and dynamic adjustment of the correction plan, which not only improves the accuracy and efficiency of correction, but also greatly reduces the uncertainty of manual intervention and reliance on operator experience.

[0041] The present invention provides a more scientific basis for the formulation of correction strategies by real-time monitoring of key parameters such as soil hardness, moisture and bearing capacity, making the correction action more accurate and effective, and better adapting to the pile layout requirements under different soil conditions. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 This is a working principle diagram of a real-time deviation correction control system for automatic pile placement on soft foundation according to the present invention;

[0043] Figure 2 It is a schematic diagram of the design of the evaluation function in the feedback evaluation unit;

[0044] Figure 3 This is the training flowchart of the correction strategy model. DETAILED DESCRIPTION

[0045] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0046] See also Figure 1-3 The present invention provides a technical solution: a real-time deviation correction control system for automatic pile arrangement on soft foundation, the system comprising:

[0047] The pile position status monitoring module is responsible for real-time monitoring of the position, inclination angle and soil parameters of the pile position. The module collects relevant data of the pile position in real time by integrating high-precision sensors and detection equipment, such as GPS locators, inclination sensors and soil parameter measuring instruments. These data include the precise position coordinates of the pile position, the degree of inclination angle, and the parameter values ​​of soil hardness, humidity and bearing capacity. The monitoring module transmits this data to other parts of the control system in real time, providing basic information for the formulation of subsequent correction strategies.

[0048] The correction model building module is responsible for building the correction strategy model and dynamically adjusting the pile position correction plan based on the monitored pile position status and the preset pile layout requirements. This module contains the following units:

[0049] State definition unit: This unit is used to describe the state space of the pile position at the current moment. The state space includes the pile position, the pile inclination angle, soil parameters, and the current corrective action. These state variables are quantified into specific values ​​or categories.

[0050] Action definition unit: This unit defines the action space for deviation correction, that is, the deviation correction measures that the system can take. The action space includes adjusting the position of the pile driver, changing the pile driving force, using auxiliary deviation correction devices, and notifying the operator to intervene.

[0051] Feedback evaluation unit: This unit designs a set of evaluation functions to quantitatively evaluate the results of the correction action. The evaluation function comprehensively considers the accuracy, efficiency, resource consumption and safety performance of the pile position correction, and provides the system with a standard for evaluating the correction effect.

[0052] Policy optimization unit: This unit uses a reinforcement learning algorithm to select the optimal corrective action under a given state based on the evaluation results provided by the feedback evaluation unit. Through continuous iterative learning, the policy optimization unit gradually optimizes the correction strategy to maximize the long-term correction effect. The reinforcement learning algorithm enables the system to gradually master the optimal correction strategy in the process of continuous trial and learning.

[0053] The control system execution module is responsible for transferring the data obtained by the pile position status monitoring module to the correction strategy model and receiving the correction strategy output by the model. This module controls the pile driver and other auxiliary equipment to perform corresponding correction actions according to the correction strategy. For example, when the correction strategy requires the adjustment of the pile driver position, the control system execution module will issue a command to the pile driver to move it to the specified position; when the pile driving force needs to be changed, the control system will adjust the pile driver's striking force parameters. By precisely controlling the actions of the pile driver and auxiliary equipment, the system can achieve accurate pile position correction.

[0054] The present invention will be further described below in conjunction with Examples 1 to 3:

[0055] Embodiment 1:

[0056] In the implementation of the feedback evaluation unit, a comprehensive evaluation function F(s,a) is designed to quantitatively evaluate the correction action a taken under a specific state s. This function comprehensively considers the accuracy, correction efficiency, resource consumption and safety performance of the pile position correction to ensure the comprehensiveness and optimization of the correction strategy.

[0057] Definition of the evaluation function F(s,a):

[0058] The evaluation function F(s,a) uses a weighted summation method to combine the four evaluation indicators of pile position correction accuracy A(s,a), correction efficiency E(s,a), resource consumption R(s,a) and safety performance S(s,a) to form a comprehensive quantitative evaluation value. The specific algorithm formula is:

[0059] F(s,a)=w1*A(s,a)+w2*E(s,a)-w3*R(s,a)+w4*S(s,a)

[0060] Among them, w1, w2, w3, and w4 are weight coefficients of the four evaluation indicators, and they satisfy w1+w2+w3+w4=1. These weight coefficients are set according to the actual application scenarios and importance to reflect the relative importance of different indicators in the correction decision.

[0061] Calculation of evaluation indicators:

[0062] (1) Accuracy A(s,a): After taking action a in state s, the accuracy of the correction is evaluated by calculating the deviation between the corrected pile position and the expected position. The smaller the deviation, the higher the accuracy, and the larger the value of A(s,a). For example, the deviation can be calculated using metrics such as Euclidean distance or Manhattan distance.

[0063] (2) Correction efficiency E(s,a): Correction efficiency is evaluated by calculating the time required for correction. The shorter the time, the higher the efficiency and the larger the value of E(s,a). In practical applications, the time required for correction can be calculated by recording the timestamps of the start and end of the correction action.

[0064] (3) Resource consumption R(s,a): Resource consumption includes energy consumption, material consumption, and labor costs. After taking action a in state s, resource consumption is evaluated by calculating the actual amount of resources consumed. The less resource consumption, the smaller the value of R(s,a) (note that there is a minus sign before R(s,a) in the formula, indicating that resource consumption is a deduction item in the evaluation value).

[0065] (4) Safety performance S(s,a): Safety performance is quantified by evaluating the safety risks generated during the correction process. The lower the safety risk, the larger the value of S(s,a). In practical applications, safety risks can be evaluated based on the risk of equipment damage, casualties, etc. that may be caused by the correction action.

[0066] Taking the deviation correction decision in a pile construction as an example, assuming that the current pile position state s is (position deviation 5cm, inclination angle 2 degrees, medium soil hardness, moderate humidity), the optional deviation correction actions a include adjusting the position of the pile driver, changing the pile driving force, and using auxiliary deviation correction devices. Based on historical data and experience, the weight coefficients are set to w1=0.4, w2=0.3, w3=0.2, and w4=0.1.

[0067] For each corrective action, the evaluation values ​​of its accuracy, correction efficiency, resource consumption and safety performance are calculated respectively, and then substituted into the evaluation function F(s,a) for calculation. Finally, the corrective action with the largest F(s,a) value is selected as the optimal decision.

[0068] For example, assuming that after taking the corrective action of adjusting the pile driver position, the calculation results are A(s,a)=0.8 (high accuracy), E(s,a)=0.7 (moderate efficiency), R(s,a)=0.5 (low resource consumption), S(s,a)=0.9 (good safety performance). Then according to the evaluation function F(s,a), the quantitative evaluation value of this action is:

[0069] F(s,a)=0.4*0.8+0.3*0.7-0.2*0.5+0.1*0.9=0.67

[0070] By comparing the quantitative evaluation values ​​of different correction actions, the system can select the optimal correction strategy to achieve intelligent decision-making and dynamic adjustment of pile position correction.

[0071] Embodiment 2:

[0072] The strategy optimization unit uses the Q-learning algorithm in the reinforcement learning algorithm to train the correction strategy model. The specific implementation steps of the model training include:

[0073] Step 1: Initialize a two-dimensional array as a Q table, whose rows represent the pile position state s and columns represent the corrective action a. Each element Q(s,a) is initialized to 0, indicating that the expected return value of taking action a in state s is 0. The pile position state s can be discretized according to the actual situation. For example, continuous variables such as position deviation, tilt angle and soil parameters can be divided into several discrete intervals, each of which corresponds to a row in the Q table. The corrective action a directly corresponds to the column in the Q table.

[0074] Step 2: Set the learning rate α, discount factor γ, and exploration rate ε. The learning rate α controls the learning speed of the model and usually takes a small value, such as 0.1 or 0.01. The discount factor γ controls the discounted value of future returns and usually takes a value between 0 and 1, such as 0.9 or 0.95. The exploration rate ε is used to balance exploration (randomly selecting actions) and utilization (selecting the action with the largest current Q value). It usually decreases as training progresses, so that the model explores different actions more in the early stages of training and utilizes known optimal actions more in the later stages of training.

[0075] Step 3: For each training round, perform the following steps:

[0076] Step 301: Initialize the pile position state s to the pile position state currently monitored in real time. In practical applications, this can be obtained through a pile position state monitoring module.

[0077] Step 302: Select a corrective action a based on the current state s and the ε-greedy strategy. Randomly select an action with a probability of ε, and select the action with the largest current Q value with a probability of 1-ε. This ensures that the model can explore new actions and use known optimal actions during training.

[0078] Step 303: Execute action a and observe the next state s' and the immediate reward r. The immediate reward r is calculated according to the evaluation function F(s,a) of the feedback evaluation unit, which comprehensively considers the accuracy, efficiency, resource consumption and safety performance of the pile position correction.

[0079] Step 304: Update the value of the Q table according to the update formula of the Q-learning algorithm. The update formula is:

[0080] Q(s,a)=Q(s,a)+α*[r+γ*max_a'Q(s',a')-Q(s,a)]

[0081] Among them, max_a'Q(s',a') represents the maximum Q value of all possible actions in the next state s'. This update formula enables the model to update the Q value based on the immediate reward of the current action and the maximum possible reward in the future, thereby gradually learning the optimal correction strategy.

[0082] Step 4: Repeat step 3 until the preset number of training rounds is reached or the value of the Q table converges. The number of training rounds can be set according to the actual situation, such as 10,000 rounds or more. The convergence of the value of the Q table can be judged by observing the changes in the Q table. If the value of the Q table changes very little in multiple consecutive rounds, it can be considered to have converged.

[0083] Step 5: After training, the obtained Q table is used as the correction strategy model. For any given pile state s, the action a with the largest Q value is selected as the optimal correction action by searching the Q table. In practical applications, this can be achieved by writing a simple search function that accepts the pile state s as input and returns the action a with the largest Q value as output.

[0084] Assume that in a certain training round, the current pile position state s is (position deviation 5cm, tilt angle 2 degrees, medium soil hardness), and the optional corrective actions a include adjusting the position of the pile driver, changing the pile driving force, and using auxiliary corrective devices. According to the current Q table and the ε-greedy strategy, the model selects a corrective action (such as adjusting the position of the pile driver). After executing the action, the next state s' is observed to be (position deviation 3cm, tilt angle 1 degree, medium soil hardness), and the immediate reward r is calculated to be 0.8 according to the evaluation function of the feedback evaluation unit (assuming that the action performs well in terms of accuracy, efficiency, resource consumption, and safety performance). Then, the value of the Q table is updated according to the update formula of the Q-learning algorithm. By repeating this process continuously, the model gradually learns what corrective actions to take under different states to obtain the greatest long-term reward. Finally, the Q table obtained after the training can be used as a correction strategy model to guide the actual pile position correction work.

[0085] Embodiment 3:

[0086] In order to train the correction strategy model, the method of generating a simulated data set is used to provide training data. The detailed steps of generating and preprocessing the simulated data set include:

[0087] For each simulated construction scenario, the state-action-reward triple (s, a, r) ​​is generated by following the steps below:

[0088] S1: Simulate the change of pile position status. According to the actual construction conditions and possible changes in pile position status, set the initial value of the pile position status, and simulate the changes of the pile position status (such as position deviation, tilt angle, soil parameters, etc.) during the construction process. For example, the initial position deviation can be set to 5cm, the tilt angle to 2 degrees, and the soil hardness to medium, and simulate how these status parameters change as the construction progresses.

[0089] S2: Simulate the execution of the corrective action. According to the simulated pile position status, select possible corrective actions for execution. Corrective actions may include adjusting the position of the pile driver, changing the pile driving force, using auxiliary corrective devices, etc. In the simulation, corrective actions are selected randomly or according to certain rules, and their execution effects are simulated.

[0090] S3: Calculate the immediate reward r. According to the preset evaluation criteria, the corrective action is evaluated and an immediate reward r is given. The evaluation criteria can comprehensively consider the accuracy, efficiency, resource consumption and safety performance of the pile position correction. For example, if the corrective action makes the pile position state closer to the expected value, consumes less resources and has a lower safety risk, a higher reward will be given.

[0091] S4: Form a state-action-reward triple (s, a, r). Combine the simulated pile state s, the corrective action a, and the calculated immediate reward r into a triple and record it.

[0092] S5: Repeat steps S1 to S4 until a sufficient number of simulation scenarios and corresponding state-action-reward triplets are generated.

[0093] After aggregating the generated state-action-reward triplets into a simulated dataset, the following preprocessing steps are performed:

[0094] Encoding: Perform one-hot encoding conversion on discrete features in the simulation data set. For example, soil hardness in pile position status may be a discrete feature with multiple values ​​such as "soft", "medium", and "hard". Through one-hot encoding, these values ​​are converted into binary vector form for model processing.

[0095] Standardization: Standardize the continuous features in the simulated data set. Using the Z-score standardization method, subtract the mean of each numerical data and divide it by its standard deviation so that the processed data conforms to the standard normal distribution. The formula for standardization is: Z = (X-μ) / σ, where X is the original data, μ is the mean of the original data, σ is the standard deviation of the original data, and Z is the standardized data. For example, for the continuous feature of position deviation in the pile position state, its mean and standard deviation in all simulation scenarios can be calculated, and the above formula can be used for standardization.

[0096] Assume that in a simulation scenario, the initial pile position state is 5cm position deviation, 2 degrees inclination angle, and medium soil hardness. After the simulation executes the corrective action of adjusting the pile driver position, the position deviation is reduced to 3cm, the inclination angle remains unchanged, and the soil hardness does not change. According to the preset evaluation criteria, the corrective action receives an immediate reward of 0.8. Thus, a state-action-reward triple is formed: (s = [5cm, 2 degrees, medium], a = adjust the pile driver position, r = 0.8).

[0097] After generating multiple simulation scenarios and corresponding triplets according to the above method, they are aggregated into a simulation data set. Then, the discrete features in the data set (such as soil hardness) are converted into one-hot encoding, and the continuous features (such as position deviation) are normalized by Z-score. The processed simulation data set can be used as training data for the correction strategy model.

[0098] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0099] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A real-time deviation correction control system for automatic pile placement on soft foundation, characterized in that: The system comprises: Pile status monitoring module, used to monitor the position, inclination angle and soil parameters of the pile in real time; A correction model construction module is used to construct a correction strategy model. The correction strategy model models the pile arrangement process as a dynamic optimization problem and dynamically adjusts the pile position correction scheme through an algorithm. The module further includes: The state definition unit is used to describe the state space of the pile position at the current moment; An action definition unit, used to define the action space for correction; The feedback evaluation unit is used to define the evaluation function and to quantitatively evaluate the results of the correction action according to the accuracy, efficiency, resource consumption and safety performance of the pile position correction; The strategy optimization unit uses a reinforcement learning algorithm to select the optimal correction action under a given state, and optimizes the correction strategy through continuous iterative learning to maximize the long-term correction effect; The control system execution module is used to transfer the data obtained by the pile position status monitoring module to the correction strategy model, receive the correction strategy output by the model, and control the pile driver and other auxiliary equipment to perform corresponding correction actions.

2. A real-time deviation correction control system for automatic pile placement on soft foundation according to claim 1, characterized in that: The soil parameters include soil hardness, moisture and bearing capacity.

3. A real-time deviation correction control system for automatic pile placement on soft foundation according to claim 1, characterized in that: The state space includes the pile position, the pile inclination angle, soil parameters and the current deviation correction action.

4. The real-time deviation correction control system for automatic pile placement on soft foundation according to claim 1 is characterized by: The action space includes adjusting the position of the pile driver, changing the pile driving force, using an auxiliary deviation correction device and notifying the operator to intervene.

5. The real-time deviation correction control system for automatic pile placement on soft foundation according to claim 1 is characterized in that: The implementation of the feedback evaluation unit includes: An evaluation function F(s,a) is defined, where s represents the state space of the pile position at the current moment; a represents the corrective action taken, which is selected from the corrective action space, including adjusting the position of the pile driver, changing the pile driving force, using auxiliary corrective devices, and notifying the operator to intervene; the evaluation function F(s,a) performs weighted summation according to the accuracy of pile position correction A(s,a), correction efficiency E(s,a), resource consumption R(s,a) and safety performance S(s,a) to obtain a quantitative evaluation value of the result of the corrective action; the specific algorithm formula is: F(s,a)=w1*A(s,a)+w2*E(s,a)-w3*R(s,a)+w4*S(s,a) Among them, w1, w2, w3, and w4 are the weight coefficients of accuracy, correction efficiency, resource consumption, and safety performance, respectively, and w1+w2+w3+w4=1; A(s,a) represents the correction accuracy of the pile position after taking action a in state s, which is obtained by calculating the deviation between the correction and the expected position; E(s,a) represents the correction efficiency, which is obtained by calculating the time required for correction; R(s,a) represents resource consumption, including energy consumption, material consumption, and labor cost; S(s,a) represents safety performance, which is obtained by evaluating the safety risks generated during the correction process.

6. The real-time deviation correction control system for automatic pile placement on soft foundation according to claim 1 is characterized in that: The strategy optimization unit uses the Q-learning algorithm in the reinforcement learning algorithm to train the correction strategy model.

7. A real-time deviation correction control system for automatic pile placement on soft foundation according to claim 6, characterized in that: The step of training the deviation correction strategy model by the strategy optimization unit includes: Step 1: Initialize the Q table. The Q table is a two-dimensional array, whose rows represent the pile state s and columns represent the correction action a. Each element Q(s,a) is initialized to 0, indicating that the expected reward value of taking action a in state s is 0; Step 2: Set the learning rate α, discount factor γ and exploration rate ε, where α controls the learning speed, γ controls the discounted value of future returns, and ε controls the balance between exploration and exploitation; Step 3: For each training round: Step 301: Initialize the pile position state s to the pile position state currently monitored in real time; Step 302: Select a corrective action a according to the current state s and the ε-greedy strategy, randomly select an action with a probability of ε, and select the action with the largest current Q value with a probability of 1-ε; Step 303: Execute action a, observe the next state s' and the immediate reward r, where the immediate reward r is calculated according to the evaluation function F(s,a) of the feedback evaluation unit; Step 304: Update the value of the Q table according to the update formula of the Q-learning algorithm. The update formula is: Q(s,a)=Q(s,a)+α*[r+γ*max_a'Q(s',a')-Q(s,a)] Among them, max_a'Q(s',a') represents the maximum Q value of all possible actions in the next state s'; Step 4: Repeat step 3 until the preset number of training rounds is reached or the value of the Q table converges; Step 5: After the training is completed, the obtained Q table is used as the correction strategy model. For any given pile position state s, the action a with the largest Q value is selected as the optimal correction action by searching the Q table.

8. The real-time deviation correction control system for automatic pile placement on soft foundation according to claim 7 is characterized in that: The training data of the correction strategy model is obtained by generating a simulated data set. The specific method is as follows: For each simulated construction scenario, simulate the change of pile position status and the execution of the correction action, record the state s, the action a, and the instant reward r given according to the preset evaluation criteria at each step, and form a state-action-reward triple (s, a, r); Aggregate the state-action-reward triplets generated by all simulation scenarios to form a simulation dataset; The data in the simulated data set is further encoded and standardized, and the processed simulated data set is used as training data for the correction strategy model.

9. A real-time deviation correction control system for automatic pile placement on soft foundation according to claim 8, characterized in that: The method of encoding the data in the simulated data set includes: performing one-hot encoding conversion on discrete features.

10. The real-time deviation correction control system for automatic pile placement on soft foundation according to claim 8, characterized in that: The method of standardizing the data in the simulated data set includes: using the Z-score standardization method to standardize the continuous features, specifically: subtracting the mean of each numerical data and dividing it by its standard deviation, so that the processed data conforms to the standard normal distribution, and the standardization formula is: Z = (X-μ) / σ, where X is the original data, μ is the mean of the original data, σ is the standard deviation of the original data, and Z is the standardized data.

Images