Concrete 3D printing path optimization method and system based on reinforcement learning

Abstract

The invention relates to the technical field of concrete 3D printing path optimization, in particular to a concrete 3D printing path optimization method and system based on reinforcement learning, and the method comprises the following steps: collecting structure, equipment and environment parameters through an equipment sensor, and preprocessing the parameters into state information; a digital twinning environment and reinforcement learning model framework is synchronously constructed, a multi-stage training module, a lightweight inference engine and a constraint adaptation module are included, embedded deployment is adapted, and a digital twinning environment generates a path optimization simulation sample; establishing a multi-factor coupled rheological agent model based on the state information, associating an environment with an equipment state, and correcting rheological parameters in real time; and collecting a physical sample. According to the method, the reinforcement learning model framework is combined with the digital twinning environment and the multi-factor rheological agent model, so that the problems that the path optimization precision is insufficient and the printing quality is unstable due to the fact that most of traditional concrete 3D printing path optimization adopts fixed path parameters or simple algorithms are solved.

Description

Technical Field

[0001] This invention relates to the field of concrete 3D printing path optimization technology, and in particular to a concrete 3D printing path optimization method and system based on reinforcement learning. Background Technology

[0002] Concrete 3D printing technology, as a core new construction technology for the digital transformation of the construction industry, completely breaks through the dependence on molds in traditional pouring construction by directly forming concrete materials layer by layer. Its core advantage of eliminating the need for formwork not only saves the entire process of formwork design, fabrication, installation, and dismantling, significantly reducing labor costs and the consumption of formwork materials such as wood and steel, but also effectively shortens the construction cycle, making it particularly suitable for projects with tight schedules. Its flexible forming characteristics support the integrated fabrication of complex geometric components such as free-form surfaces, hollow structures, and irregular curved surfaces, eliminating the need for subsequent splicing or secondary processing, perfectly adapting to the personalized and customized needs of architectural design. Furthermore, its high material utilization rate allows for precise control of concrete extrusion to achieve on-demand material feeding, significantly reducing resource waste caused by grout leakage through formwork gaps and excessive material mixing in traditional construction, aligning with the industry development trend of green building and low-carbon construction.

[0003] Traditional concrete 3D printing path optimization mostly uses fixed path parameters or simple algorithms. Because it does not fully consider the impact of environmental and equipment status changes on the rheological properties of concrete, it results in insufficient path optimization accuracy and unstable printing quality. Summary of the Invention

[0004] To overcome the above shortcomings, this invention provides a method and system for optimizing the path of concrete 3D printing based on reinforcement learning. It aims to improve the problems of insufficient path optimization accuracy and unstable printing quality caused by the fact that traditional concrete 3D printing path optimization mostly adopts fixed path parameters or simple algorithms, and does not fully consider the influence of environmental and equipment status changes on the rheological properties of concrete.

[0005] In a first aspect, the present invention provides the following technical solution: a method for optimizing the path of concrete 3D printing based on reinforcement learning, comprising the following steps: The equipment sensors collect structural, equipment, and environmental parameters, which are preprocessed into state information. A digital twin environment and reinforcement learning model framework are constructed simultaneously, including a multi-stage training module, a lightweight inference engine, and a constraint adaptation module, which are adapted for embedded deployment. The digital twin environment generates path optimization simulation samples. A multi-factor coupled rheological proxy model is established based on state information to associate the environment and equipment state and correct rheological parameters in real time. Collect physical samples and build an experience cache library by combining them with simulation samples; dynamically adjust the ratio of the two types of samples according to the accumulation stage of physical samples, and train the reinforcement learning model in stages through a multi-stage training module. Configure the path optimization module to optimize the initial path parameters output by the reinforcement learning model after training through buffer smoothing and feedforward prediction, and adapt to the device action constraints. Establish a safety control mechanism, define safe shutdown, degraded printing and normal printing modes, set judgment conditions and thresholds, and clarify priorities; The reinforcement learning model is lightweighted after training and deployed on an embedded platform; during printing, the sensor acquires state information in real time, and the path parameters are adjusted in a closed loop based on the safety mechanism and the corrected rheological parameters, and the optimized path is output through the lightweight inference engine.

[0006] By adopting the above technical solution, a reinforcement learning model framework is used in conjunction with a digital twin environment and a multi-factor rheological proxy model to real-time correct rheological parameters based on the environment and equipment status, thereby achieving dynamic optimization of path parameters. This improves upon the problem that traditional concrete 3D printing path optimization mostly uses fixed path parameters or simple algorithms, which do not fully consider the impact of changes in environment and equipment status on concrete rheological properties, resulting in insufficient path optimization accuracy and unstable printing quality.

[0007] Furthermore, the preprocessing of state information includes the following steps: Use the 3σ criterion or box plot method to remove outliers from the original parameters and eliminate redundant and duplicate data. Missing values ​​in the original parameters can be supplemented using linear interpolation or moving average methods. Min-Max standardization was used to unify the dimensions of the data after removing redundant duplicates and supplementing missing values ​​in the original parameters. The structural parameters, equipment state parameters, and environmental parameters, after unifying their dimensions, are structurally spliced ​​together to form a state vector.

[0008] Furthermore, the real-time correction of rheological parameters based on the associated environment and equipment status includes the following steps: Extract environmental parameters such as temperature and humidity from the status information, and nozzle movement speed and extrusion pressure from the equipment status information; A mapping model between environmental equipment status and rheological parameters is constructed based on historical printing data. The mapping model adopts multiple nonlinear regression or LSTM neural network. Real-time data collection of current temperature, humidity, nozzle movement speed, and extrusion pressure; input of these data into the mapping model to calculate rheological parameter corrections; Adjust the current concrete rheological parameters according to the correction amount.

[0009] Furthermore, the step of dynamically adjusting the ratio of the two types of samples according to the accumulation stage of physical samples includes the following steps: Set preset thresholds S1 and S2 for the number of physical samples, with S1 being less than S2; divide the accumulation into three stages: the initial stage is when the number of physical samples is less than S1, the middle stage is when the number of physical samples is greater than or equal to S1 but less than S2, and the later stage is when the number of physical samples is greater than or equal to S2. The sampling ratios of simulation samples and physical samples were configured for the three stages respectively, namely 8:2, 5:5, and 2:8; Real-time statistics of the cumulative number of physical samples in the experience cache library to match the current accumulation stage; Simulation samples and physical samples are selected from the experience cache according to the sampling ratio corresponding to the current stage, and used for the phased training of the reinforcement learning model.

[0010] Furthermore, the optimization of the initial path parameters output by the trained reinforcement learning model through buffer smoothing and feedforward prediction includes the following steps: Extract the coordinates of continuous path points and the corresponding nozzle movement speed parameters from the initial path parameters; The adjacent path points are buffered and smoothed, and the radius of curvature of the transition path is calculated so that the tangent angle between adjacent path segments is less than a preset threshold. Obtain the maximum acceleration and maximum velocity parameters from the device motion constraints, and calculate the estimated velocity change from the current path point to the next path point based on the feedforward prediction model; By combining the estimated values ​​of the curvature radius and velocity change of the transition segment, the moving speed and path point coordinates in the initial path parameters are adjusted to form the optimized path parameters.

[0011] Furthermore, the separate determination conditions and thresholds, and the clarification of priorities, include the following steps: Determine the criteria for safe shutdown, downgraded printing, and normal printing modes, including equipment fault level, concrete rheological parameter deviation, and degree of environmental parameter exceedance. Set thresholds for each judgment parameter: set the equipment fault level threshold to severe, moderate and slight; set the rheological parameter deviation value threshold to critical value, warning value and normal range; set the environmental parameter exceedance threshold to severe exceedance, slight exceedance and no exceedance. Define the judgment conditions for each mode: The safe shutdown mode corresponds to the equipment fault level of severe, or the rheological parameter deviation value ≥ the critical value, or the environmental parameters are seriously out of standard; the downgraded printing mode corresponds to the equipment fault level of medium, or the rheological parameter deviation value is between the warning value and the critical value, or the environmental parameters are slightly out of standard; the normal printing mode corresponds to the equipment fault level of minor, and the rheological parameter deviation value ≤ the warning value, and the environmental parameters are not out of standard. Clearly define the priority order: safe stop mode has the highest priority, followed by degraded printing mode, and normal printing mode has the lowest priority.

[0012] Furthermore, the closed-loop adjustment of path parameters based on the safety mechanism and the corrected rheological parameters includes the following steps: Real-time acquisition of the current operating mode determined by the safety control mechanism and the corrected concrete rheological parameters; In safe stop mode, path parameter adjustment is stopped and a stop command is triggered; in degraded printing mode, the path parameter adjustment range is limited to a preset safe range; in normal printing mode, it is adjusted according to a dynamic optimization strategy. Based on the corrected rheological parameters, calculate the adjustment amounts for printing speed and path spacing in the path parameters; Verify whether the adjusted path parameters meet the device action constraints. If they do, update the path parameters; otherwise, recalculate the adjustment amount.

[0013] Secondly, the present invention provides the following technical solution: a reinforcement learning-based path optimization system for concrete 3D printing, the system comprising: The data acquisition and framework construction module is used to collect structural, equipment, and environmental parameters from the device sensors and preprocess them into state information. It also simultaneously constructs a digital twin environment and reinforcement learning model framework, including a multi-stage training module, a lightweight inference engine, and a constraint adaptation module, which are adapted for embedded deployment. The digital twin environment generates path optimization simulation samples. The rheological model building module is used to build a multi-factor coupled rheological proxy model based on state information, associate the environment and equipment state, and correct rheological parameters in real time. The sample training and adaptation module is used to collect physical samples and build an experience cache library by combining them with simulation samples; the ratio of the two types of samples is dynamically adjusted according to the accumulation stage of physical samples, and the reinforcement learning model is trained in stages through the multi-stage training module. The path parameter optimization module is used to configure the path optimization module. It optimizes the initial path parameters output by the reinforcement learning model after training through buffer smoothing and feedforward prediction to adapt to device action constraints. The security mechanism configuration module is used to establish a security control mechanism, define safe shutdown, degraded printing and normal printing modes, set judgment conditions and thresholds, and clarify priorities; A closed-loop output module is deployed to perform lightweight processing of the trained reinforcement learning model and is deployed on an embedded platform. During printing, the sensor acquires state information in real time, and based on the safety mechanism and the corrected rheological parameters, the path parameters are adjusted in a closed loop, and the optimized path is output through the lightweight inference engine.

[0014] The present invention has the following beneficial effects: 1. In this invention, a reinforcement learning model framework is combined with a digital twin environment and a multi-factor rheological proxy model to real-time correct rheological parameters based on the environment and equipment status, thereby achieving dynamic optimization of path parameters. This improves upon the problem that traditional concrete 3D printing path optimization mostly uses fixed path parameters or simple algorithms, which do not fully consider the impact of changes in environment and equipment status on concrete rheological characteristics, resulting in insufficient path optimization accuracy and unstable printing quality.

[0015] 2. In this invention, the ratio of simulation samples to physical samples is dynamically adjusted according to the accumulation stage of physical samples to train the reinforcement learning model in stages, thereby improving the model's adaptability to actual printing scenarios. This improves the problem that traditional concrete 3D printing path optimization related model training mostly uses a fixed sample ratio, and because it cannot dynamically optimize the training data composition according to the accumulation of physical samples, the model has weak generalization ability and is difficult to adapt to complex printing scenarios.

[0016] 3. In this invention, by establishing a safety control mechanism to define different printing modes and clarify priorities, and by combining real-time acquired status information with corrected rheological parameters to adjust path parameters in a closed loop, the printing process can be made safe and controllable. This improves the problem that traditional concrete 3D printing path optimization often lacks layered safety control and real-time closed-loop adjustment mechanisms, and therefore cannot respond to abnormal situations in the printing process in a timely manner, resulting in frequent printing failures and high safety risks. Attached Figure Description

[0017] Figure 1 This is a flowchart of a reinforcement learning-based path optimization method for concrete 3D printing proposed in this invention. Figure 2 This is a system architecture diagram of a reinforcement learning-based path optimization system for concrete 3D printing proposed in this invention. Detailed Implementation

[0018] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] Example 1 In a first embodiment of the present invention, the present invention provides a method for optimizing the path of concrete 3D printing based on reinforcement learning, such as... Figure 1 As shown, it includes the following steps: The equipment sensors collect structural, equipment, and environmental parameters, which are preprocessed into state information. A digital twin environment and reinforcement learning model framework are constructed simultaneously, including a multi-stage training module, a lightweight inference engine, and a constraint adaptation module, which are adapted for embedded deployment. The digital twin environment generates path optimization simulation samples. Furthermore, the preprocessing into state information includes the following steps: Use the 3σ criterion or box plot method to remove outliers from the original parameters and eliminate redundant and duplicate data. Missing values ​​in the original parameters can be supplemented using linear interpolation or moving average methods. Min-Max standardization was used to unify the dimensions of the data after removing redundant duplicates and supplementing missing values ​​in the original parameters. The structural parameters, equipment state parameters, and environmental parameters, after unifying their dimensions, are structurally spliced ​​together to form a state vector.

[0020] Specifically, the input data consists of structural parameters, equipment parameters, and environmental parameters collected in real time by the device's sensors; among these, structural parameters include the printing layer height. Component cross-sectional dimensions The parameters are collected by a displacement sensor laser ranging device, including nozzle temperature. Extrusion pressure Nozzle movement speed The environmental parameters, including room temperature, are collected by temperature sensors, pressure sensors, and encoders deployed on the printing device itself. Ambient humidity Data collected through environmental monitoring sensors is first used... Criteria or box plot methods can be used to remove outliers and redundant / duplicate data from the original parameters. The criterion first calculates the mean of a certain type of parameter. and standard deviation It will exceed to Values ​​falling within a certain range are considered outliers. In the box plot method, the lower quartiles of the parameters are calculated first. and upper quartiles Calculate the interquartile range It will exceed to Values ​​outside the specified range are identified as outliers, and redundant data from repeated collections are removed simultaneously. Then, missing values ​​in the original parameters are supplemented using linear interpolation or moving average methods. Linear interpolation is used to fill in missing values ​​in the data sequence. Based on its adjacent valid data points and Through formula The missing values ​​are calculated; the moving average method is used to calculate the missing values ​​in the data sequence. Select its front and back 10 adjacent valid data points, using the formula The missing values ​​were calculated. The preset sliding window size is used; then, Min-Max standardization is applied to unify the dimensions of the data after outlier cleaning and missing value imputation, for each parameter. Through formula Map it to the interval [0,1]; where This is the minimum value of the parameter. This is the maximum value of the parameter. and The physical range is determined by statistically analyzing historical printing data or preset parameters. Finally, the structural parameters, equipment status parameters, and environmental parameters with unified dimensions are structurally spliced ​​together according to preset dimensions to form a fixed-length state vector. This state vector serves as the output result, providing standardized and structured input data for the rheological proxy model. This ensures that the model can accurately capture the correlation between various parameters and provides reliable data support for subsequent real-time correction of rheological parameters and path optimization.

[0021] By removing outliers, supplementing missing values, standardizing data units, and structuring the data, the standardization and reliability of input data are improved, providing high-quality standardized input for the construction of multi-factor rheological proxy models and subsequent path optimization.

[0022] A multi-factor coupled rheological proxy model is established based on state information to associate the environment and equipment state and correct rheological parameters in real time. Furthermore, real-time correction of rheological parameters in relation to the environment and equipment status includes the following steps: Extract environmental parameters such as temperature and humidity from the status information, and nozzle movement speed and extrusion pressure from the equipment status information; A mapping model between environmental equipment status and rheological parameters is constructed based on historical printing data. The mapping model adopts multiple nonlinear regression or LSTM neural network. Real-time data collection of current temperature, humidity, nozzle movement speed, and extrusion pressure; input into mapping model to calculate rheological parameter corrections; Adjust the current concrete rheological parameters according to the correction amount.

[0023] Specifically, a series of methods are employed to correlate environmental and equipment states, including extracting key state parameters, constructing a mapping model, calculating corrections in real time, and adjusting parameters. This allows for real-time correction of rheological parameters. The input data consists of environmental and equipment state parameters extracted from preprocessed state information; among these, temperature... Data collected by ambient temperature and humidity sensors Data collected by an ambient humidity sensor and nozzle movement speed. The extrusion pressure is collected by the displacement encoder of the printing equipment. The parameters collected by the pressure sensor at the printhead are core physical quantities reflecting the real-time printing conditions. Based on multiple sets of data accumulated from historical printing, a mapping model between the environmental equipment status and rheological parameters is constructed. If a multivariate nonlinear regression model is used, its expression is: ; in These are the rheological parameters of concrete. to These are the regression coefficients obtained by fitting historical data using the least squares method. For the random error term, if an LSTM neural network is used, then... The input feature vector is processed through an input layer, hidden layers, and an output layer, including forget gates, input gates, cell state gates, and output gates. The output is the predicted value of the corresponding rheological parameters, and the network's weight matrix is ​​also shown. Bias vector The method is determined through iterative training and optimization using historical data, and real-time acquisition of the current printing scenario. Input the established mapping model to calculate the predicted rheological parameters adapted to the current operating conditions. Rheological parameter correction amount ;in The actual rheological parameters of the concrete during the current printing process are obtained in real time through a rheometer. Adjusting the current concrete rheological parameters yields the corrected rheological parameters. The corrected rheological parameters are used as the output and are then used in the buffering, smoothing and feedforward prediction process of the path optimization module. This provides rheological characteristic data that is adapted to real-time working conditions to support the optimization of the initial path parameters, ensuring dynamic matching between the path parameters and the rheological state of the concrete, the environment, and the equipment.

[0024] This enables the construction of a mapping model by combining key environmental and equipment parameters, and real-time correction of concrete rheological parameters to adapt them to the current printing conditions. This provides accurate data support for subsequent path optimization, helping to improve the stability of the printing process and the forming effect of the components.

[0025] Collect physical samples and build an experience cache library by combining them with simulation samples; dynamically adjust the ratio of the two types of samples according to the accumulation stage of physical samples, and train the reinforcement learning model in stages through a multi-stage training module. Furthermore, dynamically adjusting the ratio of the two types of samples based on the physical sample accumulation stage includes the following steps: Set preset thresholds S1 and S2 for the number of physical samples, with S1 being less than S2; divide the accumulation into three stages: the initial stage is when the number of physical samples is less than S1, the middle stage is when the number of physical samples is greater than or equal to S1 but less than S2, and the later stage is when the number of physical samples is greater than or equal to S2. The sampling ratios of simulation samples and physical samples were configured for the three stages respectively, namely 8:2, 5:5, and 2:8; Real-time statistics of the cumulative number of physical samples in the experience cache library to match the current accumulation stage; Simulation samples and physical samples are selected from the experience cache according to the sampling ratio corresponding to the current stage, and used for the phased training of the reinforcement learning model.

[0026] Specifically, a series of methods are employed to dynamically adjust the ratio of the two types of samples based on the physical sample accumulation stage, including setting a sample quantity threshold to divide the accumulation stage, configuring the corresponding sampling ratio for the configuration stage, implementing a real-time matching stage, and selecting samples according to the ratio. Input data includes a preset physical sample quantity threshold. , and the cumulative number of physical samples in the experience cache library ;in , Based on the sample accumulation patterns of historical printing projects and the pre-defined and satisfied training requirements of reinforcement learning models. The number of physical samples stored in the experience cache is obtained through real-time statistical analysis. Physical samples are complete data sets collected during actual printing processes, including operating condition parameters, rheological parameters, and printing effect feedback. Simulation samples are corresponding data sets generated through digital twin environments simulating different printing conditions. Together, they constitute the core data of the experience cache. The process is first divided into three stages of physical sample accumulation. At the beginning, when During the middle period, when Later in the process, the sampling ratios of simulated samples and physical samples were configured for the three stages as 8:2, 5:5, and 2:8 respectively, followed by real-time statistics. It matches the current accumulation stage and selects simulation samples and physical samples from the experience cache according to the sampling ratio corresponding to the current stage. During the selection process, the randomness and representativeness of the sample selection are ensured by combining sample index matching and random sampling. The output result is a set of simulation samples and physical samples combined according to the corresponding ratio, which provides training data for the reinforcement learning model at different sample accumulation stages, so that the model can gradually improve its adaptability and generalization performance to the actual printing scenario.

[0027] The ratio of simulation to physical samples is dynamically adjusted according to the accumulation stage of physical samples, and the reinforcement learning model is trained in stages to make the model adapt to the actual printing scenario, improve the generalization ability, and provide reliable training data support for subsequent accurate path optimization.

[0028] Configure the path optimization module to optimize the initial path parameters output by the reinforcement learning model after training through buffer smoothing and feedforward prediction, and adapt to the device action constraints. Furthermore, optimizing the initial path parameters output by the trained reinforcement learning model through buffer smoothing and feedforward prediction includes the following steps: Extract the coordinates of continuous path points and the corresponding nozzle movement speed parameters from the initial path parameters; The adjacent path points are buffered and smoothed, and the radius of curvature of the transition path is calculated so that the tangent angle between adjacent path segments is less than a preset threshold. Obtain the maximum acceleration and maximum velocity parameters from the device motion constraints, and calculate the estimated velocity change from the current path point to the next path point based on the feedforward prediction model; By combining the estimated values ​​of the curvature radius and velocity change of the transition segment, the moving speed and path point coordinates in the initial path parameters are adjusted to form the optimized path parameters.

[0029] Specifically, the input data includes the initial path parameters and device action constraint parameters output by the reinforcement learning model after training, and the coordinates of continuous path points in the initial path parameters. ; The path point number and the corresponding nozzle movement speed. The maximum acceleration in the device motion constraints is obtained by the reinforcement learning model based on the training data. With maximum speed Based on the preset mechanical performance parameters of the printing equipment, the first step is to extract... and Then, buffering and smoothing are applied to adjacent path points using the formula. Calculate the radius of curvature of the transition path. ;in Given the path arc length, use the vector dot product formula. Calculate the angle between the tangents of adjacent path segments. ,make Less than the preset path smoothing threshold Then, based on the feedforward prediction model, the formula is used. Calculate the estimated velocity change from the current path point to the next path point. ;in The movement time of adjacent path points is then combined. and , through formula ;in Adjust the moving speed according to the speed and curvature adaptation coefficient of the equipment, and use the coordinate fine-tuning formula. ;in For based on By determining the fine-tuning coefficients, the coordinates of the path points are adjusted to form the optimized path parameters. The output is used to verify and execute the device action constraints, so that the optimized path parameters meet the device operating limits and improve the smoothness of the printing path and the coordination of device operation.

[0030] By optimizing initial path parameters through buffer smoothing and feedforward prediction, path abruptness is reduced, equipment motion constraints are adapted, and path smoothness and equipment operation coordination are improved, providing adaptive path support for the stable and efficient development of concrete 3D printing.

[0031] Establish a safety control mechanism, define safe shutdown, degraded printing and normal printing modes, set judgment conditions and thresholds, and clarify priorities; Furthermore, setting separate judgment conditions and thresholds, and clarifying priorities, includes the following steps: Determine the criteria for safe shutdown, downgraded printing, and normal printing modes, including equipment fault level, concrete rheological parameter deviation, and degree of environmental parameter exceedance. Set thresholds for each judgment parameter: set the equipment fault level threshold to severe, moderate and slight; set the rheological parameter deviation value threshold to critical value, warning value and normal range; set the environmental parameter exceedance threshold to severe exceedance, slight exceedance and no exceedance. Define the judgment conditions for each mode: The safe shutdown mode corresponds to the equipment fault level of severe, or the rheological parameter deviation value ≥ the critical value, or the environmental parameters are seriously out of standard; the downgraded printing mode corresponds to the equipment fault level of medium, or the rheological parameter deviation value is between the warning value and the critical value, or the environmental parameters are slightly out of standard; the normal printing mode corresponds to the equipment fault level of minor, and the rheological parameter deviation value ≤ the warning value, and the environmental parameters are not out of standard. Clearly define the priority order: safe stop mode has the highest priority, followed by degraded printing mode, and normal printing mode has the lowest priority.

[0032] Specifically, a series of methods are used to separate judgment conditions and thresholds, clarify priorities, and determine judgment parameters, set parameter thresholds, define mode judgment conditions, and clarify priority ranking. Input data includes equipment fault levels. Deviance values ​​of concrete rheological parameters Degree of environmental parameters exceeding standards and the preset thresholds corresponding to each parameter; among which Based on a comprehensive assessment of equipment operating data collected by the equipment's sensors, including motor operating current, nozzle temperature deviation, and abnormal mechanical transmission noise, the system is classified into three levels: severe, moderate, and minor. ( The real-time corrected concrete rheological parameters are obtained from the rheological parameter correction steps. These are standard rheological parameters preset based on printing process requirements. By comparing the deviations of real-time temperature and humidity collected by environmental sensors with preset standard environmental parameters, the results are categorized into three levels: severely exceeding the standard, slightly exceeding the standard, and not exceeding the standard. The threshold values ​​for each parameter are preset based on the safety operation standards of concrete 3D printing equipment, the quality requirements of concrete printing process, and engineering practice experience. Among them, the rheological parameter deviation thresholds include critical values. Warning value And the normal range, first determine , , As three core judgment parameters, corresponding thresholds are set for each parameter, and then the judgment logic for each mode is defined: The safe shutdown mode satisfies: serious The requirements are severely exceeded; a downgraded printing mode is sufficient. medium Slightly exceeded the limit, but meets the requirements in normal printing mode; slight If the limits are not exceeded, the final priority order is determined to be: safe shutdown mode is higher than degraded printing mode, and printing is degraded. The mode is higher than the normal printing mode. The output result is the operating mode corresponding to the current printing condition and the execution rules under this mode. This result is used for the closed-loop adjustment of the printing process, providing a safety constraint basis for the real-time adjustment of path parameters, and ensuring that the printing process is carried out in an orderly manner under the premise of meeting safety standards.

[0033] By establishing a safety control mechanism that includes three printing modes, and clearly defining the judgment conditions, thresholds, and priorities, it can respond promptly to equipment changes and environmental anomalies during the printing process, providing layered safety assurance for the printing process.

[0034] The reinforcement learning model is lightweighted after training and deployed on an embedded platform; during printing, the sensor acquires state information in real time, and the path parameters are adjusted in a closed loop based on the safety mechanism and the corrected rheological parameters, and the optimized path is output through the lightweight inference engine.

[0035] Furthermore, based on the safety mechanism and the corrected rheological parameters, the closed-loop adjustment of the path parameters includes the following steps: Real-time acquisition of the current operating mode determined by the safety control mechanism and the corrected concrete rheological parameters; In safe stop mode, path parameter adjustment is stopped and a stop command is triggered; in degraded printing mode, the path parameter adjustment range is limited to a preset safe range; in normal printing mode, it is adjusted according to a dynamic optimization strategy. Based on the corrected rheological parameters, calculate the adjustment amounts for printing speed and path spacing in the path parameters; Verify whether the adjusted path parameters meet the device action constraints. If they do, update the path parameters; otherwise, recalculate the adjustment amount.

[0036] Specifically, a series of methods are employed to achieve closed-loop adjustment of path parameters based on safety mechanisms and corrected rheological parameters, including acquiring safe operating modes and corrected rheological parameters, formulating adjustment strategies according to the modes, calculating path parameter adjustment amounts, and verifying closed-loop operation. Input data includes the operating mode determined by the current safety control mechanism. Corrected concrete rheological parameters Current path parameters, device action constraint parameters, and preset safety zone; path parameters include printing speed. and path spacing ;in The value is determined by the security control mechanism and is either a safe shutdown mode, a degraded printing mode, or a normal printing mode. Obtained from the real-time correction step of rheological parameters, and The maximum speed in the device motion constraint parameters is the path parameter currently being executed. Maximum acceleration and minimum path spacing Based on the mechanical performance of the printing equipment, the preset safety range includes the printing speed safety range. Safe interval between paths Based on the printing process requirements and equipment safety operation standards, the adaptation coefficient in the dynamic optimization strategy is determined. , This was obtained through statistical analysis of historical print data, firstly acquired in real time. and Then press Execute the corresponding adjustment strategy: when In safe shutdown mode, path parameter adjustment is stopped and a shutdown command is triggered. When downgrading to print mode, the path parameter adjustment range is limited to a preset safe range. In normal printing mode, adjustments are made according to a dynamic optimization strategy, followed by... Through formula Calculate the print speed adjustment amount Path spacing adjustment amount ;in To obtain the adjusted printing speed based on the standard rheological parameters preset for the printing process. Path spacing Final verification Does it meet the requirements? and , Does it meet the requirements? And it meets the range restrictions of the corresponding mode. The minimum operating speed of the device is set; if all conditions are met, the path parameters are updated. If it does not meet the requirements, recalculate. and The output is the updated path parameters after successful verification. This result is input into the lightweight inference engine, and after inference processing, the final optimized path is output to guide the real-time action execution of the printing device, ensuring the dynamic adaptation of path parameters with safety mechanisms, rheological states and device constraints.

[0037] This enables the closed-loop adjustment of printing speed and path spacing based on the operating mode determined by the safety mechanism and the corrected rheological parameters, verifies the action constraints of the adapted equipment, realizes dynamic optimization of path parameters, and outputs the optimized path adapted to the working conditions through the lightweight inference engine, so that the printing process can proceed safely, stably and efficiently.

[0038] Example 2: In a construction project, it is necessary to use concrete 3D printing technology to fabricate irregularly shaped curved decorative components. These components have complex shapes and large variations in surface curvature, leading to technical problems during construction, including poor adaptability of path parameters to real-time working conditions, insufficient component forming accuracy, delayed response to abnormal printing processes, and high safety risks. To address these issues, a reinforcement learning-based concrete 3D printing path optimization method provided in this invention was adopted. This method achieves path optimization through the collaborative operation of a data acquisition and framework construction module, a rheological model construction module, a sample training and adaptation module, a path parameter optimization module, a safety mechanism configuration module, and a closed-loop output deployment module. The steps are as follows: Figure 2 As shown. The specific implementation process of this method is as follows: First, the data acquisition and framework construction module is activated. The structural parameters of the components, the operating parameters of the printing equipment, and the environmental parameters of the construction site are collected through the equipment sensors. After outlier removal, missing value supplementation, unit unification, and structured splicing preprocessing, the data is converted into state information. At the same time, a digital twin environment and a reinforcement learning model framework containing a multi-stage training module, a lightweight inference engine, and a constraint adaptation module are constructed. The digital twin environment generates path optimization simulation samples to provide standardized, high-quality input data and a basic framework for subsequent model construction and training. Next, through the rheological model building module, key parameters such as temperature, humidity, nozzle moving speed, and extrusion pressure are extracted based on state information. A mapping relationship model is built using historical printing data to calculate and correct concrete rheological parameters in real time, so as to achieve the effect of dynamically adapting rheological parameters to the current printing conditions. Then, the sample training and adaptation module is activated to collect physical samples in the actual printing process. An experience cache library is built by combining simulation samples. The ratio of the two types of samples is dynamically adjusted according to the accumulation stage of physical samples. The reinforcement learning model is trained in stages through the multi-stage training module to improve the model's adaptability to actual printing scenarios and its generalization performance. Then, through the path parameter optimization module, the initial path parameters output by the trained model are processed to extract the coordinates of continuous path points and the nozzle movement speed. After buffering and smoothing, the path abruptness is reduced. Combined with the speed change prediction value of feedforward prediction, the parameters are adjusted to achieve the effect of optimizing the initial path parameters and adapting to the equipment action constraints. At the same time, the safety mechanism configuration module is activated to establish a safety control mechanism, defining three modes: safe shutdown, degraded printing, and normal printing. The judgment conditions and thresholds for equipment failure level, rheological parameter deviation value, and environmental parameter exceedance are set separately, and the mode priority is clarified to achieve the effect of providing hierarchical safety protection for the printing process. Finally, by deploying a closed-loop output module, the trained reinforcement learning model is lightweighted and deployed on an embedded platform. During the printing process, the sensor acquires status information in real time. Based on the operating mode determined by the safety mechanism and the corrected rheological parameters, the path parameters such as printing speed and path spacing are adjusted in a closed loop. The optimized path is output through the lightweight inference engine, achieving the effect of dynamic adaptation of path parameters and ensuring the safe, stable and efficient progress of the printing process.

[0039] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A path optimization method for concrete 3D printing based on reinforcement learning, characterized in that, Includes the following steps: The equipment sensors collect structural, equipment, and environmental parameters, which are preprocessed into state information. A digital twin environment and reinforcement learning model framework are constructed simultaneously, including a multi-stage training module, a lightweight inference engine, and a constraint adaptation module, which are adapted for embedded deployment. The digital twin environment generates path optimization simulation samples. A multi-factor coupled rheological proxy model is established based on state information to associate the environment and equipment state and correct rheological parameters in real time. Collect physical samples and build an experience cache library by combining them with simulation samples; dynamically adjust the ratio of the two types of samples according to the accumulation stage of physical samples, and train the reinforcement learning model in stages through a multi-stage training module. Configure the path optimization module to optimize the initial path parameters output by the reinforcement learning model after training through buffer smoothing and feedforward prediction, and adapt to the device action constraints. Establish a safety control mechanism, define safe shutdown, degraded printing and normal printing modes, set judgment conditions and thresholds, and clarify priorities; The trained reinforcement learning model is lightweighted and deployed on an embedded platform. During printing, sensors acquire status information in real time. Based on safety mechanisms and corrected rheological parameters, path parameters are adjusted in a closed loop, and an optimized path is output through a lightweight inference engine.

2. The method for optimizing the path of concrete 3D printing based on reinforcement learning according to claim 1, characterized in that, The preprocessing of the status information includes the following steps: Use the 3σ criterion or box plot method to remove outliers from the original parameters and eliminate redundant and duplicate data. Missing values ​​in the original parameters can be supplemented using linear interpolation or moving average methods. Min-Max standardization was used to unify the dimensions of the data after removing redundant duplicates and supplementing missing values ​​in the original parameters. The structural parameters, equipment state parameters, and environmental parameters, after unifying their dimensions, are structurally spliced ​​together to form a state vector.

3. The method for optimizing the path of concrete 3D printing based on reinforcement learning according to claim 1, characterized in that, The real-time correction of rheological parameters in relation to the associated environment and equipment status includes the following steps: Extract environmental parameters such as temperature and humidity from the status information, and nozzle movement speed and extrusion pressure from the equipment status information; A mapping model between environmental equipment status and rheological parameters is constructed based on historical printing data. The mapping model adopts multiple nonlinear regression or LSTM neural network. Real-time data collection of current temperature, humidity, nozzle movement speed, and extrusion pressure; input of these data into the mapping model to calculate rheological parameter corrections; Adjust the current concrete rheological parameters according to the correction amount.

4. The method for optimizing the path of concrete 3D printing based on reinforcement learning according to claim 1, characterized in that, The dynamic adjustment of the ratio of the two types of samples based on the physical sample accumulation stage includes the following steps: Set preset thresholds S1 and S2 for the number of physical samples, with S1 being less than S2; divide the accumulation into three stages: the initial stage is when the number of physical samples is less than S1, the middle stage is when the number of physical samples is greater than or equal to S1 but less than S2, and the later stage is when the number of physical samples is greater than or equal to S2. The sampling ratios of simulation samples and physical samples were configured for the three stages respectively, namely 8:2, 5:5, and 2:8; Real-time statistics of the cumulative number of physical samples in the experience cache library to match the current accumulation stage; Simulation samples and physical samples are selected from the experience cache according to the sampling ratio corresponding to the current stage, and used for the phased training of the reinforcement learning model.

5. The method for optimizing the path of concrete 3D printing based on reinforcement learning according to claim 1, characterized in that, The process of optimizing the initial path parameters output by the reinforcement learning model after training through buffer smoothing and feedforward prediction includes the following steps: Extract the coordinates of continuous path points and the corresponding nozzle movement speed parameters from the initial path parameters; The adjacent path points are buffered and smoothed, and the radius of curvature of the transition path is calculated so that the tangent angle between adjacent path segments is less than a preset threshold. Obtain the maximum acceleration and maximum velocity parameters from the device motion constraints, and calculate the estimated velocity change from the current path point to the next path point based on the feedforward prediction model; By combining the estimated values ​​of the curvature radius and velocity change of the transition segment, the moving speed and path point coordinates in the initial path parameters are adjusted to form the optimized path parameters.

6. The method for optimizing the path of concrete 3D printing based on reinforcement learning according to claim 1, characterized in that, The separate determination conditions and thresholds, and the clarification of priorities, include the following steps: Determine the criteria for safe shutdown, downgraded printing, and normal printing modes, including equipment fault level, concrete rheological parameter deviation, and degree of environmental parameter exceedance. Set thresholds for each judgment parameter: set the equipment fault level threshold to severe, moderate and slight; set the rheological parameter deviation value threshold to critical value, warning value and normal range; set the environmental parameter exceedance threshold to severe exceedance, slight exceedance and no exceedance. Define the judgment conditions for each mode: The safe shutdown mode corresponds to the equipment fault level of severe, or the rheological parameter deviation value ≥ the critical value, or the environmental parameters are seriously out of standard; the downgraded printing mode corresponds to the equipment fault level of medium, or the rheological parameter deviation value is between the warning value and the critical value, or the environmental parameters are slightly out of standard; the normal printing mode corresponds to the equipment fault level of minor, and the rheological parameter deviation value ≤ the warning value, and the environmental parameters are not out of standard. Clearly define the priority order: safe stop mode has the highest priority, followed by degraded printing mode, and normal printing mode has the lowest priority.

7. The method for optimizing the path of concrete 3D printing based on reinforcement learning according to claim 1, characterized in that, The closed-loop adjustment of path parameters based on the safety mechanism and the corrected rheological parameters includes the following steps: Real-time acquisition of the current operating mode determined by the safety control mechanism and the corrected concrete rheological parameters; In safe stop mode, path parameter adjustment is stopped and a stop command is triggered; in degraded printing mode, the path parameter adjustment range is limited to a preset safe range; in normal printing mode, it is adjusted according to a dynamic optimization strategy. Based on the corrected rheological parameters, calculate the adjustment amounts for printing speed and path spacing in the path parameters; Verify whether the adjusted path parameters meet the device action constraints. If they do, update the path parameters; otherwise, recalculate the adjustment amount.

8. A path optimization system for concrete 3D printing based on reinforcement learning, characterized in that, The system for a reinforcement learning-based concrete 3D printing path optimization method according to any one of claims 1-7, the system comprising: The data acquisition and framework construction module is used to collect structural, equipment, and environmental parameters from the device sensors and preprocess them into state information. It also simultaneously constructs a digital twin environment and reinforcement learning model framework, including a multi-stage training module, a lightweight inference engine, and a constraint adaptation module, which are adapted for embedded deployment. The digital twin environment generates path optimization simulation samples. The rheological model building module is used to build a multi-factor coupled rheological proxy model based on state information, associate the environment and equipment state, and correct rheological parameters in real time. The sample training and adaptation module is used to collect physical samples and build an experience cache library by combining them with simulation samples; the ratio of the two types of samples is dynamically adjusted according to the accumulation stage of physical samples, and the reinforcement learning model is trained in stages through the multi-stage training module. The path parameter optimization module is used to configure the path optimization module. It optimizes the initial path parameters output by the reinforcement learning model after training through buffer smoothing and feedforward prediction to adapt to device action constraints. The security mechanism configuration module is used to establish a security control mechanism, define safe shutdown, degraded printing and normal printing modes, set judgment conditions and thresholds, and clarify priorities; A closed-loop output module is deployed to perform lightweight processing of the trained reinforcement learning model and is deployed on an embedded platform. During printing, the sensor acquires state information in real time, and based on the safety mechanism and the corrected rheological parameters, the path parameters are adjusted in a closed loop, and the optimized path is output through the lightweight inference engine.

Images