Concrete 3D printing path optimization method and system based on reinforcement learning

Abstract

The invention discloses a concrete 3D printing path optimization method and system based on reinforcement learning, and the method comprises the following steps: representing a to-be-printed geometric model as a graphic structure composed of nodes and edges, each node being a specific position in a three-dimensional space on a printing surface, and the edge representing a feasible moving path of a printing head between the nodes; performing dynamic optimization on the movement of the spray head through a Q-learning algorithm, performing iterative calculation on a Q value, and completing dynamic adjustment on a printing path on the basis of an instant reward to obtain an optimal printing path with the minimum movement distance, the minimum sharp turn frequency and the minimum start-stop frequency; the optimal printing path data is converted into a standard G code, and the 3D printer is controlled to execute the optimal printing path based on the standard G code; by combining the solving process of the optimal Q value in the Q-learning algorithm and the construction requirements of concrete 3D printing path planning, the accurate optimal printing path planning can be efficiently completed through the self-learning-self-evolution algorithm advantages of the reinforcement learning model.

Description

Technical Field

[0001] The present invention belongs to the fields of intelligent construction and additive manufacturing, and particularly relates to a method and system for optimizing the concrete 3D printing path based on reinforcement learning. Background Technique

[0002] At the current stage, efforts will be made to build a modern construction industry system, continuously improve the market operation mechanism, improve the quality and safety supervision system, and promote the coordinated development of intelligent construction and new building industrialization. By promoting the deep integration of digital technologies and the application of green and low-carbon technologies, the industry's production methods will be innovated and energy efficiency will be upgraded, comprehensively enhancing the core competitiveness of the construction industry and driving the industry to develop to a higher level. As a landmark technology of the "Third Industrial Revolution", 3D printing is widely used in industrial design, aerospace, engineering construction and other fields, bringing a huge impact to traditional production technologies. 3D printing is based on a three-dimensional digital model (CAD), regarding a three-dimensional object as a two-dimensional layered structure, and manufacturing an object by stacking materials layer by layer, so it is also called additive manufacturing. With the continuous improvement of the technical level of building design and construction, and the continuous development of prefabricated buildings, the traditional method of mixing and pouring concrete on-site has gradually been replaced by prefabricated components and prefabricated buildings to a large extent. Given the advantages of high automation, high precision, strong consistency, and fast working efficiency demonstrated by 3D printing technology, applying 3D printing technology to the construction industry, using special concrete materials, and realizing layer-by-layer stacking through computer-controlled mechanical devices can quickly manufacture complex and non-standard geometric components, greatly improving the applicability of construction projects, and opening up a brand-new development space for future building design and forming manufacturing, with broad application prospects. Chinese Patent with Publication No. CN113733295A discloses a method for optimizing the path of concrete 3D printing, which focuses on optimizing the 3D printing path through graph theory principles and ant colony algorithms to improve the forming speed and quality. However, this patent does not concern the curing time and mechanical properties of concrete materials during the 3D printing process, and does not solve the specific impact of concrete material properties on the optimization of the 3D printing path. Chinese Patent with Publication No. CN108995220A discloses a method for planning the 3D printing path of complex thin-walled structure objects based on reinforcement learning, but it focuses on optimizing the 3D printing path through the Q-learning algorithm to improve the printing efficiency and forming effect, does not concern the specific technical challenges that may be encountered in the printing of complex thin-walled structure objects by traditional path planning methods, and does not solve the problems of stress concentration, poor material heat dissipation effect, product deformation, cracks, etc. existing in the printing of complex thin-walled structure objects by traditional path planning algorithms.

[0003] In addition to having high requirements for compressive strength like conventional concrete, the interlayer bond strength is also a necessary condition for maintaining the structural stability of 3D printed concrete. On the one hand, the printing material requires a long setting time to obtain good fluidity and extrudability, while on the other hand, it also needs to shorten the setting time to gain sufficient early strength. Therefore, the setting time is one of the important parameters in the study of the performance indicators of 3D printing materials and is also an index that needs to be considered with emphasis during printing. In traditional 3D printing path planning research, the focus is on preventing collisions between the printing equipment and the printed parts, so only the coverage rate of the printing path and the non-repetition of the path are considered. However, this far from meets the requirements of the printing efficiency of building components and the quality of the finished products, nor can it give full play to the technical advantages of concrete 3D printing in aspects such as improving material utilization rate, avoiding resource waste, enhancing operation efficiency, and shortening the construction period. Summary of the Invention

[0004] To solve the deficiencies existing in the prior art and give full play to the technical advantages of concrete 3D printing, the present invention provides a method for optimizing the 3D printing path of concrete based on reinforcement learning.

[0005] In view of the special properties of 3D printed concrete materials, the design indicators that need to be focused on in the 3D printing path planning of concrete are analyzed, and a reward function is constructed based on these constraints to achieve autonomous planning of the optimal path based on reinforcement learning. To solve the problem of material condensation and accumulation, it is required that the planned printing path does not have path overlaps while covering comprehensively; to improve the printing efficiency, it is required that the start and stop times of the printing nozzle are as few as possible and the idle travel is shortened; to enhance the quality of the finished product, the number of turns and the turning angle of the printing path should be as small as possible to reduce the impact on the quality of the finished product caused by uneven material extrusion during turning. The present invention designs reward functions according to the above multiple constraints respectively, and plans the influence factors of each constraint, further improving the generalization performance of the reinforcement learning model, so that the path planning model based on the Q-learning algorithm can adapt to models of different shapes by dynamically adjusting the influence factors of each constraint according to the complexity and structural characteristics of the printed components.

[0006] After undergoing a large number of iterative trainings, the path planning model based on Q-learning can autonomously learn the optimal 3D printing paths under various different sliced models. This method applies reinforcement learning technology to autonomously generate the optimal printing path. By considering constraints such as the turning angle of the path and the idle travel, it evaluates all possible path situations between each node in the printing graph to ensure that the printing path is smooth and collision-free, and minimizes the number of printing interruptions, saves printing time, avoids structural defects, and ensures the smooth transition and optimal deposition of materials.

[0007] To achieve the above object, in a first aspect, the present invention provides a method for optimizing the 3D printing path of concrete based on reinforcement learning, comprising the following steps: Represent the geometric model to be printed as a graph structure composed of nodes and edges, where each node represents a specific position in three-dimensional space on the printing surface, i.e., the point where the concrete will be extruded, and the edges represent the feasible movement paths of the print head between the nodes; Dynamically optimize the movement of the nozzle through the Q-learning algorithm, iteratively calculate the Q value, and autonomously complete the dynamic adjustment of the printing path based on the immediate reward to obtain an optimal printing path that minimizes the moving distance, minimizes the number of sharp turns, and minimizes the number of start and stop times; Convert the optimal printing path data into standard G-code, and control the 3D printer to execute the optimal printing path based on the standard G-code.

[0008] Further, when dynamically optimizing the movement of the nozzle through the Q-learning algorithm, the state is defined as the nozzle position, the action is defined as the movement between the graph nodes, and a reward function is used to evaluate the printing efficiency, angle smoothness, and stroke continuity.

[0009] Further, dynamically update the structure diagram. Specifically, initially, all nodes and edges are in an available state. When a certain node is printed, the printed node is marked as a traversed point, and the edges connected to the printed node are removed.

[0010] Further, when dynamically optimizing the movement of the nozzle through the Q-learning algorithm and iteratively calculating the Q value to autonomously complete the dynamic adjustment of the printing path based on the immediate reward, two methods are used for action selection: 1) Action selection at the turning point. When the print head is at a corner or acute angle position, determine all feasible next actions, and preferentially select a smoothly transitioning path through the turning reward and Q value; 2) Action selection for interlayer conversion is used to determine the available operation action selection when converting between layers, guided by the greedy strategy, where the printer selects the action with the highest Q value and explores other actions according to a certain probability.

[0011] Further, assign a Q value to each state-action pair. The Q value represents the expected future reward for taking a certain action from a state. Select the next node with the highest Q value or randomly explore other actions. The Q value is updated as the printer nozzle moves. Maximize the cumulative reward value through repeated iteration and path update. When selecting a path each time, comprehensively consider angle optimization and empty travel. The learning process will continue until the best action is found. After multiple Q iterations, it will converge to a stable value Q, and the best action is the finally learned turning strategy.

[0012] Further, the selected path comprehensively considers angle optimization and idle travel, including: the reward for the turning action is a negative reward related to the turning angle. The print head selects the action with a larger reward value, and the turning angle of the print head is smaller. If the angle is very large, the action of selecting a smoother turn will be rewarded; the start-stop action is set as a negative reward value, and the Euclidean distance between the two nodes before and after the idle travel is used as the judgment criterion. The reward value is inversely proportional to the distance between the current node and the target node, and the shorter the distance, the higher the reward. d It is inversely proportional, and the shorter the distance, the higher the reward.

[0013] In a second aspect, the present invention can also provide a concrete 3D printing system, which uses the above-mentioned concrete 3D printing path optimization method based on reinforcement learning to optimize the printing path and execute the printing action.

[0014] In a third aspect, the present invention provides a concrete 3D printing path optimization system based on reinforcement learning, including a composition module, an optimization module, and an instruction conversion and execution module; The composition module is used to represent the geometric model to be printed as a graph structure composed of nodes and edges. Each node represents a specific position in the three-dimensional space on the printing surface, that is, the point where the concrete will be extruded, and the edge represents the feasible movement path of the print head between the nodes; The optimization module is used to dynamically optimize the movement of the nozzle through the Q-learning algorithm, iteratively calculate the Q value, and autonomously complete the dynamic adjustment of the printing path based on the immediate reward to obtain the optimal printing path that minimizes the movement distance, minimizes the number of sharp turns, and minimizes the number of start-stops; The instruction conversion and execution module is used to convert the optimal printing path data into standard G-code and control the 3D printer to execute the optimal printing path based on the standard G-code.

[0015] At the same time, a computer device is provided, including a processor and a memory. The memory is used to store computer-executable programs. The processor reads part or all of the computer-executable programs from the memory and executes them. When the processor executes part or all of the computer-executable programs, it can implement the above-mentioned concrete 3D printing path optimization method based on reinforcement learning.

[0016] A computer-readable storage medium can also be provided. A computer program is stored in the computer-readable storage medium. When the computer program is executed by the processor, it can implement the above-mentioned concrete 3D printing path optimization method based on reinforcement learning.

[0017] Compared with the prior art, the present invention has at least the following beneficial effects: By combining the process of solving the optimal Q-value in the Q-learning algorithm with the construction requirements of the concrete 3D printing path planning, and leveraging the algorithmic advantages of self-learning and self-evolution of the reinforcement learning model, the present invention can efficiently complete precise optimal printing path planning, which has a significant impact on improving the 3D printing work efficiency and operation quality: By optimizing the path selection and reducing unnecessary movements, the printing time can be significantly shortened and the printing efficiency can be improved; By optimizing the interlayer path planning, the smooth transition of the print head and the interlayer bonding strength are ensured, effectively avoiding problems such as interlayer fracture or poor bonding; By adding a reward function that is inversely proportional to the number of turns and the turning angle, as well as a reward function related to restricting the start and stop times of the nozzle, the algorithm can make smoother path selections at turns and avoid sharp turning actions as much as possible, effectively reducing the defect of uneven accumulation of printing materials caused by turning, and improving the quality, precision, and stability of printing. The present invention constructs a path planning decision-making model based on the adaptive ability of Q-learning, which can dynamically adjust the printing path according to the immediate reward to adapt to the printing task requirements of different shapes and challenges in complex structures. By selecting the shortest and optimal paths, unnecessary movements and material waste can be minimized, saving energy consumption and resources. The G-code generated by the algorithm has good compatibility and can directly control the 3D printer, reducing manual intervention and improving the simplicity and automation of operation. The method for optimizing the concrete 3D printing path based on reinforcement learning provided by the present invention can significantly improve the efficiency, quality, and structural stability of concrete 3D printing, and has important application value and broad market prospects.

[0018] Furthermore, during the printing process, the structure diagram will be dynamically updated to avoid collisions between the print head and the printed part. When a certain node is printed, the printed node will be marked as a traversed point and the edges connected to it will also be removed, which can effectively prevent the print head from repeatedly accessing the printed positions and ensure the efficiency and integrity of the printing path.

[0019] Furthermore, motion instructions are generated according to the path points. The G0 instruction is responsible for the rapid movement of the print head, that is, the idle stroke, while the G1 instruction controls the extrusion movement of the print head between path points and precisely sets the extrusion amount and movement rate; By precisely controlling the basic rate and layer height of each node, it can be ensured that the concrete deposition of each layer is uniform and continuous, avoiding situations of excessive or insufficient extrusion of materials, thereby guaranteeing the finished product quality and structural stability. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 It is a technical flow chart of the method for optimizing the concrete 3D printing path based on reinforcement learning.

[0021] Figure 2It is a detailed flowchart for realizing the optimization of the concrete 3D printing path based on the Q-learning algorithm.

[0022] Figure 3 It is a diagram showing the representation method of the turning angle.

[0023] Figure 4 It is a sectional structure diagram of a concrete 3D printing model.

[0024] Figure 5 It is a demonstration diagram of the concrete 3D printing path simulation interface.

[0025] Figure 6 It is the optimal printing path result obtained by the concrete 3D printing path optimization method proposed based on the present invention.

[0026] Figure 7 It is the optimal printing path result calculated based on the ant colony algorithm.

[0027] Figure 8 It is the optimal printing path result calculated by the 3D printing path planning method for complex thin-walled structure objects based on reinforcement learning. Specific implementation manners

[0028] The invention describes an intelligent algorithm for optimizing the motion trajectory of the concrete 3D printer nozzle by using reinforcement learning technology, that is, a 3D printing path planning algorithm that can improve printing efficiency, reduce material waste, and ensure stable and continuous deposition of materials. Figure 1 The following shows the complete implementation process of 3D printing technology. First, the geometric structure of the model to be printed is obtained through 3D modeling and model slicing. Then, the optimal printing path plan is calculated based on the proposed 3D printing path planning method. Finally, G-code instructions are generated to control the movement of the 3D printer equipment and the uniform extrusion of materials.

[0029] Figure 2These are the specific operation steps for path optimization based on the Q-learning algorithm. The method uses a graph-based representation of the printing surface, where nodes represent key points and edges represent feasible paths for nozzle movement. By adopting Q-learning technology, dynamic optimization of the printhead movement is achieved. The state is defined as the printhead position, the action is defined as the movement between graph nodes, and a reward function is used to evaluate printing efficiency, angle smoothness, stroke continuity, etc. The algorithm iteratively calculates the Q-values and autonomously adjusts the print path dynamically based on immediate rewards, and can determine an optimal print path that minimizes the movement distance, the number of sharp turns, and the number of start / stop times. The optimal print path generated by the Q-learning algorithm significantly improves printing quality, printing efficiency, and material utilization rate, and can provide a robust, adaptable, and scalable solution for improving the performance of concrete 3D printing systems.

[0030] Step 1: Graphical Representation of Path Planning The path planning algorithm first represents the loaded geometric model as a graph structure composed of nodes and edges. Each node represents a specific position in three-dimensional space on the printing surface, that is, the point where concrete will be extruded, and the edges represent the feasible movement paths of the printhead between the nodes, forming a complete print path network. In the initial stage, the structure diagram is modeled as an undirected graph, as Figure 3 shown, allowing the printhead to move freely between nodes; to avoid collisions between the printhead and the printed part, the structure diagram will be dynamically updated during the printing process. Specifically, initially all nodes and edges are in an available state. When a certain node is printed, the printed node is marked as a traversed point and the edges connected to it are also removed. This design can effectively prevent the printhead from repeatedly accessing the printed positions, ensuring the efficiency and integrity of the print path. In addition, by dynamically updating the structure diagram, a reasonable and efficient path can be dynamically adjusted and planned in real time according to the changes in the printing state.

[0031] The print structure diagram is stored and managed through an adjacency matrix. The adjacency matrix is used to represent the connection relationship between nodes and is dynamically updated during the printing process. In the initial stage, the graph structure generated according to the input data includes the coordinates of each node and the connection paths between nodes. The adjacency matrix is a two-dimensional matrix A. Assuming there are N nodes in the print path, the size of the adjacency matrix is , where the matrix element is defined as when there is a connection between node and node , and when there is no connection between node and node , During the printing process, the adjacency matrix is dynamically adjusted according to the current printing status. When a certain node is printed, the node is marked as inactive, and the edges connected to it are also removed. For example, when node 2 is marked as printed, the edges connected to node 2 are removed, and the matrix is updated to This dynamic update mechanism can effectively avoid redundant path planning and ensure that the print head does not revisit the printed part. At the same time, by updating the connection matrix in real time, it can also adapt to the changing printing environment and always plan a reasonable, efficient, and collision-free printing path, further improving the printing efficiency and the quality of the finished product.

[0032]

[0033] Among them, the elements of the matrix represent whether there is an edge between node and node . 1 indicates the existence of an edge, 0 indicates the non-existence of an edge, and there is no passage path.

[0034]

[0035] Step 2: Q-learning and optimization (1) Q-learning framework The present invention is based on Q-learning, which is a model-free reinforcement learning that helps the print head decide which operation to take in each state based on the experience obtained in the past. The Q-learning algorithm maintains a Q-table, where each entry represents the expected future reward for a state-action pair.

[0036] State representation: The state of the printer includes the current node and the last visited node, and the printer updates its state when moving along the printing path.

[0037] Exploration and exploitation: The balance between exploration and exploitation is achieved through the greedy parameter in each action selection. As the algorithm learns, the efficiency of the printing path will be improved, so that the best actions will be utilized more and more.

[0038] Action selection: In each state, considering the immediate reward, the next action is selected according to the highest Q value. If there are multiple actions with the same Q value, actions can be randomly selected for exploration.

[0039] Reward function: The reward received after executing an action is used to update the Q value, and the reward comes from the angular deviation and the travel distance between nodes.

[0040] The update calculation of the Q value refers to the following Bellman equation:

[0041] Among them, Q(s,a) is the Q-value of the state-action pair (s,a), R(s,a) is the reward for taking action a in state s, λ is the discount factor, which is used to control the influence of future rewards on the calculation of Q-values. is the discount factor, which is used to balance immediate rewards and future rewards. α is the learning rate, which is used to control the degree to which new rewards overwrite prior knowledge.

[0042] This iterative process ensures that the printer gradually improves the path, and over time, learns to select the most efficient and accurate movements. The printer is responsible for managing the state of the printing process, tracking the current and previous positions of the nozzle, defining the possible actions of the printer, and collecting rewards based on the actions of the printer to adjust the path, ensuring smooth movement and reducing errors in the printed structure. The state of the printer is represented by a series of position nodes on the slicing geometry chart, and the actions correspond to the movements between these positions.

[0043] The printer uses two methods for action selection: 1) Action selection at turns. When the print head is at a corner or acute angle position, all feasible next actions are determined, and the path with a smooth transition is preferentially selected through the turning reward and Q-value to avoid structural defects and material deposition problems caused by sharp turns; 2) Action selection for interlayer transitions is used to determine the available operation actions when transitioning between layers. Guided by the greedy strategy, the printer selects the action with the highest Q-value, but explores other actions with a certain probability to ensure exploration during learning.

[0044] (2) Corner optimization In concrete 3D printing, actions at acute corners are likely to cause the path of the print head to be non-smooth, thereby affecting the material deposition efficiency and the structural integrity of the printed object. To solve this problem, corner optimization is proposed, aiming to improve the movement efficiency of the print head and reduce printing defects caused by sharp turns by optimizing the path planning at corners.

[0045] During the printing process, if the print head turns, the moving speed will decrease, and the greater the turning angle, the greater the speed reduction value. Therefore, the reward for the turning action is a negative reward related to the turning angle. The print head selects the action with a larger reward value, and its turning angle is smaller. If the angle is very large, it will reward the selection of a smoother turning action, which minimizes the possibility of inefficient movement of the printer or errors caused by sharp turns.

[0046] The current position of the print head is node v1, v0 is the previous node it visited, and the next action a is selected to node v2, where θ is the angle between vector v0v1 and vector v1v2, as Figure 3 shown.

[0047]

[0048] where θ is the angle between the vectors, and the reward is adjusted to optimize the path. represents the immediate reward when performing action a in state s. is pi.

[0049] Assign a Q-value to each state-action pair (s,a), which represents the expected future reward for taking a specific action from a certain state. Select the next node with the highest Q-value or randomly explore other actions. The Q-value is updated as the printer nozzle moves, and the learning process continues until the best action is found. After multiple Q-iterations, it converges to a stable value Q, which is the learned final turning strategy.

[0050] The path optimization of the print head at the corner is achieved through the Q-learning model, using the turning reward and the travel distance as the main reward mechanisms. The system selects the optimal action or explores new paths at each state to dynamically update the Q-value, ultimately achieving a smooth transition of the path and efficient printing.

[0051] (3) Start-stop optimization Too many start-stop actions performed by the print head will result in idle travel and also increase the printing time. To minimize the start-stop actions of the print head, the start-stop actions can be set as negative reward values. Use the Euclidean distance between the two nodes before and after the idle travel as the judgment criterion, and the reward value is d inversely proportional to the distance between the current node and the target node. The shorter the distance, the higher the reward.

[0052]

[0053] This function evaluates the distances between different nodes, optimizes the route to minimize idle travel, and avoids unnecessary detours. By selecting the shortest path between nodes, the overall printing time and energy consumption are reduced. The reward is calculated based on the distance between the current position and the next position. The shorter the distance, the higher the reward. The Q-value is updated after each iteration to improve the path selection.

[0054] The distances between nodes are also incorporated into the path planning. Longer travel distances are affected to reduce unnecessary movements, optimize material deposition, and minimize the overall printing time. This is particularly important in concrete 3D printing because excessive travel between distant points may waste time and cause inconsistent extrusion, thereby affecting the final quality of the printed structure.

[0055] (4) Path search The path search algorithm realizes dynamic path optimization through the Q-learning reinforcement learning mechanism, optimizes the turns and starts / stops in the path respectively, and finally determines the optimal printing path. Its main goal is to maximize the cumulative reward value through repeated iteration and path update to ensure the efficient movement of the print head and printing quality. The path search process starts from a predefined starting node and is updated in each iteration to explore the state of the path and mark it as deactivated to prevent repeated access to the printing area. This mechanism can effectively avoid path redundancy and improve search efficiency. Each selected path comprehensively considers angle optimization and idle travel to ensure that the path of the print head is both smooth and efficient. After multiple iterations, the Q value gradually converges, and finally the optimal path is found. This path not only maximizes the reward in path selection but also ensures the continuity of the path, thus realizing an efficient printing process.

[0056] Step 3: G-code generation After determining the best path for each layer, the system converts the path data into the standard G-code format for controlling the movement and extrusion process of the 3D printer. The generation of G-code includes a series of instructions to ensure that the print head moves efficiently along the optimized path. The system generates movement instructions based on the path points. The G0 instruction is responsible for the rapid movement of the print head, that is, the idle travel, while the G1 instruction controls the extrusion movement of the print head between the path points and precisely sets the extrusion amount and movement rate. For each layer, the system updates the z coordinate to ensure that the print head moves to the correct layer position and smoothly transitions to the next layer. The system dynamically adjusts the extrusion rate of the material according to the travel distance and extrusion coefficient between the path points to ensure the uniformity and continuity of concrete deposition. In the non-printing state, the system generates rapid movement instructions to reduce the idle travel time and improve the printing efficiency. The height, extrusion rate, and path planning of each layer are encoded into the G-code to ensure the consistency and stability of the printing process. The finally generated G-code can be directly used to control the 3D printer, enabling it to precisely execute the path planning in actual operation, realize efficient and continuous concrete deposition, and ensure the quality and integrity of the printed structure. Using the above method, for Figure 4 a model cross-section structure shown, the algorithm simulation of the printing path is carried out in the 3D printing software CURA, as Figure 5 shown, and the Figure 6 optimal printing path shown can be obtained. From the experimental data, the optimal printing path planned by the algorithm proposed in the present invention needs to experience 80 turns and 3 starts / stops in total. The first start / stop is the blue loop between the printing starting point, that is, the 1st marked point; the second start / stop is the green loop between the 2nd marked points; the third start / stop is the gray loop between the 3rd marked points. The idle lift travel is shown as the red dotted line in the figure.

[0057] In concrete 3D printing, deceleration is required during turning, especially sharp turns, to maintain path accuracy. On the one hand, this will lead to a reduction in the overall printing efficiency. On the other hand, due to the change in the nozzle movement speed, it will not match the material extrusion speed, resulting in complex situations such as material accumulation or insufficient extrusion, affecting the adhesion of the material layer and the stability of the overall structure. In addition, the number of starts and stops and the selection of start and stop points will also affect the printing speed and the quality of the finished product. Each start and stop requires restarting and extrusion, which not only takes a certain amount of time and leads to a decrease in the overall printing efficiency, but also the higher the start and stop frequency, the worse the continuity of the printed material, and it will seriously affect the integrated strength of the structure.

[0058] Therefore, for these two important evaluation indicators, the 3D printing path planning method based on reinforcement learning proposed in the present invention is compared with the traditional path planning method based on the ant colony algorithm and the 3D printing path planning method for complex thin-walled structure objects based on reinforcement learning. From Figure 6 、 Figure 7 and Figure 8 The simulation results show that: the optimal printing path planned by the algorithm proposed in the present invention needs to go through 80 turns and 3 starts and stops in total. The printing path planned by the ant colony algorithm needs to go through 81 turns and 4 starts and stops. The 3D printing path planning method for complex thin-walled structure objects based on reinforcement learning needs to go through 69 turns and 5 starts and stops. It can be seen that, whether compared in terms of the number of turns or the number of starts and stops, the method proposed in the present invention is better than the ant colony algorithm, and the number of starts and stops is less than that of the 3D printing path planning method for complex thin-walled structure objects based on reinforcement learning. Therefore, it also proves that the proposed 3D printing path planning method for concrete based on reinforcement learning is feasible and can be superior in improving the printing efficiency and the quality of the finished product.

[0059] On the basis of already providing the 3D printing path optimization method for concrete based on reinforcement learning, the present invention can also load the above method into a concrete 3D printing system, optimize the printing path based on the above 3D printing path optimization method for concrete based on reinforcement learning and execute the printing action.

[0060] Embodiment 2, based on the concept of the above method, the present invention also provides a 3D printing path optimization system for concrete based on reinforcement learning, including a composition module, an optimization module, and an instruction conversion and execution module; The composition module is used to represent the geometric model to be printed as a graph structure composed of nodes and edges. Each node represents a specific position in the three-dimensional space on the printing surface, that is, the point where the concrete will be extruded, and the edge represents the feasible movement path of the print head between the nodes; The optimization module is used to dynamically optimize the movement of the nozzle through the Q-learning algorithm, iteratively calculate the Q value, and autonomously adjust the printing path based on the immediate reward to obtain an optimal printing path that minimizes the movement distance, the number of sharp turns, and the number of start / stop operations. The instruction conversion and execution module is used to convert the optimal printing path data into standard G-code and control the 3D printer to execute the optimal printing path based on the standard G-code.

[0061] On the other hand, the present invention provides a computer-readable storage medium. A computer program is stored in the computer-readable storage medium. When the computer program is executed by a processor, the method for optimizing the concrete 3D printing path based on reinforcement learning according to the present invention can be implemented.

[0062] The present invention can also provide a computer device, including a processor and a memory. The memory is used to store computer-executable programs. The processor reads the computer-executable programs from the memory and executes them. When the processor executes the computer-executable programs, the method for optimizing the concrete 3D printing path based on reinforcement learning according to the present invention can be implemented.

[0063] The computer device can be a laptop computer, a desktop computer, or a workstation.

[0064] The processor can be a central processing unit (CPU), a graphics processing unit (GPU), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), or a field-programmable gate array (FPGA).

[0065] For the memory of the present invention, it can be an internal storage unit of a laptop computer, a desktop computer, or a workstation, such as a memory or a hard disk; it can also use an external storage unit, such as a mobile hard disk or a flash card.

[0066] Computer-readable storage media can include computer storage media and communication media. Computer storage media includes volatile and non-volatile, removable and non-removable media implemented by any method or technology for storing information such as computer-readable instructions, data structures, program modules, or other data. Computer-readable storage media can include: read-only memory (ROM, Read Only Memory), random access memory (RAM, Random Access Memory), solid state drives (SSD, Solid State Drives), or optical discs, etc. Among them, random access memory can include resistive random access memory (ReRAM, Resistance Random Access Memory) and dynamic random access memory (DRAM, Dynamic Random Access Memory).

Claims

1. A concrete 3D printing path optimization method based on reinforcement learning, characterized in that: The following steps are involved: The geometric model to be printed is represented as a graph structure consisting of nodes and edges, where each node represents a specific position in the three-dimensional space on the printing surface, i.e., the point where the concrete will be extruded, and the edge represents a feasible movement path of the printing head between the nodes; The Q-learning algorithm is used to dynamically optimize the movement of the nozzle, iteratively calculate the Q value, and dynamically adjust the printing path based on the instant reward to obtain the optimal printing path that minimizes the moving distance, the number of sharp turns, and the number of starts and stops. The optimal printing path data is converted into a standard G code, and the 3D printer is controlled to execute the optimal printing path based on the standard G code.

2. The method for optimizing concrete 3D printing path based on reinforcement learning according to claim 1, characterized in that: When dynamically optimizing the nozzle motion through the Q-learning algorithm, the state is defined as the nozzle position, the action is defined as the movement between graph nodes, and the reward function is used to evaluate the printing efficiency, angle smoothness, and stroke continuity.

3. The method for optimizing concrete 3D printing path based on reinforcement learning according to claim 1, characterized in that: The structure graph is updated dynamically. Specifically, all nodes and edges are in an available state initially. When a node is printed, the printed node is marked as a traversed point, and the edges connected to the printed node are removed.

4. The method for optimizing concrete 3D printing path based on reinforcement learning according to claim 1, characterized in that: The Q-learning algorithm is used to dynamically optimize the movement of the nozzle, iteratively calculate the Q value, and autonomously complete the dynamic adjustment of the printing path based on the instant reward. Two methods are used to select actions: 1) Action selection at the corner. When the print head is at a corner or sharp angle, all feasible next actions are determined, and the path with smooth transition is preferentially selected through the turning reward and Q value; 2) Action selection for inter-layer transitions is used to determine the available action choices when transitioning between layers, guided by a greedy strategy where the printer selects the action with the highest Q-value and explores other actions according to a certain probability.

5. The method for optimizing concrete 3D printing path based on reinforcement learning according to claim 1, characterized in that: A Q value is assigned to each state-action pair, and the Q value represents the expected future reward of taking a certain action from a state. The next node with the highest Q value is selected or other actions are randomly explored. The Q value is updated as the printer nozzle moves. The cumulative reward value is maximized through repeated iterations and path updates. Each time a path is selected, angle optimization and empty stroke are comprehensively considered. The learning process will continue until the best action is found. After multiple iterations, Q will converge to a stable value Q. The best action is the final turning strategy learned.

6. The method for optimizing concrete 3D printing path based on reinforcement learning according to claim 5, characterized in that: The path selection comprehensively considers angle optimization and empty travel, including: the reward for the turning action is a negative reward related to the turning angle. The print head selects an action with a larger reward value, and the turning angle of the print head is smaller. If the angle is large, the action of selecting a smoother turn will be rewarded; the start and stop actions are set to negative reward values, and the Euclidean distance between the two nodes before and after the empty travel is used as the judgment standard. The reward value is related to the distance between the current node and the target node. d Inversely proportional, the shorter the distance, the higher the reward.

7. A concrete 3D printing system, characterized in that: Based on the reinforcement learning-based concrete 3D printing path optimization method described in any one of claims 1-6, the printing path is optimized and the printing action is executed.

8. A concrete 3D printing path optimization system based on reinforcement learning, characterized in that: It includes a composition module, an optimization module and an instruction conversion execution module; The composition module is used to represent the geometric model to be printed as a graph structure composed of nodes and edges, where each node represents a specific position in the three-dimensional space on the printing surface, that is, the point where the concrete will be extruded, and the edge represents the feasible movement path of the print head between the nodes; The optimization module is used to dynamically optimize the movement of the nozzle through the Q-learning algorithm, iteratively calculate the Q value, and autonomously complete the dynamic adjustment of the printing path based on the instant reward to obtain the optimal printing path that minimizes the moving distance, minimizes the number of sharp turns, and minimizes the number of starts and stops; The instruction conversion execution module is used to convert the optimal printing path data into standard G code, and control the 3D printer to execute the optimal printing path based on the standard G code.

9. A computer device, characterized in that: The invention comprises a processor and a memory, wherein the memory is used to store a computer executable program, and the processor reads part or all of the computer executable program from the memory and executes it. When the processor executes part or all of the computer executable program, the method for optimizing the concrete 3D printing path based on reinforcement learning according to any one of claims 1 to 6 can be implemented.

10. A computer-readable storage medium, characterized in that: A computer program is stored in the computer-readable storage medium. When the computer program is executed by the processor, the method for optimizing the concrete 3D printing path based on reinforcement learning according to any one of claims 1 to 6 can be implemented.

Images