A laser cutting path planning method based on reinforcement learning

Abstract

This invention discloses a laser cutting path planning method based on reinforcement learning, comprising: S1, establishing a task scenario model; S2, performing rasterization processing based on the spatial state matrix to construct a cutting state vector; S3, inputting the cutting state vector into a PathFormer network to extract spatial topological features and global semantic information of the path, generating a high-dimensional path feature embedding; S4, constructing a multi-objective reward function based on the feature embedding and candidate path actions, and calculating the instantaneous reward value; S5, training a policy network using an improved APPO algorithm, updating the Actor and Critic parameters to jointly optimize path length, energy consumption, and hot zone distribution; S6, mapping the optimal path sequence back to the actual coordinate system to generate control commands, driving the laser head to perform cutting according to the planned path. This invention achieves adaptive planning and energy-balanced control of the laser cutting path, improving cutting efficiency, accuracy, and processing stability.

Description

Technical Field

[0001] This invention relates to the field of reinforcement learning and path optimization control, and in particular to a laser cutting path planning method based on reinforcement learning. Background Technology

[0002] With the rapid development of intelligent manufacturing and precision machining technologies, laser cutting, as an important means of processing metal and non-metal materials, is widely used in fields such as automobile manufacturing, aerospace, shipbuilding, and electronic device processing. Traditional laser cutting path planning methods mostly rely on fixed rules, heuristic search, or offline optimization algorithms to generate paths through preset geometric constraints and processing sequences. However, in actual cutting environments, the shapes of sheet metal are complex, the layout of parts is varied, and the distribution of heat-affected zones is dynamically changing. Traditional algorithms lack adaptability when dealing with complex working conditions and struggle to balance processing efficiency, energy consumption distribution, and cutting quality.

[0003] Currently, most path planning methods are still based on static models, lacking feedback on the real-time status of the cutting process and unable to automatically adjust the path strategy according to equipment operating parameters and thermal energy changes. Some methods only optimize the path length or processing time, ignoring the comprehensive constraints of multi-dimensional objectives such as energy consumption, thermal zone equalization, and material deformation, resulting in short local paths but high overall energy consumption or excessive heat concentration. At the same time, existing systems generally use offline calculation methods, making it difficult to dynamically correct the planning results during processing, which reduces the system's adaptive control capability.

[0004] Furthermore, traditional algorithms mostly rely on empirical parameter settings or single objective function design, failing to introduce intelligent optimization frameworks with self-learning capabilities. This makes it difficult to continuously improve cutting strategies in complex, nonlinear, and time-varying environments. Especially in multi-path, multi-workpiece collaborative cutting tasks, the effects of path intersection and thermal zone superposition are significant. Conventional planning methods cannot effectively assess the energy coupling and heat-affected zone diffusion patterns between different paths, leading to unstable cutting quality, increased risk of sheet metal thermal deformation, and decreased energy efficiency, severely limiting the intelligent and efficient development of laser cutting equipment.

[0005] Therefore, how to provide a laser cutting path planning method based on reinforcement learning is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0006] One objective of this invention is to propose a laser cutting path planning method based on reinforcement learning. This invention utilizes the PathFormer network to extract spatial topological features and global semantic information of the cutting state vector. It models the dependencies between path segments through a multi-head attention mechanism, achieving a high-dimensional representation of the sheet metal geometry and heat-affected zone features. Subsequently, based on an improved APPO algorithm, combined with policy uncertainty regularization and state entropy enhancement terms, the Actor and Critic networks are trained asynchronously in a multi-threaded manner, forming a policy update structure that jointly optimizes path length, energy consumption, and heat zone distribution. This invention establishes a reinforcement learning decision framework integrating high-dimensional modeling of path features, policy distribution constraints, and dynamic adjustment of exploration intensity. It enables precise control and energy consumption optimization of laser head path planning in complex cutting environments, significantly improving cutting accuracy, processing stability, and energy utilization. This effectively solves the problems of single path planning, unstable policy updates, and decreased cutting quality caused by uneven heat distribution in existing laser cutting methods.

[0007] A laser cutting path planning method based on reinforcement learning according to an embodiment of the present invention includes the following steps: S1. Collect the operating parameters of the laser cutting equipment and the geometric information of the workpiece, establish a laser cutting task scenario model, and output the spatial state matrix; S2. Based on the spatial state matrix, the board area is rasterized and the uncut area is marked. The state vector is defined and the cutting state vector is constructed. S3. Input the cutting state vector into the PathFormer network, model the relationship and sequence dependency between different path segments through the multi-head attention mechanism, extract spatial topological features and global semantic information of the path, and output high-dimensional path feature embedding. S4. Based on the high-dimensional path feature embedding and each candidate path action, calculate the path length, turning angle, cutting energy consumption and hot zone distribution, construct a multi-objective reward function, and output the instant reward value. S5. Based on the cutting state vector, high-dimensional path feature embedding, and immediate reward value, train the policy network using the improved APPO algorithm, update the Actor and Critic parameters based on multi-threaded asynchronous experience replay, and output the optimal cutting path sequence. S6. Map the optimal cutting path sequence back to the actual coordinate system of the laser cutting equipment, generate laser cutting control commands, and drive the laser head to perform cutting operations according to the planned path.

[0008] Optionally, S1 specifically includes: S11. Read the operating parameters of the laser cutting equipment through the data acquisition device to obtain the current values ​​of cutting speed, acceleration, power and jet pressure. Import the CAD drawings or 3D model files of the workpiece using geometric modeling to analyze the outline of the plate and the position and size information of the parts to be cut. S12. Synchronize the collected operating parameters and the analyzed geometric information according to a unified timestamp to form a set of status data of the laser cutting equipment and the workpiece; S13. On the computer, call the scene modeling, establish a laser cutting task scene model based on the state data set, divide the plate plane into coordinate grid areas, record the corresponding geometric boundary values, operating parameters and initial energy consumption coefficients for each grid node, and generate a spatial state matrix.

[0009] Optionally, S2 specifically includes: S21. Perform data preprocessing on the spatial state matrix, divide it into equally spaced grid areas according to the geometric boundaries of the board, assign a unique coordinate index to each grid area, read the position coordinates of the part to be cut, determine whether each grid area is located in the area to be cut, and mark the grid area that does not contain any cutting path as the uncut area. S22. Obtain the coordinate information of the current position of the laser head, find the corresponding grid area in the spatial state matrix and mark it as the current position of the laser head, calculate the start and end points of each path segment according to the cutting path planning file, extract the set of path segments that have not yet been cut, and save them as the set of remaining paths. S23. Based on the thermal field simulation model or temperature sensor data, extract the thermal zone distribution characteristics, and use power and jet pressure to calculate the instantaneous energy consumption of each cutting path segment. Add up the energy consumption of each segment to obtain the cumulative energy consumption. Combine the values ​​of the current position of the laser head, the remaining path set, the thermal zone distribution and the cumulative energy consumption in a unified format into a set of reinforcement learning state vectors, generate the cutting state vector and output it.

[0010] Optionally, S3 specifically includes: S31. Expand the cutting state vector in the order of the remaining path set, and divide the path segment into path sub-segments of a set length according to the endpoint coordinates, Euclidean distance and turning angle of each path segment, and generate a path sub-segment sequence. S32. Construct multi-dimensional feature entries for each path segment. Concatenate the distance from the current position of the laser head to the starting point of the segment, the segment length, the turning angle with the previous segment, the mean and variance of the thermal zone distribution, and the cumulative energy consumption increment in a predetermined order to obtain the path segment feature vector. S33. Input the feature vectors of the path segments into the PathFormer network, model the geometric dependencies and energy consumption relationships between different path segments through relative position encoding and local-global joint attention structure, extract spatial topological features and global semantic information of the path, and output high-dimensional path feature embeddings.

[0011] Optionally, the PathFormer network specifically includes: The feature vectors of the path segments are input into the embedding layer in sequence. A linear mapping is performed on each feature dimension to obtain the initial embedding vector. The relative position encoding based on the Euclidean distance between path segments, the relative turning angle and the distance between uncut regions is superimposed to form a geometrically aware path sequence embedding. The path sequence embedding is divided into multiple local windows, each containing several continuous path segments. Multi-head attention computation is performed within the local window. The dot product similarity of the query, key, and value vectors is calculated for each attention head. After softmax normalization, the local attention weights are obtained. Several key path segments are selected as anchor nodes in the global scope. Cross-window attention, i.e. global attention weight, is calculated with anchor nodes as the center. It is then fused with local attention weight to form a local-global joint attention structure. Multiply the local-global joint attention weights element by element with the corresponding value vectors and sum them to output the context feature vector. Then, perform gated residual connections and layer normalization on the context features in sequence. Input the two-layer feedforward network. After the first layer is dimensionality-upgraded, it is activated by ReLU. The second layer is dimensionality-back-injected and outputs the normalized intermediate feature representation. Based on the connectivity between path segments, whether they cross uncut regions, and whether they violate hot zone constraints, a topology mask matrix and a hot zone mask matrix are constructed to mask the corresponding positions in the attention matrix, so that attention calculation is limited to physically feasible paths. An energy consumption bias term is introduced in the attention scoring process. The cumulative energy consumption increment is weighted proportionally and added to the scoring result, so that low-energy paths are given priority in the attention weight calculation, forming an energy-adaptive path association mechanism. The intermediate features after masking and energy consumption bias modulation are divided into geometric feature channels and thermal-energy consumption feature channels. These channels are then input into independent multi-head attention layers and feedforward networks to extract channel features. The two channel features are then fused element by element through a gating mechanism to obtain the fused feature representation. Insert a path classification label vector before each path segment, input the fused feature sequence into the final aggregation layer, perform adaptive span pooling, and use the output of the path classification label vector at the corresponding position as the global semantic representation of the path segment; The global semantic representations of all path segments are concatenated in order of path number to form a high-dimensional path feature embedding that integrates spatial topological features and global semantic information of the path, and then output.

[0012] Optionally, S4 specifically includes: S41. Read the high-dimensional path feature embedding and the current candidate path action, extract the starting coordinates and ending coordinates of the corresponding path segment, calculate the Euclidean distance between the two points as the path length value, calculate the included angle according to the direction vector of the adjacent path segment, convert the included angle into a turning angle, and increase the path turning penalty by a preset ratio when the turning angle is greater than the set threshold. S42. Calculate the cutting energy consumption based on the path segment length and power. Multiply the laser power by the cutting time to obtain the energy consumption value and add it to the cumulative energy consumption. Read the temperature data within the current path segment coverage area from the heat zone distribution, calculate the average temperature and variance, and increase the heat zone distribution reward according to the preset ratio when the temperature variance does not exceed the preset threshold. S43. Combine the path turning penalty and the hot zone distribution reward according to the set weights to construct a multi-objective reward function. Bind the instant reward value calculated by the multi-objective reward function to the corresponding candidate path number and output the instant reward list for use in training the reinforcement learning policy network.

[0013] Optionally, S5 specifically includes: S51. Read the current laser head position, remaining path set, hot zone distribution and cumulative energy consumption information from the cutting state vector, and combine the spatial topological features and global semantic information of the path contained in the high-dimensional path feature embedding to select candidate path actions and execute them. Calculate the corresponding instant reward value and the cutting state vector at the next moment based on the action result. Combine the cutting state vector, candidate path actions, instant reward value, cutting state vector at the next moment and termination flag into experience data entries and write them into the experience replay queue in chronological order. S52. In the experience replay queue, a multi-threaded asynchronous sampling method is adopted. Multiple sampling threads are started, each thread independently reads experience data entries and randomly extracts samples. The sampling results are cached in memory according to the set batch size. When all threads have completed their respective sampling tasks, the data batches output by each thread are merged to form a training batch dataset. S53. For each training batch dataset, calculate the probability ratio between the current policy network and the old policy network under the same behavior, input the probability ratio into the near-end pruning target, and perform a truncation operation on the ratio that exceeds the pruning range. S54. Based on the immediate reward value and the state value output by the Critic network, calculate the difference between the two to obtain the advantage estimate. Multiply the advantage estimate by the probability ratio and take the negative as the loss function of the Actor network. Perform gradient descent to update the Actor network parameters according to the loss function. S55. Read the instant reward value and target discount reward, calculate the mean square error of the state value, and use it as the optimization target of the Critic network. During the training process, the improved APPO algorithm is used to iteratively update the policy network and value network. S56. After each training round, write the updated Actor network parameters and Critic network parameters into the old policy network respectively, and repeat the policy training and parameter update steps until the policy loss and value loss converge or reach the preset training rounds, and set the current policy network parameters. S57. Use the set policy network to perform policy reasoning on the remaining path set, select path actions in sequence according to the output action probability distribution, generate the optimal cutting path sequence and output it.

[0014] Optionally, the improved APPO algorithm specifically includes: In the training batch dataset, the difference in probability distribution between the current policy network and the old policy network under the same behavior is calculated. The average of the squared difference between the action probability output by the current policy network and the corresponding probability of the old policy network is calculated to obtain the policy distribution deviation value and construct the policy uncertainty regularization term. The policy uncertainty regularization term is added to the loss function of the Actor network in the form of a scaling factor. In each training batch, the variance of the instant reward value change rate and the hot zone distribution in the cut state vector are read and weighted to obtain the state change complexity. When the state change complexity is greater than the preset threshold, the entropy gain is increased by a preset ratio, and vice versa. The state entropy gain is multiplied by a preset entropy weight coefficient to form the state entropy enhancement term, which is then added to the loss function of the Actor network. When the policy network parameters are updated, the policy uncertainty regularization term and the state entropy enhancement term are respectively used in the backpropagation calculation of the loss function. The gradient result obtained is then weighted and fused with the gradient of the advantage estimation term to update the Actor network parameters. Repeat the parameter update step, and record the weight changes of the policy uncertainty regularization term and the state entropy enhancement term after each training round. After the loss function of the Actor network is applied, keep the parameters of the Actor and Critic networks, and output the final policy network.

[0015] Optionally, S6 specifically includes: mapping the optimal cutting path sequence back to the actual coordinate system of the laser cutting equipment, completing the coordinate transformation based on the calibration relationship between the equipment origin and the plate coordinates, generating laser cutting control commands, and driving the laser head to perform cutting operations along the planned path.

[0016] The beneficial effects of this invention are: This invention proposes a reinforcement learning-based laser cutting path planning method. Addressing issues in existing laser cutting methods such as fixed path planning, uneven energy consumption distribution, insufficient control of the heat-affected zone, and lagging policy updates, it constructs a reinforcement learning decision-making system that integrates PathFormer network feature extraction with an improved APPO algorithm for policy optimization. By performing spatial topology modeling and sequence dependency analysis on the cutting state vector through the PathFormer network, it accurately captures the relationships between path segments and the global cutting semantics, achieving high-precision representation of complex sheet geometry and heat zone distribution characteristics. Combined with an improved APPO algorithm incorporating policy uncertainty regularization and state entropy enhancement terms, the Actor and Critic network parameters are dynamically updated under a multi-threaded asynchronous experience replay mechanism, effectively balancing the optimization objectives of path length, energy consumption, and heat zone distribution. This invention possesses advantages such as comprehensive path feature representation, stable policy updates, balanced energy consumption distribution, and strong self-learning capabilities, improving the system's stability and energy utilization in complex processing environments. Attached Figure Description

[0017] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0018] Figure 1 This is a flowchart of a laser cutting path planning method based on reinforcement learning proposed in this invention; Figure 2 This is a flowchart of the cutting state feature extraction and high-dimensional path representation construction based on the PathFormer network proposed in this invention; Figure 3 This is a flowchart of the multi-objective strategy optimization and optimal cutting path generation based on the improved APPO algorithm proposed in this invention. Detailed Implementation

[0019] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0020] refer to Figure 1-3 A laser cutting path planning method based on reinforcement learning includes the following steps: S1. Collect the operating parameters of the laser cutting equipment and the geometric information of the workpiece. The operating parameters include cutting speed, acceleration, power and jet pressure. The geometric information includes the outline of the plate and the position and size of the part to be cut. Establish a laser cutting task scene model and output a spatial state matrix. S2. Based on the spatial state matrix, the plate area is rasterized and the uncut area is marked. The current position of the laser head, the set of remaining paths, the distribution of hot zones and the cumulative energy consumption are defined as state vectors and the cutting state vector is constructed. S3. Input the cutting state vector into the PathFormer network, model the relationship and sequence dependency between different path segments through the multi-head attention mechanism, extract spatial topological features and global semantic information of the path, and output high-dimensional path feature embedding. S4. Based on the high-dimensional path feature embedding and each candidate path action, calculate the path length, turning angle, cutting energy consumption and hot zone distribution, construct a multi-objective reward function with path turning penalty, energy consumption penalty and hot zone distribution reward as the core, and output the instant reward value. S5. Based on the cutting state vector, high-dimensional path feature embedding and immediate reward value, train the policy network using the improved APPO algorithm, introduce policy uncertainty regularization term and state entropy enhancement term, update the Actor and Critic parameters based on multi-threaded asynchronous experience replay, jointly optimize path length, energy consumption and hot zone distribution, and output the optimal cutting path sequence. S6. Map the optimal cutting path sequence back to the actual coordinate system of the laser cutting equipment, generate laser cutting control commands, and drive the laser head to perform cutting operations according to the planned path.

[0021] This implementation significantly improves the intelligence level and processing stability of laser cutting path planning. By collecting equipment operating parameters and workpiece geometric information, a spatial state matrix is ​​constructed to achieve scenario-based modeling and standardized data representation of the cutting task. A cutting state vector is established based on a gridded plate region to accurately describe the laser head position, remaining path set, and hot zone distribution characteristics. After inputting the state vector into the PathFormer network, a multi-head attention mechanism is used to extract spatial topology and global semantic features between paths, achieving a high-dimensional representation of complex geometric structures and thermal relationships. Combined with an improved APPO algorithm, the convergence process of reinforcement learning is stabilized by introducing a policy uncertainty regularization term and a state entropy enhancement term, and sample utilization and optimization efficiency are improved through multi-threaded asynchronous experience replay. The optimal path sequence output by the system achieves balanced energy consumption distribution, continuous and smooth path flow, and adaptive hot zone control, effectively reducing thermal deformation and burr generation during cutting, improving processing accuracy, cutting efficiency, and equipment energy utilization, and demonstrating significant dynamic optimization and self-learning capabilities in complex workpiece processing.

[0022] In this embodiment, S1 specifically includes: S11. Read the operating parameters of the laser cutting equipment through the data acquisition device to obtain the current values ​​of cutting speed, acceleration, power and jet pressure. Import the CAD drawings or 3D model files of the workpiece using geometric modeling to analyze the outline of the plate and the position and size information of the parts to be cut. S12. Synchronize the collected operating parameters and the analyzed geometric information according to a unified timestamp to form a set of status data of the laser cutting equipment and the workpiece; S13. On the computer, call the scene modeling, establish a laser cutting task scene model based on the state data set, divide the plate plane into coordinate grid areas, record the corresponding geometric boundary values, operating parameters and initial energy consumption coefficients for each grid node, and generate a spatial state matrix.

[0023] In this embodiment, S2 specifically includes: S21. Perform data preprocessing on the spatial state matrix, divide it into equally spaced grid areas according to the geometric boundaries of the board, assign a unique coordinate index to each grid area, read the position coordinates of the part to be cut, determine whether each grid area is located in the area to be cut, and mark the grid area that does not contain any cutting path as the uncut area. S22. Obtain the coordinate information of the current position of the laser head, find the corresponding grid area in the spatial state matrix and mark it as the current position of the laser head, calculate the start and end points of each path segment according to the cutting path planning file, extract the set of path segments that have not yet been cut, and save them as the set of remaining paths. S23. Based on the thermal field simulation model or temperature sensor data, extract the thermal zone distribution characteristics, and use power and jet pressure to calculate the instantaneous energy consumption of each cutting path segment. Add up the energy consumption of each segment to obtain the cumulative energy consumption. Combine the values ​​of the current position of the laser head, the remaining path set, the thermal zone distribution and the cumulative energy consumption in a unified format into a set of reinforcement learning state vectors, generate the cutting state vector and output it.

[0024] In this embodiment, S3 specifically includes: S31. Expand the cutting state vector in the order of the remaining path set, and divide the path segment into path sub-segments of a set length according to the endpoint coordinates, Euclidean distance and turning angle of each path segment, and generate a path sub-segment sequence. S32. Construct multi-dimensional feature entries for each path segment. Concatenate the distance from the current position of the laser head to the starting point of the segment, the segment length, the turning angle with the previous segment, the mean and variance of the thermal zone distribution, and the cumulative energy consumption increment in a predetermined order to obtain the path segment feature vector. S33. Input the feature vectors of the path segments into the PathFormer network, model the geometric dependencies and energy consumption relationships between different path segments through relative position encoding and local-global joint attention structure, extract spatial topological features and global semantic information of the path, and output high-dimensional path feature embeddings.

[0025] This implementation introduces the PathFormer network to perform high-dimensional feature modeling and spatial semantic representation of the laser cutting path, achieving intelligent optimization and adaptive energy consumption control of the cutting path under complex geometric structures. The cutting state vector is expanded according to the remaining path set, and path segments are divided based on the endpoint coordinates, Euclidean distance, and turning angle. A multi-dimensional feature vector containing information such as length, angle, thermal distribution, and energy consumption increment is constructed and input into the PathFormer network for encoding. This network simultaneously captures short-range geometric dependencies and long-range topological relationships between path segments through a local-global joint attention structure. An energy consumption bias term and a topological mask matrix are introduced into the attention scoring, giving higher attention weight to low-energy-consumption, thermally balanced path features, thereby achieving path structure constraints and dynamic energy consumption adjustment. The high-dimensional path feature embedding output after gated residual fusion and feedforward network normalization can comprehensively reflect the spatial topology, energy distribution, and thermal impact characteristics of the cutting path. This implementation method can effectively improve the global optimization capability and convergence stability of cutting path planning, making path generation more in line with physical feasibility and energy consumption balance requirements, and significantly improving processing accuracy, heat zone uniformity and path continuity in complex cutting tasks.

[0026] In this embodiment, the PathFormer network specifically includes: The feature vectors of the path segments are input into the embedding layer in sequence. A linear mapping is performed on each feature dimension to obtain the initial embedding vector. The relative position encoding based on the Euclidean distance between path segments, the relative turning angle and the distance between uncut regions is superimposed to form a geometrically aware path sequence embedding. The path sequence embedding is divided into multiple local windows, each containing several continuous path segments. Multi-head attention computation is performed within the local window. The dot product similarity of the query, key, and value vectors is calculated for each attention head. After softmax normalization, the local attention weights are obtained. Several key path segments are selected as anchor nodes in the global scope. Cross-window attention, i.e. global attention weight, is calculated with anchor nodes as the center. It is then fused with local attention weight to form a local-global joint attention structure, which is used to capture short-range geometric dependencies and long-range topological relationships between paths. Multiply the local-global joint attention weights element by element with the corresponding value vectors and sum them to output the context feature vector. Then, perform gated residual connections and layer normalization on the context features in sequence. Input the two-layer feedforward network. After the first layer is dimensionality-upgraded, it is activated by ReLU. The second layer is dimensionality-back-injected and outputs the normalized intermediate feature representation. Based on the connectivity between path segments, whether they cross uncut regions, and whether they violate hot zone constraints, a topology mask matrix and a hot zone mask matrix are constructed to mask the corresponding positions in the attention matrix, so that attention calculation is limited to physically feasible paths, thereby improving structural constraints and computational efficiency. An energy consumption bias term is introduced in the attention scoring process. The cumulative energy consumption increment is weighted proportionally and added to the scoring result, so that low-energy paths are given priority in the attention weight calculation, forming an energy-adaptive path association mechanism. The intermediate features after masking and energy consumption bias modulation are divided into geometric feature channels and thermal-energy consumption feature channels. These channels are then input into independent multi-head attention layers and feedforward networks to extract channel features. The two channel features are then fused element by element through a gating mechanism to obtain the fused feature representation. Insert a path classification label vector before each path segment, input the fused feature sequence into the final aggregation layer, perform adaptive span pooling, and use the output of the path classification label vector at the corresponding position as the global semantic representation of the path segment; The global semantic representations of all path segments are concatenated in order of path number to form a high-dimensional path feature embedding that integrates spatial topological features and global semantic information of the path, and then output.

[0027] In this embodiment, S4 specifically includes: S41. Read the high-dimensional path feature embedding and the current candidate path action, extract the starting coordinates and ending coordinates of the corresponding path segment, calculate the Euclidean distance between the two points as the path length value, calculate the included angle according to the direction vector of the adjacent path segment, convert the included angle into a turning angle, and increase the path turning penalty by a preset ratio when the turning angle is greater than the set threshold. S42. Calculate the cutting energy consumption based on the path segment length and power. Multiply the laser power by the cutting time to obtain the energy consumption value and add it to the cumulative energy consumption. Read the temperature data within the current path segment coverage area from the heat zone distribution, calculate the average temperature and variance, and increase the heat zone distribution reward according to the preset ratio when the temperature variance does not exceed the preset threshold. S43. Combine the path turning penalty and the hot zone distribution reward according to the set weights to construct a multi-objective reward function. Bind the instant reward value calculated by the multi-objective reward function to the corresponding candidate path number and output the instant reward list for use in training the reinforcement learning policy network.

[0028] In this embodiment, S5 specifically includes: S51. Read the current laser head position, remaining path set, hot zone distribution and cumulative energy consumption information from the cutting state vector, and combine the spatial topological features and global semantic information of the path contained in the high-dimensional path feature embedding to select candidate path actions and execute them. Calculate the corresponding instant reward value and the cutting state vector at the next moment based on the action result. Combine the cutting state vector, candidate path actions, instant reward value, cutting state vector at the next moment and termination flag into experience data entries and write them into the experience replay queue in chronological order. S52. In the experience replay queue, a multi-threaded asynchronous sampling method is adopted. Multiple sampling threads are started, each thread independently reads experience data entries and randomly extracts samples. The sampling results are cached in memory according to the set batch size. When all threads have completed their respective sampling tasks, the data batches output by each thread are merged to form a training batch dataset. S53. For each training batch dataset, calculate the probability ratio of the current policy network and the old policy network under the same behavior, input the probability ratio into the near-end pruning target, and perform a truncation operation on the ratio that exceeds the pruning range to limit the policy update magnitude and prevent excessive policy distribution shift. S54. Based on the immediate reward value and the state value output by the Critic network, calculate the difference between the two to obtain the advantage estimate. Multiply the advantage estimate by the probability ratio and take the negative as the loss function of the Actor network. Perform gradient descent to update the Actor network parameters according to the loss function. S55. Read the immediate reward value and the target discount reward, calculate the mean square error of the state value, and use it as the optimization objective of the Critic network. Minimize the value error to improve the prediction accuracy of the Critic network for future rewards. During the training process, the improved APPO algorithm is used to iteratively update the policy network and the value network. S56. After each training round, write the updated Actor network parameters and Critic network parameters into the old policy network to keep the new and old policy networks synchronized in time for subsequent iterative training. Repeat the policy training and parameter update steps until the policy loss and value loss converge or reach the preset training rounds, and set the current policy network parameters. S57. Use the set policy network to perform policy reasoning on the remaining path set, select path actions in sequence according to the output action probability distribution, generate the optimal cutting path sequence and output it.

[0029] This implementation introduces an improved APPO algorithm to jointly optimize the policy network and value network in laser cutting path planning, achieving dynamic decision-making and energy consumption balance control of the cutting path under multi-objective reinforcement learning. The system selects candidate path actions based on the cutting state vector and high-dimensional path feature embedding, calculates immediate reward values, and generates empirical data entries. Multi-threaded asynchronous sampling and batch merging are used to form a training batch dataset, improving sample utilization and training efficiency. During the policy update phase, a near-end pruning target constraint is used to constrain the update amplitude, preventing excessive policy distribution shift. A policy uncertainty regularization term is introduced to stabilize policy convergence, and a state entropy enhancement term dynamically adjusts the exploration intensity in high-complexity scenarios, enabling the policy to achieve an adaptive trade-off between energy consumption and path continuity. During training, the Actor and Critic network parameters iterate alternately, and old policies are updated synchronously, ensuring stable policy iteration and accurate value prediction. The final generated policy network can select the optimal path sequence based on the action probability distribution, achieving globally optimal planning and hot zone balance control of the cutting path. This implementation significantly improves the stability and generalization ability of policy training, effectively reduces energy consumption fluctuations, and improves cutting quality and equipment operating efficiency.

[0030] In this embodiment, the improved APPO algorithm specifically includes: In the training batch dataset, the probability distribution difference between the current policy network and the old policy network under the same behavior is calculated. The average of the squared difference between the action probability output by the current policy network and the corresponding probability of the old policy network is calculated to obtain the policy distribution deviation value and construct the policy uncertainty regularization term. The policy uncertainty regularization term is added to the loss function of the Actor network in the form of a proportional coefficient to limit the distribution offset between the new and old policies during the policy update stage and improve the stability of policy update. In each training batch, the variance of the instant reward value change rate and the hot zone distribution in the cut state vector are read and weighted to obtain the state change complexity. When the state change complexity is greater than the preset threshold, the entropy gain is increased by a preset ratio, and vice versa. The state entropy gain is multiplied by a preset entropy weight coefficient to form the state entropy enhancement term, which is then added to the loss function of the Actor network. When the policy network parameters are updated, the policy uncertainty regularization term and the state entropy enhancement term are respectively used in the backpropagation calculation of the loss function. The gradient result obtained is then weighted and fused with the gradient of the advantage estimation term to update the Actor network parameters. Repeat the parameter update step, and record the weight changes of the policy uncertainty regularization term and the state entropy enhancement term after each training round. After the loss function of the Actor network is applied, keep the parameters of the Actor and Critic networks, and output the final policy network.

[0031] In this embodiment, S6 specifically includes: mapping the optimal cutting path sequence back to the actual coordinate system of the laser cutting equipment, completing the coordinate transformation based on the calibration relationship between the equipment origin and the plate coordinates, generating laser cutting control commands, and driving the laser head to perform cutting operations along the planned path.

[0032] Example 1: To verify the practical feasibility and performance advantages of this invention in complex industrial cutting environments, it was applied to the intelligent laser processing center of a large sheet metal manufacturing enterprise in a certain province. This enterprise produces over 1.2 million metal parts annually and has 16 fiber laser cutting machines with power ranging from 3kW to 8kW, cutting various materials including stainless steel, aluminum alloy, and galvanized sheet. The experimental scenario selected four machines in similar operating conditions within the workshop for comparative testing. The testing period was from March to April 2025, lasting 60 days, during which a total of 2480 cutting tasks were performed, with a total cutting length of approximately 11200 meters.

[0033] In practical deployment, the method of this invention interfaces with the equipment control system to collect real-time operating parameters such as laser power, jet pressure, feed speed, and acceleration. It then combines these parameters with the workpiece CAD model to generate the geometric contour of the sheet metal and the distribution data of the parts, constructing a cutting task scenario model. The PathFormer network extracts the spatial topological features and global semantic information of the cutting state vector, capturing the intersection and energy superposition relationships between multiple paths during the path generation stage, achieving a high-dimensional feature representation of the heat-affected zone. Subsequently, a policy network is trained based on an improved APPO algorithm, introducing a policy uncertainty regularization term and a state entropy enhancement term. The Actor and Critic parameters are optimized in multi-threaded asynchronous experience replay, achieving a joint balance between path length, energy consumption, and heat zone distribution.

[0034] During the experimental testing phase, the company selected tasks with the same workpiece design and compared the traditional heuristic shortest path planning method with the method of this invention. For each batch of tasks, the actual cutting time, average energy consumption, thermal deformation deviation, edge burr rate, and number of path switching times were recorded. The data comparison is shown below:

[0035] Table 1. Performance Comparison of the Invention Method and Traditional Laser Cutting Path Planning Method

[0036] Based on the comparative data shown in Table 1, it can be seen that the laser cutting path planning method based on reinforcement learning proposed in this invention exhibits significant advantages over traditional path planning algorithms in key indicators such as cutting efficiency, energy consumption control, thermal deformation suppression, and path smoothness. Regarding cutting efficiency, this invention achieved significant improvements in tests on different sheet types, with the average cutting time reduced by approximately 18% to 22% compared to traditional methods. For example, the cutting time for 304 stainless steel sheet decreased from 40.4 seconds to 32.6 seconds, and for galvanized sheet from 55.8 seconds to 44.7 seconds. This indicates that the high-dimensional path feature embedding extracted through the PathFormer network can effectively capture the spatial topological relationship of the cutting path, significantly reducing invalid path movement.

[0037] In terms of energy consumption, the average energy consumption per unit length decreased by approximately 21%, with aluminum alloy sheets decreasing from 0.098 kWh / m to 0.076 kWh / m. This demonstrates that the improved APPO algorithm achieves a dynamic balance between path length and energy consumption distribution in multi-objective optimization. Regarding cutting quality, the average thermal deformation decreased by approximately 25% overall, and the peak temperature of the hot zone decreased by an average of over 30°C. This indicates that the system, optimized using reinforcement learning strategies, effectively suppresses heat accumulation, ensuring sheet forming accuracy and cutting stability.

[0038] Meanwhile, the number of path switching times decreased by an average of about 30%, and the edge glitch rate dropped from an average of 4.5% to 2.6%, indicating that reinforcement learning strategies have the ability to adaptively optimize continuity and processing quality in dynamic environments.

[0039] Overall, this invention achieves intelligent decision-making and dynamic balance control of the cutting path through high-dimensional feature modeling of the PathFormer network and strategy optimization mechanism of the improved APPO algorithm. This not only improves cutting efficiency and energy utilization, but also significantly improves processing accuracy and thermal uniformity, verifying the feasibility and technological advancement of this invention in intelligent manufacturing scenarios.

[0040] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A laser cutting path planning method based on reinforcement learning, characterized in that, Includes the following steps: S1. Collect the operating parameters of the laser cutting equipment and the geometric information of the workpiece, establish a laser cutting task scenario model, and output the spatial state matrix; S2. Based on the spatial state matrix, the board area is rasterized and the uncut area is marked. The state vector is defined and the cutting state vector is constructed. S3. Input the cutting state vector into the PathFormer network, model the relationship and sequence dependency between different path segments through the multi-head attention mechanism, extract spatial topological features and global semantic information of the path, and output high-dimensional path feature embedding. S4. Based on the high-dimensional path feature embedding and each candidate path action, calculate the path length, turning angle, cutting energy consumption and hot zone distribution, construct a multi-objective reward function, and output the instant reward value. S5. Based on the cutting state vector, high-dimensional path feature embedding, and immediate reward value, train the policy network using the improved APPO algorithm, update the Actor and Critic parameters based on multi-threaded asynchronous experience replay, and output the optimal cutting path sequence. The improved APPO algorithm specifically includes: In the training batch dataset, the difference in probability distribution between the current policy network and the old policy network under the same behavior is calculated. The average of the squared difference between the action probability output by the current policy network and the corresponding probability of the old policy network is calculated to obtain the policy distribution deviation value and construct the policy uncertainty regularization term. The policy uncertainty regularization term is added to the loss function of the Actor network in the form of a scaling factor. In each training batch, the variance of the instant reward value change rate and the hot zone distribution in the cut state vector are read and weighted to obtain the state change complexity. When the state change complexity is greater than the preset threshold, the entropy gain is increased by a preset ratio, and vice versa. The state entropy gain is multiplied by a preset entropy weight coefficient to form the state entropy enhancement term, which is then added to the loss function of the Actor network. When the policy network parameters are updated, the policy uncertainty regularization term and the state entropy enhancement term are respectively used in the backpropagation calculation of the loss function. The gradient result obtained is then weighted and fused with the gradient of the advantage estimation term to update the Actor network parameters. Repeat the parameter update step, record the weight changes of the policy uncertainty regularization term and the state entropy enhancement term after each training round, and after the loss function of the Actor network is applied, keep the parameters of the Actor and Critic networks, and output the final policy network. S6. Map the optimal cutting path sequence back to the actual coordinate system of the laser cutting equipment, generate laser cutting control commands, and drive the laser head to perform cutting operations according to the planned path.

2. The laser cutting path planning method based on reinforcement learning according to claim 1, characterized in that, S1 specifically includes: S11. Read the operating parameters of the laser cutting equipment through the data acquisition device to obtain the current values ​​of cutting speed, acceleration, power and jet pressure. Import the CAD drawings or 3D model files of the workpiece using geometric modeling to analyze the outline of the plate and the position and size information of the parts to be cut. S12. Synchronize the collected operating parameters and the analyzed geometric information according to a unified timestamp to form a set of status data of the laser cutting equipment and the workpiece; S13. On the computer, call the scene modeling, establish a laser cutting task scene model based on the state data set, divide the plate plane into coordinate grid areas, record the corresponding geometric boundary values, operating parameters and initial energy consumption coefficients for each grid node, and generate a spatial state matrix. 3.The laser cutting path planning method based on reinforcement learning according to claim 1, wherein, S2 specifically includes: S21. Perform data preprocessing on the spatial state matrix, divide it into equally spaced grid areas according to the geometric boundaries of the board, assign a unique coordinate index to each grid area, read the position coordinates of the part to be cut, determine whether each grid area is located in the area to be cut, and mark the grid area that does not contain any cutting path as the uncut area. S22. Obtain the coordinate information of the current position of the laser head, find the corresponding grid area in the spatial state matrix and mark it as the current position of the laser head, calculate the start and end points of each path segment according to the cutting path planning file, extract the set of path segments that have not yet been cut, and save them as the set of remaining paths. S23. Based on the thermal field simulation model or temperature sensor data, extract the thermal zone distribution characteristics, and use power and jet pressure to calculate the instantaneous energy consumption of each cutting path segment. Add up the energy consumption of each segment to obtain the cumulative energy consumption. Combine the values ​​of the current position of the laser head, the remaining path set, the thermal zone distribution and the cumulative energy consumption in a unified format into a set of reinforcement learning state vectors, generate the cutting state vector and output it.

4. The laser cutting path planning method based on reinforcement learning according to claim 1, characterized in that, S3 specifically includes: S31. Expand the cutting state vector in the order of the remaining path set, and divide the path segment into path sub-segments of a set length according to the endpoint coordinates, Euclidean distance and turning angle of each path segment, and generate a path sub-segment sequence. S32. Construct multi-dimensional feature entries for each path segment. Concatenate the distance from the current position of the laser head to the starting point of the segment, the segment length, the turning angle with the previous segment, the mean and variance of the thermal zone distribution, and the cumulative energy consumption increment in a predetermined order to obtain the path segment feature vector. S33. Input the feature vectors of the path segments into the PathFormer network, model the geometric dependencies and energy consumption relationships between different path segments through relative position encoding and local-global joint attention structure, extract spatial topological features and global semantic information of the path, and output high-dimensional path feature embeddings.

5. The laser cutting path planning method based on reinforcement learning according to claim 4, characterized in that, The PathFormer network specifically includes: The feature vectors of the path segments are input into the embedding layer in sequence. A linear mapping is performed on each feature dimension to obtain the initial embedding vector. The relative position encoding based on the Euclidean distance between path segments, the relative turning angle and the distance between uncut regions is superimposed to form a geometrically aware path sequence embedding. The path sequence embedding is divided into multiple local windows, each containing several continuous path segments. Multi-head attention computation is performed within the local window. The dot product similarity of the query, key, and value vectors is calculated for each attention head. After softmax normalization, the local attention weights are obtained. Several key path segments are selected as anchor nodes in the global scope. Cross-window attention, i.e. global attention weight, is calculated with anchor nodes as the center. It is then fused with local attention weight to form a local-global joint attention structure. Multiply the local-global joint attention weights element by element with the corresponding value vectors and sum them to output the context feature vector. Then, perform gated residual connections and layer normalization on the context features in sequence. Input the two-layer feedforward network. After the first layer is dimensionality-upgraded, it is activated by ReLU. The second layer is dimensionality-back-injected and outputs the normalized intermediate feature representation. Based on the connectivity between path segments, whether they cross uncut regions, and whether they violate hot zone constraints, a topology mask matrix and a hot zone mask matrix are constructed to mask the corresponding positions in the attention matrix, so that attention calculation is limited to physically feasible paths. An energy consumption bias term is introduced in the attention scoring process. The cumulative energy consumption increment is weighted proportionally and added to the scoring result, so that low-energy paths are given priority in the attention weight calculation, forming an energy-adaptive path association mechanism. The intermediate features after masking and energy consumption bias modulation are divided into geometric feature channels and thermal-energy consumption feature channels. These channels are then input into independent multi-head attention layers and feedforward networks to extract channel features. The two channel features are then fused element by element through a gating mechanism to obtain the fused feature representation. Insert a path classification label vector before each path segment, input the fused feature sequence into the final aggregation layer, perform adaptive span pooling, and use the output of the path classification label vector at the corresponding position as the global semantic representation of the path segment; The global semantic representations of all path segments are concatenated in order of path number to form a high-dimensional path feature embedding that integrates spatial topological features and global semantic information of the path, and then output.

6. The laser cutting path planning method based on reinforcement learning according to claim 1, characterized in that, S4 specifically includes: S41. Read the high-dimensional path feature embedding and the current candidate path action, extract the starting coordinates and ending coordinates of the corresponding path segment, calculate the Euclidean distance between the two points as the path length value, calculate the included angle according to the direction vector of the adjacent path segment, convert the included angle into a turning angle, and increase the path turning penalty by a preset ratio when the turning angle is greater than the set threshold. S42. Calculate the cutting energy consumption based on the path segment length and power. Multiply the laser power by the cutting time to obtain the energy consumption value and add it to the cumulative energy consumption. Read the temperature data within the current path segment coverage area from the heat zone distribution, calculate the average temperature and variance, and increase the heat zone distribution reward according to the preset ratio when the temperature variance does not exceed the preset threshold. S43. Combine the path turning penalty and the hot zone distribution reward according to the set weights to construct a multi-objective reward function. Bind the instant reward value calculated by the multi-objective reward function to the corresponding candidate path number and output the instant reward list for use in training the reinforcement learning policy network.

7. The laser cutting path planning method based on reinforcement learning according to claim 1, characterized in that, S5 specifically includes: S51. Read the current laser head position, remaining path set, hot zone distribution and cumulative energy consumption information from the cutting state vector. Combine the spatial topology features and path global semantic information, select candidate path actions and execute them. Calculate the corresponding instant reward value and the cutting state vector at the next moment. Combine the cutting state vector, candidate path actions, instant reward value, cutting state vector at the next moment and termination flag into experience data entries and write them into the experience replay queue in chronological order. S52. In the experience replay queue, a multi-threaded asynchronous sampling method is adopted. Multiple sampling threads are started, each thread independently reads experience data entries and randomly extracts samples. The sampling results are cached in memory according to the set batch size. When all threads have completed their respective sampling tasks, the data batches output by each thread are merged to form a training batch dataset. S53. For each training batch dataset, calculate the probability ratio between the current policy network and the old policy network under the same behavior, input the probability ratio into the near-end pruning target, and perform a truncation operation on the ratio that exceeds the pruning range. S54. Based on the immediate reward value and the state value output by the Critic network, calculate the difference between the two to obtain the advantage estimate. Multiply the advantage estimate by the probability ratio and take the negative as the loss function of the Actor network. Perform gradient descent to update the Actor network parameters according to the loss function. S55. Read the instant reward value and target discount reward, calculate the mean square error of the state value, and use it as the optimization target of the Critic network. During the training process, the improved APPO algorithm is used to iteratively update the policy network and value network. S56. After each training round, write the updated Actor network parameters and Critic network parameters into the old policy network respectively, and repeat the policy training and parameter update steps until the policy loss and value loss converge or reach the preset training rounds, and set the current policy network parameters. S57. Use the set policy network to perform policy reasoning on the remaining path set, select path actions in sequence according to the output action probability distribution, generate the optimal cutting path sequence and output it.

8. The laser cutting path planning method based on reinforcement learning according to claim 1, characterized in that, S6 specifically includes: mapping the optimal cutting path sequence back to the actual coordinate system of the laser cutting equipment, completing the coordinate transformation based on the calibration relationship between the equipment origin and the plate coordinates, generating laser cutting control commands, and driving the laser head to perform cutting operations along the planned path.

Images