Plate utilization rate and energy consumption collaborative optimization method based on reinforcement learning

Abstract

The invention discloses a plate utilization rate and energy consumption collaborative optimization method based on reinforcement learning, and the method comprises the steps: obtaining the geometric data of a part to be subjected to stock layout and the kinetic parameters of equipment, and constructing a graph structure containing a machining energy consumption characteristic component; node features are extracted by using a graph attention network into which an energy consumption gating mechanism is introduced; determining a target part and a rough area through a macroscopic strategy network, and outputting an accurate pose in combination with a microcosmic execution network; establishing a virtual physical field containing inertia and heat accumulation components in the two-dimensional space, and calculating the potential energy value of the target part and the plate coverage area increment; a reward signal is generated based on the difference value, and network parameters are updated through a strategy gradient algorithm; according to the method, the energy consumption and the thermal effect are quantified into soft constraints through the virtual physical field, and cooperative reduction of the processing energy consumption and the thermal deformation risk is achieved while the utilization rate of the plates is maximized.

Description

Technical Field

[0001] This invention relates to the field of computer-aided manufacturing technology, and in particular to a method for co-optimizing sheet metal utilization and energy consumption based on reinforcement learning. Background Technology

[0002] With the rapid development of intelligent manufacturing technology, sheet metal cutting and blanking, as a key link in industrial production, directly impacts material costs and production efficiency due to its technological level. Traditional computer-aided nesting techniques largely rely on heuristic algorithms (such as genetic algorithms and simulated annealing algorithms) or geometrically rule-based critical polygon (NFP) methods, primarily aiming to find non-overlapping layouts of parts in a two-dimensional plane to maximize sheet metal utilization. In recent years, deep reinforcement learning (DRL), due to its powerful feature extraction and sequence decision-making capabilities, has been gradually applied to solve combinatorial optimization problems with high-dimensional state spaces and complex constraints. By constructing end-to-end neural network models to replace traditional manual heuristic rules, significant progress has been made in improving the efficiency and automation of nesting calculations.

[0003] However, existing technologies suffer from two significant shortcomings: First, traditional nesting algorithms and cutting path planning are typically treated as two independent sequential processes, prioritizing the maximization of geometric utilization before generating the cutting path. This decoupled "nesting first, planning later" approach ignores kinematic constraints during the cutting process, often resulting in high material utilization but requiring the cutting head to make frequent long-distance idle movements or rapid acceleration and deceleration between different parts, leading to a surge in machine tool energy consumption and a decrease in processing efficiency. Second, most existing reinforcement learning nesting algorithms treat parts merely as static geometric pixels or contours, lacking the ability to perceive processing dynamics (such as machine tool acceleration limitations and inertia) and physical field effects (such as sheet metal deformation caused by heat accumulation during cutting). This makes it difficult to predict and avoid the high energy consumption risks associated with complex trajectories during the nesting decision-making stage, and thus fails to achieve a globally optimal balance between material cost savings and reduced processing energy consumption. Summary of the Invention

[0004] The purpose of this section is to outline some aspects of embodiments of the present invention and to briefly describe some preferred embodiments. Simplifications or omissions may be made in this section, as well as in the abstract and title of this application, to avoid obscuring the purpose of these documents; however, such simplifications or omissions should not be construed as limiting the scope of the invention.

[0005] In view of the aforementioned existing problems, this invention is proposed. Therefore, this invention provides a reinforcement learning-based method for the coordinated optimization of sheet metal utilization and energy consumption, to address the problems mentioned in the background art.

[0006] To address the aforementioned technical problems, this invention provides the following technical solution: a reinforcement learning-based method for co-optimizing board utilization and energy consumption, comprising: Obtain the geometric contour data of the set of parts to be laid out and the dynamic parameters of the cutting equipment, map each part in the set of parts to be laid out as a node of a graph structure, and generate a feature vector for each node. The feature vector is formed by splicing the geometric feature components of the geometric contour data and the processing energy consumption feature components calculated from the dynamic parameters. The graph structure is input into a preset graph attention network, the attention coefficients between nodes are calculated, and the attention coefficients are weighted according to the numerical differences of the node feature vectors. The output includes a global state vector and a node embedding vector containing the topological relationship and feature similarity between parts. The global state vector is input into the macro-strategy network, which outputs the index of the target part and the rough placement area. At the same time, the node is embedded into the vector input into the micro-execution network, which outputs the coordinates, rotation angle and cutting entry point of the target part within the rough placement area. A virtual physical field containing inertial potential energy field components and thermal accumulation potential energy field components is established in the two-dimensional space where the plate is located. The potential energy value of the target part at the coordinates and the increase in the plate coverage area brought by the target part are calculated. A reward signal is generated based on the difference between the potential energy value and the incremental area of ​​the board coverage. The parameters in the macro-policy network and the micro-execution network are updated using the reward signal through a policy gradient algorithm.

[0007] As a preferred embodiment of the reinforcement learning-based method for co-optimizing sheet metal utilization and energy consumption described in this invention, the step of calculating the processing energy consumption characteristic components includes: Calculate the curvature distribution data of the part's geometric profile along the path; Based on the maximum acceleration and maximum speed limit parameters of the cutting equipment, calculate the speed limit curve under the curvature distribution data; The path integral of the reciprocal of the speed limit curve is used to obtain the dynamic complexity value, which is then used as the characteristic component of processing energy consumption.

[0008] As a preferred embodiment of the reinforcement learning-based method for co-optimizing board utilization and energy consumption described in this invention, the step of weighting the attention coefficients includes: The absolute value of the difference between the processing energy consumption characteristic components of two connected nodes in the computation graph structure; Construct a gated weight function that is negatively correlated with the absolute value of the difference; The original attention coefficients are multiplied by the output value of the gating weight function to obtain the corrected attention coefficients.

[0009] As a preferred embodiment of the reinforcement learning-based method for co-optimizing board utilization and energy consumption described in this invention, the input data of the micro-execution network further includes: The current physical state vector of the cutting equipment, which includes the position coordinates of the cutting head at the end of the last action and the residual velocity vector; The micro-execution network outputs action commands based on the combination of the node embedding vector and the physical state vector.

[0010] As a preferred embodiment of the reinforcement learning-based method for co-optimizing plate utilization and energy consumption described in this invention, the virtual physical field containing inertial potential energy field components is established, including: Using the position coordinates of the cutting head at the end of the last action as the field center, calculate the position vector of the point to be evaluated relative to the field center, and the dot product of the position vector and the residual velocity vector; Based on the value of the dot product and the Euclidean distance between the location to be evaluated and the center of the field, the scalar value of the inertial potential energy field at that location is calculated.

[0011] As a preferred embodiment of the reinforcement learning-based method for co-optimizing plate utilization and energy consumption described in this invention, the virtual physical field containing the thermal accumulation potential energy field component is established, including: Create a two-dimensional thermodynamic numerical matrix corresponding to the dimensions of the sheet material; After each part position is determined, the value of the corresponding region at that position is increased, and a calculation logic that decays with the time step is applied to the full matrix value. The values ​​in the two-dimensional thermodynamic numerical matrix are directly mapped to scalar values ​​of the thermal cumulative potential energy field.

[0012] As a preferred embodiment of the reinforcement learning-based method for co-optimizing board utilization and energy consumption described in this invention, the process of updating the parameters in the macro-policy network and the micro-execution network further includes an overlapping constraint processing step, which includes: The differentiable rasterization algorithm is used to convert the currently output part position parameters into a probability density image; Calculate the pixel superposition value of the probability density image and the probability density image of the existing parts on the plate; Construct an overlap loss function based on the pixel overlay value, calculate the gradient of the overlap loss function with respect to the micro-execution network output parameters, and incorporate the gradient into the network parameter update calculation.

[0013] As a preferred embodiment of the reinforcement learning-based method for co-optimizing board utilization and energy consumption described in this invention, the feature vector further includes: A perforation energy consumption characteristic component, which is a scalar calculated based on the product of the material thickness of the part to be sampled and the perforation power parameter preset by the cutting equipment.

[0014] As a preferred embodiment of the reinforcement learning-based method for co-optimizing sheet metal utilization and energy consumption described in this invention, after obtaining the geometric contour data of the set of parts to be laid out, the method further includes: Perform a Fourier descriptor transform on the geometric contour data, and select the first N Fourier coefficients as input data for generating geometric feature components.

[0015] As a preferred embodiment of the reinforcement learning-based method for co-optimizing board utilization and energy consumption described in this invention, the graph attention network is composed of multiple layers of graph convolutional neural networks stacked together, and the global state vector is obtained by performing global average pooling on all node features output by the last layer of graph convolutional neural network.

[0016] Compared with existing technologies, the beneficial effects of this solution are: 1. This invention introduces "processing energy consumption characteristic components" calculated from equipment dynamic parameters (such as maximum acceleration and perforation power), and uses an "energy consumption gating mechanism" to correct the graph attention network. This enables the neural network to perceive the physical properties of the part's processing in the early stages of decision-making, changing the local optimum dilemma caused by the traditional "layout first, planning later" approach. This allows the reinforcement learning algorithm to automatically find combinations with good dynamic compliance and low energy consumption while pursuing high density layout (high utilization), effectively reducing production costs. 2. By establishing an "inertial potential energy field" and a "thermal accumulation potential energy field" in the reinforcement learning environment, the inertial potential energy field uses the residual velocity vector to guide the cutting head to move with the momentum, reducing the servo energy consumption and machine tool vibration caused by sudden stops and turns. The thermal accumulation potential energy field, by simulating the heat diffusion process, forces the policy network to adopt a "jump-cutting" strategy, avoiding thermal deformation of the sheet metal and out-of-tolerance dimensions of parts caused by local overheating, thereby predicting and avoiding processing risks at the algorithm level. 3. Furthermore, to address the issue of non-differentiability in traditional geometric intersection detection, this invention utilizes a differentiable rasterization algorithm to construct an overlap loss function based on probability density images. This function not only determines whether parts overlap but also provides gradient information about position and angle, explicitly instructing the microscopic execution network "where to move to eliminate overlap." This significantly accelerates the training and convergence speed of the neural network and ensures the geometric legitimacy of the output nesting scheme. Attached Figure Description

[0017] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. Wherein: Figure 1 This is a flowchart illustrating the overall process of a reinforcement learning-based method for co-optimizing board utilization and energy consumption, as described in one embodiment of the present invention. Detailed Implementation

[0018] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the protection scope of the present invention.

[0019] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0020] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in different places in this specification does not necessarily refer to the same embodiment, nor is it a single or selective embodiment that is mutually exclusive with other embodiments.

[0021] This invention is described in detail with reference to the schematic diagrams. When detailing the embodiments of this invention, for ease of explanation, the cross-sectional views illustrating the device structure may be partially enlarged, not adhering to the usual scale. Furthermore, the schematic diagrams are merely examples and should not be construed as limiting the scope of protection of this invention. In actual fabrication, the three-dimensional spatial dimensions of length, width, and depth should be included.

[0022] Furthermore, in the description of this invention, it should be noted that the terms "upper," "lower," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. These terms are used solely for the convenience of describing the invention and for simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the invention. In addition, the terms "first," "second," or "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0023] Unless otherwise explicitly specified and limited, the terms "installation," "connection," and "joining" in this invention should be interpreted broadly. For example, they can refer to fixed connections, detachable connections, or integral connections; similarly, they can refer to mechanical connections, electrical connections, or direct connections, or indirect connections through an intermediate medium, or internal connections between two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0024] Example 1 Reference Figure 1 This is the first embodiment of the present invention, which provides a method for co-optimizing board utilization and energy consumption based on reinforcement learning, including: S1. Obtain the geometric contour data of the set of parts to be laid out and the dynamic parameters of the cutting equipment. Map each part in the set of parts to be laid out as a node of the graph structure and generate the feature vector of each node. The feature vector is spliced ​​together by the geometric feature components of the geometric contour data and the processing energy consumption feature components calculated from the dynamic parameters. Furthermore, the set of parts to be laid out is parsed from the CAD file (such as DXF or DWG format). For each part Extract its closed contour coordinate point sequence. At the same time, in order to perceive the physical limitations in the actual processing, read the dynamic parameters of the cutting equipment from the configuration file of the CNC cutting system. Specifically, these dynamic parameters mainly include: the machine tool's maximum permissible linear speed, maximum permissible acceleration, and drilling power reference parameters; Furthermore, in order to ensure that the part features have rotational invariance during the nesting process, in this embodiment, coordinate points are not used directly, but Fourier descriptors are used to characterize the geometry of the part. Specifically, the parts The contour is discretized into Count the points to be evenly spaced, and assign the two-dimensional coordinates of each point. Mapping complex numbers on the complex plane ,in, The imaginary unit, ; Furthermore, a Discrete Fourier Transform (DFT) is performed on the complex sequence to obtain the frequency domain coefficients. : in, Indicates the first Fourier coefficients; Furthermore, to eliminate the effects of translation, scaling, and starting point position, the frequency domain coefficients also need to be normalized; that is, by selecting the previous... order (e.g.) The modulus of low-frequency coefficients constitutes the geometric characteristic components. : in, Used for normalization scale, It reflects the shape characteristics of the part's contour at different frequencies (such as overall roundness, triangular features, etc.); It should be noted that the role of this geometric feature component is to enable the neural network to quickly identify the macroscopic shape of the parts (such as aspect ratio and convexity / concavity) through low-frequency coefficients, thereby determining the geometric matching degree between parts in the macroscopic strategy network. Furthermore, since existing technologies typically only focus on the geometric area of ​​parts and ignore the nonlinear influence of shape complexity on processing energy consumption, the present invention quantifies the "processing cost" of each part by constructing and calculating a processing energy consumption characteristic component, wherein the characteristic component is represented by two parts: the dynamic complexity value and the perforation energy consumption characteristic component. Specifically, the purpose of dynamic complexity is to quantify the constraint effect of the part contour on the cutting speed. When the part contour has a large curvature (such as sharp corners or small arcs), the machine tool must decelerate to ensure accuracy. This deceleration caused by the shape will prolong the processing time, which in turn leads to the accumulation of energy consumption of auxiliary equipment (such as lasers, air compressors, and servo motors). The process of calculating the dynamic complexity is as follows: S101. Calculate the path length parameter of the part contour. Curvature distribution on ; S102. Construct a restricted speed curve based on the machine tool dynamics parameters. This curve reflects the highest theoretical speed that the cutting head can reach at each point on the contour under the physical constraint of satisfying the maximum acceleration of the machine tool: in, This refers to the maximum linear speed of the cutting equipment; This is the maximum acceleration of the cutting equipment; For the outline in position Instantaneous curvature at the point; To prevent the minimum value where the denominator is zero; S103. Perform path integration on the reciprocal of the constrained velocity curve to obtain the numerical value of the dynamic complexity. : in, For parts circumference; It needs to be explained that the physical meaning of this integral value is: the minimum theoretical time required to cut the part under ideal dynamic constraints, that is, The larger the part, the more complex its features, and the higher its energy consumption per unit area. The neural network can adjust the layout order accordingly, for example, by arranging such parts in areas or paths with better heat dissipation. Specifically, regarding the energy consumption characteristic component of piercing, considering that the piercing process accounts for a significant proportion of energy consumption in thick plate processing, the piercing process needs to be treated as an independent characteristic dimension, resulting in: in, The thickness of the material to be arranged in the sample; The correction factor for material thermophysical properties (such as the difference in enthalpy of fusion between carbon steel and stainless steel) is a dimensionless parameter used to normalize the processing difficulty of different materials. A common material (such as Q235 carbon steel) can be selected as the benchmark and set to 1.0. For other materials, the correction factor can be determined by the ratio of its key thermophysical properties (such as melting point, thermal conductivity, and enthalpy of fusion) to the benchmark material. For example, it can be defined as: Correction factor = (target material with enthalpy of fusion / benchmark material with enthalpy of fusion) × b, where b is an empirical adjustment factor, usually between 0.8 and 1.2. Preset perforation energy consumption parameters per unit thickness for the cutting equipment; Furthermore, by combining the two parts mentioned above, the processing energy consumption characteristic component can be obtained, which is represented as: ; Furthermore, to achieve collaborative optimization, the algorithm must simultaneously access information from both the "shape" and "energy consumption" dimensions. Therefore, the geometric feature components calculated above are concatenated with the machining energy consumption feature components to generate each part node. Comprehensive feature vector : in, This indicates that Z-score standardization (zero-mean normalization) is used to eliminate the order-of-magnitude differences between different physical units. It's important to emphasize that for the processing energy consumption feature components, the maximum and minimum values ​​from their historical datasets are collected and mapped to... The interval is used to ensure the stability of the input values ​​for subsequent graph attention networks; It should be noted that constructing a graph structure ,in, As a set of nodes, the graph can be constructed as a fully connected graph in the initial state, that is, there is an edge connection between any two part nodes. Through this mapping, each node in the graph structure is no longer just a geometric polygon, but an entity carrying a "shape fingerprint" and a "processing cost label", which enables the subsequent graph attention network to automatically avoid the continuous processing of high-energy-consuming parts while pursuing high utilization, so as to achieve multi-objective collaborative optimization. S2. Input the graph structure into the preset graph attention network, calculate the attention coefficients between nodes, and weight the attention coefficients according to the numerical differences of the node feature vectors. Output a global state vector and node embedding vector containing the topological relationship and feature similarity between parts. It should be noted that the purpose of this step is to use the powerful relational reasoning ability of graph neural networks to extract high-order features of "which parts are suitable for being grouped together" from an unordered set of parts. Unlike traditional graph attention networks, the present invention introduces an attention correction mechanism based on physical properties to force the network to focus on parts that are similar in dynamics and energy consumption characteristics, thereby reducing the dramatic changes in working conditions during the cutting process. Furthermore, the content generated in step S1 includes The graph structure of each component node is input into the graph attention network. For any node in the graph... Its input feature vector is ,in, For feature dimensions; Specifically, to enhance the expressive power of the input feature vector, a shared, learnable linear transformation matrix is ​​used. Projecting the features of all nodes maps the input feature vector to a higher-dimensional latent space, enabling the spliced ​​geometric feature components and processing energy consumption feature components to be initially fused, and extracting the implicit hybrid features. : Furthermore, because neural networks need to determine nodes... For nodes The importance of the nodes is considered; therefore, in this embodiment, a self-attention mechanism is used to compute the nodes. Its neighboring nodes Attention coefficient between This is used to indicate the importance of nodes: in, This represents a vector concatenation operation; This represents the weight vector of a single-layer feedforward neural network. It is a nonlinear activation function (negative slope is usually taken as 0.2) used to handle nonlinear relationships; It is important to note that the correlation score obtained at this point... It only reflects the similarity at the data level and has not yet taken into account the compatibility of physical processing; Furthermore, in actual cutting, if adjacent parts differ greatly in shape complexity and energy consumption requirements (for example, an extremely complex gear followed by a simple rectangle), the machine tool servo system will experience severe acceleration and deceleration oscillations, which is detrimental to energy consumption control. Therefore, the present invention introduces an "energy consumption gating mechanism," the construction steps of which are as follows: S201. Extract the processing energy consumption feature components calculated in step S1 from the node features. (i.e., dynamic complexity) and perforation energy consumption ), calculate connected nodes and The absolute difference in energy consumption characteristics between : in, The L1 norm (Manhattan distance) is used to measure the degree of incompatibility between the two in terms of physical processing properties. S202. Construct a gate function that is negatively correlated with the difference. The larger the difference in this function, the smaller the weight. in, The hyperparameter (temperature coefficient) for controlling sensitivity. This refers to the gating weight value, whose physical meaning is to suppress attention connections between parts that differ greatly in dynamic characteristics, encourage the neural network to prioritize the combination of parts with "similar physical properties", thereby inducing the policy network to generate a stable sampling sequence. S203, adjust the original attention coefficients Multiply by the gating weights and perform Softmax normalization to obtain the final normalized attention coefficients. : in, For nodes The set of neighboring nodes; Furthermore, to stabilize the learning process, a multi-head attention mechanism is employed. Assuming that... Each node has an independent attention head. Output node embedding vector For splicing the calculation results of each head: in, For non-linear activation functions (such as ELU); Specifically, after stacking and propagating through multiple (e.g., 3) graph convolutional layers, the final output is a node embedding vector matrix. It contains the local position information of each part in the entire topology and its physical compatibility with its neighbors. Furthermore, this part of the node embedding vector will be directly input into the subsequent micro-execution network to determine the specific pose of the part. In addition, in order for the macro strategy network to grasp the current overall layout progress and the distribution of remaining parts, a global descriptor also needs to be generated; Specifically, global average pooling is performed on all node features output by the last layer of the graph neural network to obtain the global state vector. : It should be noted that, through global average pooling, It can reduce the dimensionality of the overall feature distribution of the parts set (e.g., whether there are many complex small parts left, or whether only regular large parts are left). At the same time, this vector will be used as the input of the macro-policy network to help it decide which type of parts should be processed first, thereby balancing the utilization rate of sheet metal and energy consumption at the macro level. S3. Input the global state vector into the macro-strategy network to output the index of the target part and the rough placement area. At the same time, embed the node into the vector and input it into the micro-execution network to output the coordinates, rotation angle and cutting entry point of the target part within the rough placement area. It should be noted that this step adopts a hierarchical reinforcement learning (HRL) architecture, which aims to decompose the complex nesting and path planning problem into two levels: "macro-level decision-making" and "micro-level execution". The macro-level is responsible for global planning, solving the problems of "which parts to select" and "approximately where to place them" to optimize the board filling rate. The micro-level is responsible for local fine-tuning, solving the problems of "specific orientation" and "cutting process path" to reduce idle travel energy consumption. Specifically, in this embodiment, the macro-policy network consists of a 3-layer fully connected neural network (MLP), with each layer containing 256 neurons. The activation function is ReLU, and the output layer generates a probability distribution using the Softmax function. The micro-execution network adopts an Actor-Critic architecture. The Actor network receives a concatenated input of local state vectors and node embedding vectors, which is processed through 4 fully connected layers to output Gaussian distribution parameters of the actions. The Critic network shares the first two layers of parameters with the Actor network and is used to evaluate the state value. Furthermore, the macro-policy network aims to plan the sorting order from a global perspective, and to integrate the global state vector generated in step S2. The input macro-policy network consists of multiple fully connected layers, and outputs the probability distribution of the target part index and the probability distribution of the placement area through two independent branches at the output end. Specifically, the macro-policy network will output a dimension of probability vector ( (This represents the total number of parts). Simultaneously, to ensure the effectiveness of the macro-policy network's actions, the probability vector needs to be filtered using the current part's "unspaced mask" to ensure that only parts that have not yet been padded are selected. Based on this, the macro-policy network receives the current global state vector... As input, after computation by the macro-strategy network, a selection probability distribution for all parts is output. Subsequently, based on this probability distribution, the part indices of the target grid region that needs to be sorted at the current moment are sampled (extracted). : The specific task of the macro-policy network is to select the identity index of the parts; The following indicates the condition, namely, the current observed global state vector. Under the premise; It represents any possible action in the action space, that is, any index in the set of all parts to be laid out; Then, the node embedding vector matrix generated from step S2 Extract the corresponding target node embedding vector ; Furthermore, in the process of the above output selection probability distribution, it is also necessary to discretize the board material into... The grid region, the macro-policy network outputs a dimension of probability vector This is used to indicate which grid area of ​​the sheet metal the target part should be placed in; It should be noted that the physical significance of selecting grid areas is that the macro strategy network tends to select areas that can maximize the use of the remaining space of the sheet metal and form a compact cluster with the already laid-out parts, thus laying the foundation for optimizing the utilization rate of the sheet metal. Specifically, since the micro-execution network needs to perform fine-grained operations based on macro-instructions, in order to enable the micro-network to perceive the local environment, this embodiment constructs a local state vector that includes both the "target itself" and "environmental constraints". : in, The local height map or occupancy raster feature of the target mesh region is obtained by extracting local features from the current occupancy state map of the board using a convolutional neural network (CNN). This feature is used to characterize the remaining geometric space of the region. A region of size is extracted centered on the target mesh region. (For example The local occupancy state map of a pixel is used as the input to the CNN, which consists of 3 convolutional layers and 2 fully connected layers. The final output is a feature vector of fixed dimensions (e.g., 128 dimensions) to represent the remaining geometric space of the region. The coordinates of the cutting end point of the previous laid-out part The physical meaning of this parameter is that it enables the macro- and micro-networks to plan the entry point of the current part based on the position of the previous point, thereby minimizing the "empty travel" distance of the cutting head between parts and thus achieving energy consumption optimization. The residual velocity vector of the cutting head at this point enables the micro-execution network to directly perceive the current motion inertia, thereby actively selecting actions that minimize the machine tool's acceleration and deceleration energy consumption when deciding on the next cutting entry point. Furthermore, The input is fed into a micro-execution network (typically an Actor-Critic architecture Actor network), which outputs a continuous action vector. : It's important to note that although this micro-execution network outputs a continuous action vector, it doesn't directly output definite action values. Instead, it outputs statistical parameters of the action distribution (mean). and standard deviation ); in, This is the coordinate offset, representing the normalized offset of the reference center point of the target part relative to the center point of the target mesh region. Its value range is typically limited to... Based on this, the final layout coordinates The calculation is as follows: in, The center coordinates of the target grid region. The physical dimensions of the target grid region; Specifically, in the above continuous motion vector, the rotation angle To represent the rotational orientation of the part, and to facilitate network convergence, sine and cosine values ​​are typically output, or normalized angle values ​​are directly output and mapped to... ; It is important to emphasize that the selection of the rotation angle directly affects the fit between parts (utilization rate) and the change in angular velocity during cutting (energy consumption). Specifically, in the aforementioned continuous action vector, the cutting entry point parameter... For a normalized scalar This is used to determine the starting point of the laser cutting head on the contour of the part. Here, we assume the part... The perimeter of the outline is The path distance from the starting point on the contour is... Micro-execution networks learn (The previous point's ending position) and the current starting point are... The Euclidean distance between the determined locations will tend to choose a location that is far from the desired location. Recent This value greatly reduces the energy consumption of the servo motor during non-cutting time; It should be noted that, through the above steps, the present invention transitions from macroscopic graph relationship reasoning to microscopic geometric continuous control, completing the entire process decision-making from "what to choose" to "how to cut", ensuring the optimal synergy between material utilization and processing energy consumption; S4. Establish a virtual physical field containing inertial potential energy field components and thermal accumulation potential energy field components in the two-dimensional space where the plate is located, and calculate the potential energy value of the target part at the coordinate and the increase in the plate coverage area brought by the target part. Furthermore, an environmental feedback mechanism is constructed to transform the simple geometric layout problem into an energy minimization problem in the physical field. By introducing a virtual potential energy field, the "difficult-to-quantify" energy consumption and thermal deformation risk are transformed into specific values ​​(potential energy values). This allows reinforcement learning to automatically avoid high-energy-consumption and high-risk actions while pursuing the utilization rate of the board material (i.e., obtaining area increment rewards) under the "soft constraint" of the potential energy field. Furthermore, since traditional nesting algorithms often ignore the motion state of the cutting head when it moves from the previous position to the current position, it leads to frequent sudden stops and violent turns. This not only increases the energy consumption of the servo motor, but also aggravates the vibration of the machine tool. Based on this, the present invention establishes an inertial potential energy field, which is used to characterize the continuity and compliance of the machine tool motion. Specifically, the process of establishing an inertial potential energy field is as follows: S401.1 Read the position coordinates of the cutting head at the end of the last action. and the residual velocity vector at this time ,in, This represents the current "momentum flow direction" of the machine tool. Following this flow direction for the next cut consumes the least energy, while going against this flow direction requires a huge amount of energy to decelerate and accelerate in the opposite direction. S401.2, Let the coordinates of the target part cutting entry point (starting point) output by the micro-execution network be... Calculate the position vector of the point to be evaluated relative to the field center (i.e., the previous end point). : Then, calculate the position vector. With the residual velocity vector The dot product is used to construct the inertial potential energy scalar by combining it with Euclidean distance. : in, The distance traveled is the empty travel distance (Euclidean distance). The longer the distance, the higher the potential energy (the greater the energy consumption). For each item, the weight coefficient is given, and follows the rules. The principle; when When the target point is ahead of the residual velocity, the cosine term is positive. Since it is set to minus, it will reduce the potential energy value at that point, which means that the algorithm "encourages" the cutting head to use inertia to move with the momentum. when When the target point is behind and a turnaround is required, the cosine term is negative, which will significantly increase the potential energy value at that point, meaning that the algorithm "penalizes" sharp angle turning operations. Furthermore, since a large amount of heat is generated during laser cutting, if a local area of ​​the plate is cut in a short period of time, the heat cannot be dissipated in time, causing thermal expansion and deformation of the plate, which in turn leads to out-of-tolerance dimensions or even scrapping of the parts. Based on this, the present invention also establishes a thermal accumulation potential energy field, which is used to force reinforcement learning to learn the "jump cutting" strategy, that is, to cut alternately in different areas of the plate to achieve global thermal balance and prevent local overheating and deformation. Specifically, the process of establishing a thermal accumulation potential energy field is as follows: S402.1 Create a two-dimensional thermodynamic numerical matrix that is proportional to the physical dimensions of the board. Initially, the entire field was at 0 (normal temperature). S402.2, at each time step First, we simulate the natural cooling process by applying a time-step decay logic to the entire matrix: in, The thermal diffusion attenuation coefficient (e.g., 0.95) is used to simulate the dissipation of heat over time. S402.3 After determining the location of the target part, identify the grid area covered by the part. Increase the thermal value of the area by a specific amount of heat input. (Proportional to the perimeter or area of ​​the part): Specifically, this heat input The specific calculation method is as follows: in, It is the perimeter of the target part. It is the heat input coefficient per unit length related to cutting power and speed. It is a fixed heat input value associated with the perforation process. and Pre-settings can be made based on experience data from the cutting equipment; At this moment, the target part is at its current position. The accumulated potential energy at the point Defined as the sum of the background thermal values ​​of the region where the part is located (in superposition) Before): It should be noted that if the algorithm attempts to place the new part in a In regions with very high values ​​(i.e., regions that have just been cut and whose residual heat has not dissipated), the potential energy term will be very large, thus generating negative feedback in the reward function; Furthermore, by weighted summing the components (potential energy values) contained in the two physical fields (inertial potential energy field and thermal accumulation potential energy field), the total potential energy value of the target part in its current pose can be obtained. : in, and These are inertial weight and thermal weight, each with an initial value of 0.5, used to adjust the proportion of energy consumption and processing quality in the optimization objective; Furthermore, the increase in sheet metal coverage area caused by the target part is calculated simultaneously. ,get: In this embodiment, the geometric area of ​​the part is directly used. As a basic reward; It should be noted that this step successfully quantified the benefits brought about by "making this cut" ( ) and the physical costs incurred ( The difference between the benefits and physical costs will serve as a core signal to guide the continuous evolution of the macro-policy network and the micro-execution network in reinforcement learning, ultimately learning to find the optimal balance between "more cutting and more use" and "less running and less heat". S5. Generate a reward signal based on the difference between the potential energy value and the incremental area of ​​the board coverage. Use the reward signal to update the parameters in the macro-policy network and the micro-execution network through the policy gradient algorithm. It should be noted that this step aims to construct a multi-objective composite reward function and combine it with a differentiable physical constraint mechanism to drive reinforcement learning to evolve through trial and error. Traditional placement algorithms usually adopt an offline mode of "placement first and evaluation later", while the solution of this invention enables the neural network to perceive the "economic benefits" (area increment) and "physical cost" (potential energy consumption) of each placement action through real-time feedback reward signals, thereby adaptively adjusting the strategy. Furthermore, in order to achieve synergistic optimization of board utilization and energy consumption, geometric gains and physical penalties must be unified into a single scalar signal. Specifically, the incremental area of ​​the board material covered by the calculation based on step S4. Total potential energy value Construct the current time step Basic reward signal : in, This is the geometric utilization reward coefficient, used to amplify area benefits and encourage the network to fill as many boards as possible. This represents the total area of ​​the board material, used for normalization to ensure numerical stability. This is the physical field penalty coefficient, used to suppress high-energy-consuming actions; The hyperbolic tangent function is used to convert the potential energy value... Mapped to This range is designed to prevent the potential energy value from exploding due to excessive heat accumulation or excessively long idle travel in a certain area, thereby compromising the stability of training. It should be noted that, in the above In the calculation, when Very large (filling in large parts) and When the temperature is very low (smooth cutting, no heat concentration), reward If the result is positive, the behavior is reinforced; conversely, if the cutting head has to make a long return trip (high inertial potential energy) or operate in a high-temperature area (high thermal potential energy) in order to insert a part, then... Enlargement, leading to If the value decreases or even becomes negative, a penalty will be imposed. Furthermore, in reinforcement learning nesting, agents are prone to "cheating" to obtain high area rewards by overlapping parts. Traditional geometric intersection detection (such as the NFP algorithm) only returns a "yes / no" boolean value, which is non-differentiable and cannot be directly used for backpropagation in neural networks. Therefore, this invention introduces a differentiable rasterization technique to calculate the overlap loss, as follows: S501, using a Gaussian blur kernel or a differentiable rendering function The part position parameters (coordinates) output by the micro-execution network will be used. and angle Convert it into a probability density image In this image, the pixel value inside the part is 1, and it smoothly transitions to 0 at the boundary. This smooth transition ensures the existence of a gradient. The transformation process is as follows: S502. Obtain the cumulative probability density image of all existing parts on the board. Calculate the pixel-level superposition value of the two as a measure of their degree of overlap: S503, Define overlap loss This term not only serves as part of the loss function but can also be added as a significant negative reward. middle: in, For overlapping penalty weights; It should be noted that, through calculation We can obtain gradients about position and angle, which means that when overlap occurs, the neural network not only knows that "something is wrong", but also knows "which direction to move (gradient direction) to reduce overlap". This directional guidance signal enables the micro-execution network to quickly converge to the non-overlapping valid solution space. Furthermore, using the reward signal and loss term obtained from the above calculations, a policy gradient-based algorithm (such as PPO-proximal policy optimization algorithm) is used to jointly update the macro-policy network and the micro-execution network. It is important to note that since the actions are generated using a reparameterization technique, the overlap loss is... The gradient of an action can be passed back to the parameters of the micro-execution network using the chain rule. The final total loss function includes not only the policy gradient loss from reinforcement learning but also this overlapping loss term; that is, the comprehensive loss function is defined as follows: for: in, These are the weighting coefficients for each loss term, which are the same as the aforementioned weighting coefficients, ranging from [0,1] and following the rules... ; Among them, strategy loss Aimed at maximizing the expected cumulative reward: in, The advantage function represents the degree of advantage of the current action relative to the average level, and its calculation depends on the reward signal. ; Specifically, in the comprehensive loss function, value loss Used to train a critic network to accurately predict the value of the current state, thus reducing variance; entropy regularization term. This is used to encourage the network to maintain a certain degree of exploratory nature and prevent premature convergence to a local optimum (e.g., always sorting in a fixed order). Furthermore, the gradient of the comprehensive loss function with respect to the network parameters is calculated using the backpropagation algorithm. And use the Adam optimizer to update the weight parameters of all neural networks; It should be noted that, through the above steps, the present invention not only mathematically guarantees the correctness of the optimization direction, but also physically achieves a leap from "geometric layout" to "physical sensing manufacturing".

[0025] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. The solutions in the embodiments of this application can be implemented using various computer languages, such as the object-oriented programming language Java and the interpreted scripting language JavaScript.

[0026] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0027] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0028] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0029] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.

[0030] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A method for co-optimizing board utilization and energy consumption based on reinforcement learning, characterized in that, include: Obtain the geometric contour data of the set of parts to be laid out and the dynamic parameters of the cutting equipment, map each part in the set of parts to be laid out as a node of a graph structure, and generate a feature vector for each node. The feature vector is formed by splicing the geometric feature components of the geometric contour data and the processing energy consumption feature components calculated from the dynamic parameters. The graph structure is input into a preset graph attention network, the attention coefficients between nodes are calculated, and the attention coefficients are weighted according to the numerical differences of the node feature vectors. The output includes a global state vector and a node embedding vector containing the topological relationship and feature similarity between parts. The global state vector is input into the macro-strategy network, which outputs the index of the target part and the rough placement area. At the same time, the node is embedded into the vector input into the micro-execution network, which outputs the coordinates, rotation angle and cutting entry point of the target part within the rough placement area. A virtual physical field containing inertial potential energy field components and thermal accumulation potential energy field components is established in the two-dimensional space where the plate is located. The potential energy value of the target part at the coordinates and the increase in the plate coverage area brought by the target part are calculated. A reward signal is generated based on the difference between the potential energy value and the incremental area of ​​the board coverage. The parameters in the macro-policy network and the micro-execution network are updated using the reward signal through a policy gradient algorithm.

2. The reinforcement learning-based method for co-optimizing board utilization and energy consumption as described in claim 1, characterized in that, The step of calculating the processing energy consumption characteristic components includes: Calculate the curvature distribution data of the part's geometric profile along the path; Based on the maximum acceleration and maximum speed limit parameters of the cutting equipment, calculate the speed limit curve under the curvature distribution data; The path integral of the reciprocal of the speed limit curve is used to obtain the dynamic complexity value, which is then used as the characteristic component of processing energy consumption.

3. The reinforcement learning-based method for co-optimizing board utilization and energy consumption as described in claim 1, characterized in that, The step of weighting the attention coefficients includes: The absolute value of the difference between the processing energy consumption characteristic components of two connected nodes in the computation graph structure; Construct a gated weight function that is negatively correlated with the absolute value of the difference; The original attention coefficients are multiplied by the output value of the gating weight function to obtain the corrected attention coefficients.

4. The reinforcement learning-based method for co-optimizing board utilization and energy consumption as described in claim 1, characterized in that, The input data for the micro-execution network also includes: The current physical state vector of the cutting equipment, which includes the position coordinates of the cutting head at the end of the last action and the residual velocity vector; The micro-execution network outputs action commands based on the combination of the node embedding vector and the physical state vector.

5. The reinforcement learning-based method for co-optimizing board utilization and energy consumption as described in claim 4, characterized in that, Establishing the virtual physical field containing the inertial potential energy field component includes: Using the position coordinates of the cutting head at the end of the last action as the field center, calculate the position vector of the point to be evaluated relative to the field center, and the dot product of the position vector and the residual velocity vector; Based on the value of the dot product and the Euclidean distance between the location to be evaluated and the center of the field, the scalar value of the inertial potential energy field at that location is calculated.

6. The reinforcement learning-based method for co-optimizing board utilization and energy consumption as described in claim 1, characterized in that, Establishing the virtual physical field containing the thermally accumulated potential energy field component includes: Create a two-dimensional thermodynamic numerical matrix corresponding to the dimensions of the sheet material; After each part position is determined, the value of the corresponding region at that position is increased, and a calculation logic that decays with the time step is applied to the full matrix value. The values ​​in the two-dimensional thermodynamic numerical matrix are directly mapped to scalar values ​​of the thermal cumulative potential energy field.

7. The reinforcement learning-based method for co-optimizing board utilization and energy consumption as described in claim 1, characterized in that, The process of updating the parameters in the macro-policy network and the micro-execution network also includes an overlapping constraint processing step, which includes: The differentiable rasterization algorithm is used to convert the currently output part position parameters into a probability density image; Calculate the pixel superposition value of the probability density image and the probability density image of the existing parts on the plate; Construct an overlap loss function based on the pixel overlay value, calculate the gradient of the overlap loss function with respect to the micro-execution network output parameters, and incorporate the gradient into the network parameter update calculation.

8. The reinforcement learning-based method for co-optimizing board utilization and energy consumption as described in claim 1, characterized in that, The feature vector also includes: A perforation energy consumption characteristic component, which is a scalar calculated based on the product of the material thickness of the part to be sampled and the perforation power parameter preset by the cutting equipment.

9. The reinforcement learning-based method for co-optimizing board utilization and energy consumption as described in claim 1, characterized in that, After obtaining the geometric contour data of the set of parts to be laid out, the process also includes: Perform a Fourier descriptor transform on the geometric contour data, and select the first N Fourier coefficients as input data for generating geometric feature components.

10. The reinforcement learning-based method for co-optimizing board utilization and energy consumption as described in claim 1, characterized in that, The graph attention network is composed of multiple stacked graph convolutional neural networks, and the global state vector is obtained by performing global average pooling on all node features output by the last layer of the graph convolutional neural network.

Images