Method for selecting part to be placed on sheet

Abstract

The invention relates to a method for selecting a part to be placed on a sheet using a computer, comprising the following steps: I) encoding a geometric feature of each part in a respective geometric information vector, where, for each part, one of the geometric features is a projected area of the part in a plan view; iI) generating a graph, the geometric features of each part being assigned at a time to a node of the graph, all nodes being connected in pairs at a time by an edge, the method comprising the following further steps in the inference phase: XIV) estimating parameters GCIs for all pairs of parts, each GCI being a measure of the degree of satisfaction to which the pairs of parts can be placed on the same sheet; xV) assigning each GCI to a respective edge of the graph, the respective edge passing through two nodes representing the part to which the respective GCI is assigned; xVI) determining weights of edges of the graph, each edge weight depending on the GCI associated with the edge; xVII) assigns the parts to the respective sheets by means of an optimization method by determining sub-graphs of the graph passing through said nodes, the nodes passed by the respective sub-graphs representing the parts to be placed on the sheets, the sum of the projected areas of the parts on each sub-graph being at most equal to the respective sheet size.

Description

Background Art

[0001] The present invention relates to a method of selecting parts to be placed on a sheet using a computer.

[0002] DE 10 2020 129 293 A1 discloses a method for generating a cutting plan for cutting parts from a sheet of plate-like material. In particular, a plan for nesting the parts is generated so that material waste is kept to a minimum.

[0003] However, this approach does not allow for flexible part selection to reduce the amount of material waste generated when cutting parts from the sheet.

[0004] Purpose of the Invention

[0005] It is an object of the present invention to provide a method for dispensing parts to a metal sheet in such a way that the waste of material produced when cutting the parts from the metal sheet is minimized. Summary of the invention

[0006] This object is achieved by a method according to claim 1. The dependent claims relate to advantageous embodiments.

[0007] The method according to the invention comprises the following steps:

[0008] I. Encoding the geometric features of each part in a corresponding geometric information vector, wherein, for each part, one of the geometric features is the projected area of ​​the part in the plane view;

[0009] II. Generate a graph in which each part's geometric feature is assigned once to a node of the graph, in which all nodes are connected pairwise by one edge at a time,

[0010] Among other things, the method comprises the following additional steps in the inference phase:

[0011] XIV. Estimating parameters GCI (geometric compatibility index or indices) for all pairs of parts, where each GCI is a measure of how well the pair of parts can be placed on the same sheet;

[0012] XV. assigning each GCI to a corresponding edge of the graph, the corresponding edge passing through two nodes representing the parts to which the corresponding GCI is assigned;

[0013] XVI. Determine the weights of the edges of the graph, wherein each edge weight depends on the GCI associated with the edge;

[0014] XVII. Allocate parts to corresponding sheets by determining a subgraph of the graph passing through the nodes with the aid of an optimization method, wherein the nodes passed through by the corresponding subgraph represent parts to be placed on the sheet, and wherein the sum of the projected areas of the parts on each subgraph is at most equal to the corresponding sheet size.

[0015] According to the invention, a set of parts is divided into subsets, each of which is arranged on a respective sheet. The subsets are determined so that the parts of the subset optimally cover the respective sheet. The selection of suitable parts for the respective sheet simplifies the subsequent arrangement of the parts on the sheet. In particular, the method for determining the arrangement of parts on a particular sheet can be applied to a relatively small number of parts previously determined for the sheet in question. Furthermore, material losses during the cutting of the parts are greatly reduced.

[0016] The geometric information vector contains information about the shape and size of the corresponding part, for example, data about the angles between the sides of the part, the length of the sides of the part, and the curvature of the part.

[0017] The relationship between the weight w of a corresponding edge of the graph and the GCI assigned to this edge according to step XVI is in particular given by: w=1-GCI.

[0018] The graph is preferably designed as a fully connected geometric graph.

[0019] In an advantageous embodiment of the method, the optimization method in step XVII is performed using a quantum computer. In particular, a qubit can be assigned to each node of the graph. Thus, advantageously, the high computing power of a quantum computer can be used while keeping the number of required qubits relatively low.

[0020] In some embodiments of the method, one or more steps of the method are performed by a first neural network, preferably a graph neural network, particularly preferably a graph convolutional neural network. The geometric information vectors associated with the parts are represented as nodes of the graph. The graph neural network is then particularly useful for classifying the nodes according to which sheet the part represented by the node belongs to.

[0021] In a preferred embodiment of the method, step XIV, step XV and / or step XVI are performed by a class actor module, wherein the class actor module preferably comprises a first neural network. The class actor module can be trained by a reinforcement learning method to relatively quickly learn a strategy by which parts of different shapes can be allocated to a given sheet in an optimal manner.

[0022] In an advantageous variant of the method, the allocation of the parts to the respective sheets according to step XVII is performed with the aid of an optimization method for solving a capacity-constrained vehicle routing problem (CVRP). The optimization process for solving the CVRP enables the determination of the optimal allocation of the parts to the respective sheets in a relatively short time. In particular, the weights of the edges correspond to the distances between the locations in the CVRP. The areas of the parts encoded in the nodes of the graph in the plan view correspond to the capacities or demands allocated to the respective locations in the CVRP. The sheets on which the parts are placed correspond to the vehicles in the CVRP.

[0023] In particular, the algorithm for solving CVRP can be implemented on a quantum computer to group the nodes of the graph into subsets, where the nodes in each subset correspond to parts on a corresponding sheet.

[0024] According to a preferred embodiment of the method, the sheets have the same sheet size. This simplifies the application of the method for solving the CVRP in order to find the optimal allocation of a given part to the corresponding sheet.

[0025] In an advantageous embodiment of the method, determining the GCI for all pairs of parts in a training phase performed before the inference phase comprises the following steps:

[0026] III. setting an estimated value of GCI, wherein each estimated value of the GCI is assigned to a pair of parts respectively;

[0027] IV. providing first edge weights to the edges of the graph, wherein each first edge weight depends on a corresponding GCI assigned to the edge;

[0028] V. assigning the parts to the corresponding sheets by determining the subgraph of the graph passing through the node according to step XVII;

[0029] VI. Determine the reward function value of each sub-graph as a measure of how much surface area of ​​the corresponding sheet is covered by the part on each sub-graph after the part is arranged on the corresponding sheet;

[0030] VII. determining a suitability value for each subgraph in the form of an average of the output values ​​of the corresponding subgraph, wherein the output values ​​take into account the estimated value of the GCI, wherein the average is determined by taking into account all the output values ​​of the corresponding subgraph;

[0031] VIII. Re-determine the GCI for each subgraph so that the reward loss is reduced on each subgraph, where the reward loss depends on the reward function value and the fitness value;

[0032] IX. Repeat steps VI to VIII until the reward loss meets the first termination criterion;

[0033] X. Estimating a second edge weight of an edge of the graph, in particular estimating the second edge weight by steps XIV to XVI;

[0034] XI. Determine an edge weight loss at an edge of the graph, wherein the edge weight loss depends on the second edge weight and the GCI at the corresponding edge determined in step VIII;

[0035] XII. Repeating step X and step XI, wherein estimating the second edge weight is performed so that the edge weight loss is minimized until the edge weight loss satisfies the second termination criterion;

[0036] XIII. Repeat step III to step XII, wherein the estimated value of GCI in step III is set to the second edge weight finally determined in step XII, until the estimated value of GCI meets the third termination criterion.

[0037] In this embodiment, the estimation method for estimating the second edge weight is advantageously optimized during a training phase in order to find a suitable value of the GCI for a given part in as few steps as possible.

[0038] In particular, the relationship between the first weight w of the respective edge and the GCI associated with the edge is given by the following formula: w=1-GCI. The same applies to the second weight of the edge. The average value according to step VII is particularly determined as an arithmetic mean. The reward loss according to step VIII is particularly calculated by a Laplace loss function L1, which depends on the reward function value and the fitness value.

[0039] The reward function value according to step VI may be calculated, for example, as the ratio of the sum of the projected areas of the parts in the plan view and the corresponding sheet size for each sub-view.

[0040] Preferably, the output values ​​according to step VII are determined by passing the nodes of the corresponding subgraph through a first successive graph convolutional neural network layer and a second successive graph convolutional neural network (GCNConv) layer, wherein the GCNConv layer initially uses as input an estimate of the GCI, such as the estimate from step III. The output of the first GCNConv layer is provided as input to the second GCNConv layer via a message passing function that takes into account the estimate of the GCI, the number of edges connected to each node in the subgraph, information about which nodes are connected together, weighted geometric information encoded in the nodes by the geometric information vector and / or the trainable weights of the GCNConv layer. Then, the average value is determined according to step VII using the output values ​​of the second GCNConv layer.

[0041] In some embodiments of the method, step X, step XI and / or step XII are performed by a class actor module. The training process, in particular step X, step XI and / or step XII trains the class actor module so that the class actor module can find the GCI alone when a new part is presented to the class actor module without additional input.

[0042] In an advantageous variant of the method, step III, step IV, step VII, step VIII and / or step IX are performed by a class critic module, wherein the class critic module in particular comprises a second neural network, wherein in addition to the second neural network the class critic module preferably also comprises a third neural network, wherein the third neural network receives the result data of the second neural network. The class critic module can be used to improve the class actor module in a training phase by specifying the values ​​of the GCI parameters to be implemented by the class actor module without any external (human) input. By using a two-layer class critic module, the workload of determining the GCI can be reduced. To this end, the layers are designed in particular as convolutional layers.

[0043] In particular, all output data of the class critic module, preferably all output data of the third neural network belonging to the class critic module, are averaged into a single value corresponding to the suitability value in step VII.

[0044] The edge weight loss according to step X is preferably determined by a loss function, preferably a Gaussian loss function L2, which depends on the second edge weight (the value of GCI determined by the class actor module) and the first edge weight (the value of the edge weight determined by the class critic module).

[0045] According to an advantageous embodiment of the method, the redetermination of the GCI according to step VIII is performed using error backpropagation to minimize the reward loss of the corresponding subgraph. Backpropagation allows the GCI to be determined according to step VIII in a relatively time-saving manner so that the reward loss function is minimized. Backpropagation is used in particular in combination with the Laplace loss function L1 mentioned above.

[0046] In some variations of the method, the first termination criterion according to step IX is given by the reward loss being less than a first default value, and / or the second termination criterion according to step XII is given by the edge weight loss at the edge of the graph being less than a second default value, and / or the third termination criterion according to step XIII is given by the estimated value of GCI changing less than a third default value after undergoing steps III to XII. In particular, each individual edge weight loss is less than the second default value. Preferably, the change in each individual estimated value of GCI is less than the third default value.

[0047] In another embodiment of the method, a metaheuristic method is used to determine the GCI in step XIV. A metaheuristic method can be used to determine an approximate value of the GCI to obtain a comparative value of the GCI obtained by the method according to steps XV to XVII. If a value of the GCI cannot be found by the method according to steps XV to XVII, a metaheuristic method can also be used.

[0048] In an advantageous variant of the method, the parts are arranged on a sheet and subsequently produced. If the parts are manufactured using this method, material waste during part production is significantly reduced.

[0049] Further advantages of the present invention can be seen from the description and the drawings. Likewise, the above-mentioned features and the features still further described can each be used individually or in any combination. The embodiments shown and described should not be understood as a conclusive list, but have exemplary properties for describing the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1a Examples of geometrically compatible parts are shown;

[0051] Figure 1b Examples of geometrically incompatible parts are shown;

[0052] Figure 2a A schematic diagram shows a sheet before part nesting;

[0053] Figure 2b Schematically shows a sheet in which additional triangular parts are attached to Figure 2a

[0054] Arrange the parts shown in the figure;

[0055] Figure 2c Schematic diagram of two circular parts and Figure 2a The arrangement of the parts shown in FIG.

[0056] Figure 3 Schematically shows an overview of the method according to the invention;

[0057] Figure 4a A rasterized image of a single part is shown, where the geometry of the part is encoded by an autoencoder;

[0058] Figure 4b A rasterized image showing a layout of several parts, wherein the geometry of each part is encoded by an autoencoder;

[0059] Figure 5 Schematically illustrates an actor-like module and a critic-like module used in the method according to the present invention;

[0060] Figure 6 A block diagram schematically illustrates the data collection and updating steps in the learning routine according to the present invention;

[0061] Figure 7a An example of a Capacity Constrained Vehicle Routing Problem (CVRP) is schematically shown;

[0062] Figure 7b schematically shows a geometric relationship graph (GRG) used in the method according to the invention;

[0063] Figure 8 Examples of large GRGs and subgraphs of GRGs are schematically shown;

[0064] Figure 9a A first example of using an edge weight loss for testing a first model of the method according to the invention is shown;

[0065] Figure 9b A first example of reward loss using a first model for testing the method according to the invention is shown;

[0066] Fig.10a A second example of using an edge weight loss for testing a second model of the method according to the invention is shown;

[0067] Fig.10b A second example of reward loss using a second model for testing the method according to the invention is shown;

[0068] Fig.11a shows parts in an example data set used for testing the method according to the invention before a pre-processing step;

[0069] Fig.11b shows parts in an example data set used for testing the method according to the invention after a pre-processing step;

[0070] Fig.12a shows a visualization of the parameters GCI for a first set of parts using a first model for testing the method according to the invention;

[0071] Figure 12b shows a visualization of the parameters GCI for a first set of parts using a second model for testing the method according to the invention;

[0072] Fig.12c shows a visualization of the parameters GCI for a second set of parts using the first model for testing the method according to the invention;

[0073] Fig.12dshows a visualization of the parameters GCI for a second set of parts using a second model for testing the method according to the invention;

[0074] Fig.13a The nesting of parts with regular shapes using nesting software known in the prior art is shown.

[0075] Fig.13b shows the nesting of 500 parts with regular shapes using the first model for testing the method according to the present invention;

[0076] Fig.13c shows the nesting of 2000 parts with regular shapes using the first model for testing the method according to the present invention;

[0077] Fig.13d shows the nesting of 500 parts with regular shapes using the second model for testing the method according to the present invention;

[0078] Fig.13e shows the nesting of 2000 parts with regular shapes using the second model for testing the method according to the present invention;

[0079] Fig.14a The nesting of parts with irregular shapes using nesting software known in the prior art is shown.

[0080] Fig.14b shows the nesting of 500 parts with irregular shapes using the first model for testing the method according to the present invention;

[0081] Fig.14c shows the nesting of 2000 parts with irregular shapes using the first model for testing the method according to the present invention;

[0082] Fig.14d shows the nesting of 500 parts with irregular shapes using the second model for testing the method according to the present invention;

[0083] Fig.14e shows the nesting of 2000 parts with irregular shapes using the second model for testing the method according to the present invention;

[0084] Fig.15 The process of determining the GCI for all pairs of parts in a training phase of the method for selecting parts to be placed on a sheet according to the invention is schematically illustrated. DETAILED DESCRIPTION

[0085] Reducing material waste and operation time is the main goal of the cutting and packing problem (C&P). The solution of the C&P problem includes many steps, including grouping the items to be packed and arranging the clustered items on a large object. The current algorithm uses a meta-heuristic method to directly solve the arrangement problem without explicitly dealing with the grouping problem. In this work, the following new process is proposed for the packing problem: the new process starts with grouping the items to be packed and then arranging them on a large object. To this end, a new concept, namely the geometric compatibility index (GCI), is introduced and explained. Items with higher GCI should be clustered together. Since estimating GCI is an unsupervised problem, a reinforcement learning-based framework is proposed, which includes two graph neural networks (GNNs) trained in an actor-critic manner. In order to group the items into clusters, the estimated GCI in the graph is modeled as the inverse of the distance in the capacity-constrained vehicle routing problem (CVRP) and is solved using a meta-heuristic method. Experiments on a private dataset with both regularly shaped parts and irregularly shaped parts show that the proposed algorithm can achieve a significant reduction in operation time (34% to 50%) compared to open source nesting software, while obtaining a similar pruning loss on regular parts and a threefold improvement in pruning loss on irregular parts.

[0086] I. Introduction

[0087] The cut and pack problem (C&P problem for short) is related to many areas of operations research (OP) and has been extensively studied for decades. Over the past few decades, the C&P problem has been referred to in the literature by different terms [1] to [3].

[0088] C&P is an NP-hard problem with the same common logical structure that can be described as follows. The input is two sets of geometric data elements, namely a set of large objects and a set of small items. Then, the goal is to select a set of several or all small items, group these items into one or more subsets, and finally pack each of the obtained subsets on the corresponding large object in a certain pattern (also called layout) while satisfying some geometric conditions [4]. The first geometric condition to be satisfied is that the small items in each subset must be completely laid out in the sheet. And the second geometric condition is that all small items located on the same large object cannot overlap. The remaining unused area on the layout that is not covered by small items is usually regarded as trimming loss or waste. The goal of the C&P problem is to reduce this waste in order to maximize material utilization and reduce operation time. From an economic point of view, reducing trimming loss is very beneficial in very large-scale production such as metal sheet, wood and textile production, because reducing trimming loss can lead to material savings and therefore reduce production costs.

[0089] In general, the solution to the C&P problem can involve using a set of some or all large objects and a set of some or all small items, depending on the purpose and constraints of the problem. Considering the broad constraints, a formal formulation of five subproblems can be derived [3]:

[0090] 1) Large object selection problem: Select the large object on which the small items should be loaded or cut

[0091] 2) Small item selection problem: Select the small item to be filled or cut

[0092] 3) Grouping problem: Group the selected items into subsets

[0093] 4) Allocation problem: Allocating subsets of small items to large objects

[0094] 5) Layout problem: Arrange the small items on each of the selected large objects by first finding the order of the small items and then by placing these small items on the large objects with respect to geometric conditions.

[0095] In almost all C&P industries, the small items to be packed are provided by the customer, and therefore the second subproblem is ignored. If similar large objects are to be used, which is usually the case, the first and fourth subproblems are discarded. Therefore, the C&P problem is actually reduced to only the third and fifth subproblems, namely, grouping the parts to be packed into clusters and finding a layout for each cluster by arranging the small items of each cluster on the large object.

[0096] In most of the C&P industry, the C&P process is called a nesting process. Small items are also often referred to as parts, and large objects are referred to as sheets. The latter terms, nesting, parts, and sheets, will be used in this work from now on.

[0097] The fifth subproblem, the "layout problem", has been solved extensively over the past few decades, assuming that only one sheet is used for all parts and therefore that part clustering is not required. The layout problem has been extensively studied using exact methods [5] to [9], heuristic methods

[10] to

[14] , metaheuristic methods

[15] to

[18] , supervised machine learning

[19] ,

[20] , and reinforcement learning

[21] to

[26] . In contrast, the third subproblem, the "grouping problem", has generally been ignored. Known currently existing commercial nesting software solves the grouping problem implicitly as part of the layout problem. It starts with a single sheet for placing all parts, then discards parts that do not fit on the current sheet and places them on a new sheet.

[0098] In this work, a new nesting process is postulated, in which the grouping problem is solved directly before the layout problem, which will have a positive impact on both material utilization and nesting time. We believe that the pruning loss can be reduced if the clustered parts are already geometrically fitted together before placing them on the sheet (Figure 1). This in turn will reduce the nesting time due to the application of metaheuristic methods to a smaller number of parts in each cluster. Therefore, in this work, it is proposed to explicitly solve the grouping problem using a learning-based approach. The layout problem will be treated in subsequent work and is beyond the scope of this work.

[0099] To the best of our knowledge, this is the first learning-based approach for solving the grouping problem in the context of nesting. The grouping problem is an unsupervised learning task, since there are no labels or hints about the geometric fitness between the parts to be nested initially. In this work, the geometric fitness of the parts is quantified by a value called the "Geometric Compatibility Index" (GCI). With the help of a Graph Neural Network (GNN), the GCI is modeled as the weights of the edges of a fully connected Geometric Relation Graph (GRG). A meta-heuristic approach is used in a reinforcement learning (RL) manner to generate targets for updating the GNN parameters.

[0100] The workflow of this work is as follows. In Section III, the practical motivation for the concept of geometric compatibility and its impact on the pruning loss are provided. Then, in Section IV, the definitions and assumptions about the concept of geometric compatibility are introduced. In Section V, the current research status related to estimating edges in GNNs and their relationship with this work are presented, and some examples that will be used later in the methodology are also shown. In Section VI, an example of a dataset is shown and the preprocessing applied to the dataset is revealed. The architecture of the framework for estimating the GCI value of a given set of parts is also detailed. In Section VII-A, the impact of different nonlinear reward functions on the framework is studied, and their results are discussed. In Section VII-B, the performance of the framework is compared with existing open source nesting software. Finally, in Section VIII, this work is summarized and future research directions are proposed.

[0101] In summary, the main contributions of this work are as follows:

[0102] 1) A new process for pattern arrangement is proposed, which starts by solving the grouping problem before the layout problem.

[0103] 2) Two new concepts are defined, namely GRG and GCI, which will enable solving the grouping problem.

[0104] 3) To estimate GCI, we propose a novel RL-based framework that deploys two GNNs in an actor-critic manner.

[0105] 4) It is proposed to solve GRG as a capacity-constrained vehicle routing problem (CVRP) by using a metaheuristic approach, while the parts are grouped based on GCI.

[0106] 5) We demonstrate that the model can be trained on over 2,000 parts with irregular shapes. We report results on two test splits of a private dataset from real-world sheet metal production. The results show that the best model can achieve considerable improvements in operation time (34% to 50%), a 70% reduction in scrap in one test split, and comparable pruning losses in another test split.

[0107] II. Problem Statement

[0108] In this section, we formally define the grouping problem. Suppose there are N parts to be arranged on K sheets. n Indicates the area of ​​the part in g n Represents the geometric information of the part, and uses s k represents the area of ​​the sheet. In this representation, the geometric information encodes the inner and outer contours of each part. In this work, it is considered that g n is the latent space R d where d is the dimension of the latent space. Try to find It indicates whether part n will be assigned to sheet k. Parts assigned to the same sheet are grouped together. The optimization problem can be stated as follows:

[0109]

[0110] BB k is the rectangular bounding box enclosing all the parts arranged on sheet k. First, we solve the layout problem of sheet k, that is, Calculate BB after all parts are arranged on sheet k k Formula (1a) means finding the material waste on each sheet Minimized The constraint in (1b) states that each part must be assigned to one and only one sheet. The second constraint in (1c) states that the total area of ​​all nested parts on sheet k must not exceed the area of ​​the sheet s. k . And the last constraint in (1d) requires that each sheet must be used to its maximum capacity.

[0111] The optimization problem is ill-defined because Area(BBk) is an unknown nonlinear function of the geometry and area of ​​the nested parts. Area(BBk) can be expressed as:

[0112]

[0113] The solution to the grouping problem lies in finding a mapping function π that transforms the geometric information g of the parts into n 、The area of ​​the part a n And the sheet area s k Mapping to coefficients This coefficient determines the grouping.

[0114] III. Motivation of the Proposed Method

[0115] To find a solution to the grouping problem, we try to approximate the mapping:

[0116]

[0117] Since this mapping is difficult to estimate, we propose to decompose it into two mappings by introducing an intermediate value that can be learned from the data and then used as a criterion for grouping. For the intermediate value, a new concept is defined - the geometric compatibility index (GCI). The mapping π can be expressed as The first function π1 in the proposed solution attempts to obtain the geometric information of all parts To estimate the GCI between all parts. The set of all estimated GCIs is denoted as {GCI n,m}, where n and m are two different parts. The second function π2 estimates the coefficients according to GCI For example, parts with high GCI should be grouped together, while parts with low GCI should belong to different groups. In short, the mapping function can be expressed as:

[0118]

[0119] In the next two subsections, the proposed solution for estimating the mapping functions π1 and π2 is explained.

[0120] A. Mapping from Parts to GCI

[0121] The goal of this mapping function π1 is to estimate a quantitative criterion for clustering parts. It is assumed that geometric compatibility between parts is a reasonable choice for the grouping criterion. However, GCI is an abstract concept that is illustrated by a simple example before a formal definition is given in Section IV.

[0122] To show how choosing geometrically compatible parts can lead to greater material savings, three different simple parts (square, triangle and circle) are shown in Figure 2. In the example, a grey canvas is used to represent the area 22500mm 2A square sheet on which the parts are to be nested. Also assumed are two previously nested parts (Fig. 2(a)), which are shown in green and blue and have a diameter of 2812.5 mm. 2 and 7225mm 2 Then, the free area of ​​the resulting sheet is 12462.5mm 2 , which is theoretically enough to nest two more orange triangle parts with the same size as the green triangle part. By trying to nest the two new orange triangles, it is obvious that the next triangle will not fit on the sheet in this layout. The only option that allows the placement of a new orange triangle is to change the layout by moving the blue square slightly downwards (Figure 2(b)). However, this also prevents the placement of the second orange triangle, despite the fact that the available area on the grey sheet is theoretically still enough to accommodate the second orange triangle. On the other hand, a square with a radius of 30 mm and an area of ​​2827.43 mm 2 That is, a red circle of the same area as the green and orange triangles will fit perfectly into the sheet without even changing the layout. Furthermore, it leaves room for a second red circle to be laid out (Fig. 2(c)). Therefore, it can be said that the orange triangle is not as geometrically compatible with the blue square and the green triangle as it is with the red circle.

[0123] Alternatively, the red circle has a higher geometric compatibility with the nested part and therefore leaves more space for another red circle. Therefore, this increases area utilization and reduces material waste. In terms of quantity, the wasted area of ​​nesting only a single orange triangle is 9650mm 2 , while the wasted area of ​​​​the two red circles is 6807.6mm 2 Therefore, higher geometric compatibility saves more area for new parts to be nested and reduces trimming loss. Note that geometric compatibility should not be confused with area suitability. The red circle and orange triangle have the same area, the former is geometrically compatible, while the latter is not. In addition, the area is less than or equal to 3257.32mm 2 Circles of different sizes will also fit perfectly without changing the layout.

[0124] As shown in the example above, the GCI estimation problem cannot be estimated in a pairwise manner without observing the other parts in the group. To this end, we first assume that all given parts form a large group. In order to estimate the GCI between all parts, we choose to model the parts as nodes in a graph and model the GCI as weighted edges between nodes. This graph is called a geometric relation graph (GRG), and a graph neural network (GNN) is used to estimate the weights of its edges.

[0125] B. Mapping from GCI to Groups

[0126] After estimating the GCI, it is proposed to use CGI as the criterion for grouping. Mapping π2 can be seen as the solution of another optimization problem, which tries to find the optimal set of parts that should be selected together to achieve the maximum sum of GCI of different sheets. This optimization problem can be expressed as:

[0127]

[0128] This new formulation, while satisfying the constraints in (1b) to (1d), is similar to the typical Capacity Constrained Vehicle Routing Problem (CVRP) formulation, which attempts to find an optimal set of routes for a fleet of vehicles to travel. In this analogy, GCI is the inverse of distance (distance = 1-GCI), sheets are vehicles with limited capacity (area) and parts are cities in the typical CVRP problem. To solve π2, a metaheuristic approach is used instead of an exact method to find a good solution in an acceptable time.

[0129] C. Learning Paradigm

[0130] It is clear from subsection III-A that the GCI cannot be specifically estimated in a defined way before solving the layout problem. Furthermore, it is not possible to obtain a label for the GCI because the GCI between parts changes whenever the grouping strategy changes. Using a brute force approach and trying all grouping permutations to generate GCI labels for supervised learning is very computationally expensive. To this end, it is proposed to learn the GCI by maximizing a reward function in a nesting environment using reinforcement learning (RL). The reward function R is chosen as the ratio between the area of ​​a group of parts and the area of ​​the sheet on which these parts are nested, and is calculated for each group of parts as follows:

[0131]

[0132] IV. Definitions: GRG and GCI

[0133] In this section, formal definitions of GCI and GRG are given.

[0134] Definition 1 (GRG). GRG is a fully connected bidirectional weighted graph, representing the parts to be arranged together with their geometric relationships.

[0135] Graph: GRG depicts the parts to be arranged as nodes / vertices in space, and depicts the geometric relationships of nodes / vertices as edges connecting nodes / vertices. GRG has nodes v n ∈V={v1,……,v n}, where each node is represented by a geometric information vector g n ER d express.

[0136] Fully connected weighted graph: To outline all geometric relationships between parts, all nodes of the GRG are connected by edges. Each part is connected to itself and to each other. GRG is connected to the two parts by the connecting edge e nm The weight w nm ∈[0, 1] to limit the two parts v n With v m The affinity of the geometric relationship between them.

[0137] Bipartite graph: In the example of subsection III-A, packing the triangular parts (green) before the square parts has high geometric compatibility, whereas packing the triangular parts (orange) after the square parts has low geometric compatibility. This means that the order in which the parts are packed affects the geometric fitness of these parts, which is represented by the directed edges between the nodes in the GRG, i.e., w nm ≠w mn .

[0138] Definition 2 (GCI): GCI is a scalar variable with fixed bounds that is computed graph-wise rather than pair-wise. The value of GCI is the edge e in GRG. nm The weight w nm ,Right now, It expresses the geometric relationship between two parts that are to be nested together on the same sheet in the sense of their geometric compatibility rather than their area compatibility.

[0139] Scalar variable with fixed bounds: In order to specifically understand the substantial impact of GCI on packing, GCI is designed as a scalar value ranging from 0 to 1. A GCI of 0 means no geometric compatibility, while a GCI of 1 is the highest compatibility index.

[0140] · Graph-by-graph rather than pair-by-pair: GCI is not determined by geometrically evaluating how well each pair of parts fit together (in Figure 2(a) the green triangle looks compatible with the blue square, but the orange triangle is barely compatible with the same rectangle in Figure 2(b)), but rather by laying out all the parts of a GRG graph / subgraph on a sheet and evaluating the resulting layout. Since there is no explicit computational method for GCI, it can instead be inferred via machine learning.

[0141] Representing geometric compatibility rather than area compatibility: Two different nodes in the GRG that are compatible in area with a third node, i.e. the sum of their areas will fit in the sheet, do not necessarily have to have the same GCI as that node. This is the case with the orange triangle and the red circle in Figure 2 and their compatibility with the blue square.

[0142] V. Related Work

[0143] A. Cutting and filling

[0144] Cut and Pack Problem. The C&P problem exists in various industries and has been studied intensively for a long time

[27] . The C&P problem appears repeatedly in the literature and has several name variations, such as the layout design problem in

[28] , the spatial configuration problem in

[29] , the cutting or trimming loss problem, the box or strip filling problem, the vehicle, pallet or container loading problem, the packing problem, the knapsack problem, etc. The C&P problem appears in various forms and variations, depending on the purpose of the problem at hand and the constraints present during the problem treatment. The main objectives of the C&P problem are to reduce the computational time and maximize the material utilization by reducing the trimming loss and improving the packing efficiency. The latter objective is particularly important for large industries with large-scale production, as reducing the total trimming loss leads to significant savings

[30] . The most common packing problem in the literature is the layout problem. Over the years, several techniques have been developed to solve the layout problem, which can be categorized as: exact methods [6], [7],

[31] , heuristic methods

[10] to

[12] ,

[32] and meta-heuristic methods

[15] to

[18] . Recent methods

[25] ,

[26] ,

[33] utilize deep reinforcement learning (DRL).

[0145] Contour similarity and GCI. The concept of “contour similarity” was introduced in the literature of 2D binning

[34] ,

[35] in order to increase compactness and reduce scrap while solving the layout problem. The goal of contour similarity is to match new parts to unoccupied closed polygons of the sheet with similar contours. Time-exhaustive methods for finding contour similarity were first studied by searching for the optimal angle of the new part with a fixed angle step size

[36] –

[38] . More recently, contour similarity in the field of additive manufacturing has been studied by Freeman chain codes in

[34] and using the longest similarity subsequence (LCSS) method in

[35] . In this work, the concept of geometric compatibility is introduced, which differs from the concept of contour similarity in different aspects. (1) Contour similarity is a special case of geometric compatibility; if two parts are geometrically compatible, they will be nested together on the same sheet, but the two parts are not necessarily adjacent on the sheet, as is the case with contour similarity. (2) Contour similarity is computed pairwise and would be time consuming if computed for all possible pairs within a set of parts. On the other hand, GCI is computed graph-by-graph and considers the relationships between all parts in the GRG simultaneously. For example, in

[35] , an iteration-based algorithm, LCSS, is used to compute contour similarity, which has a time complexity of O(n·m) in the best case, where n and m are the lengths of the two input sequences. GCI is estimated using a learning-based algorithm, which has a time complexity of O(1) at inference time. (3) Finally, contour similarity targets the layout problem, unlike GCI which focuses on the grouping problem.

[0146] B. Graph Neural Networks

[0147] GNNs are widely used for classification. There are three typical classification tasks in graphs: node classification, edge classification, and node and edge classification. Classification tasks are either based only on node features (e.g., GraphSAGE

[39] and DeepWalk

[40] ) or on both node features and edge features (e.g., EGNN

[41] ). Graph convolutional neural network (GCNConv) is a convolution operator first introduced in

[42] . GCNConv is one of the most widely used architectures for classification in graphs. Most of the graph neural network models share a largely common architecture based on aggregating information from neighboring nodes into the current node using a message passing function. The main message passing function used in the forward path of GCNConv is given by

[0148]

[0149] in

[0150]

[0151] In equation (7), A is the adjacency matrix of the input graph indicating which nodes are connected together. If the input graph is a fully connected graph, the adjacency matrix will be a matrix of unity, which is suitable for GRG. An adjacency matrix that represents the insertion of self-loops, i.e., each node is connected to itself, is depicted by the identity matrix I. When an adjacency matrix contains values ​​indicating how tightly nodes are connected to each other, rather than just values ​​indicating the connectivity of nodes (zeros and ones), the adjacency matrix is ​​also called an "edge weight matrix." is a diagonal node degree matrix, where the diagonal elements correspond to the number of edges connected to the node, i.e. In addition, X represents the matrix of node feature vectors, is a trainable weight matrix for the GCNConv layer, and X′ is the weighted aggregation of features of neighboring nodes.

[0152] In this work, the values ​​of the adjacency matrix are determined with the help of GCI. However, GCI is not known a priori and therefore has to be learned. Learning GCI corresponds to the edge labeling problem. However, current methods such as EGNN

[41] cannot be used because they assume the existence of some node labels which are not present in the case of this work. For this reason, a hybrid method using DRL is proposed which combines previous techniques to overcome the problem of missing labeled data.

[0153] C.CVRP

[0154] The vehicle routing problem (VRP) is an NP-hard optimization problem and is essentially a generalization of the well-known traveling salesman problem (TSP). The vehicle routing problem is used in supply chain management to design optimal routes for a fleet of vehicles to serve customers under a set of constraints. The common constraint between all VRP types is to minimize the global distance between customers and the number of vehicles required to serve all customers. VRP has been extensively studied in the literature using many heuristic methods and meta-heuristic algorithms surveyed in

[43] ,

[44] . There are two main variants of VRP: VRP with time window (VRPTW) and VRP with constrained capacity (CVRP). The third constraint of VRPTW is to serve all customers within a predefined time window. CVRP assumes that all vehicles in the fleet have uniform capacity and that the customer demand for individual items is known. In order to solve CVRP, a third additional constraint must be satisfied: each vehicle must not exceed its capacity. This work draws an analogy between GRG and CVRP to group parts based on their GCI.

[0155] D. Actor-Critic Reinforcement Learning

[0156] Reinforcement learning is a decision-making framework in which an agent represents a problem solver and an environment represents the problem to be solved. At each state in the problem to be solved, the agent interacts with the environment by choosing an action to apply. By applying an action, the environment updates its state and returns to the agent the new state and a scalar reward that evaluates the quality of the action taken. At any state, the agent chooses an action by following a policy. The agent's goal is to optimize its policy by maximizing the total cumulative reward, also known as the return. To decide what actions to take in future states, the agent indirectly uses past experience by constructing a reward estimation function called a value function. A widely used agent paradigm is the actor-critic paradigm, in which the actor tries to optimize the policy and the critic tries to optimize the value function. Actor-critic algorithms can be classified based on two criteria

[45] : whether a discount

[46] to

[48] or average reward

[49] to

[51] setting is used; and whether the standard (also known as vanilla) gradient

[52] to

[54] or natural gradient

[55] ,

[56] is used to update the parameters of the policy. The central concept behind all actor-critic algorithms is to use a value function estimated and optimized by the critic to steer the actor's strategy in the direction of improved performance.

[0157] VI. Methodology

[0158] In this section, the proposed approach to solve the grouping problem is presented. First, an overview of the different components of the approach is presented, followed by the architecture of the part encoding, actor and critic modules, and the learning routines.

[0159] A. Overview of the Proposed Method

[0160] The method consists of finding the mappings π1 and π2. The proposed framework consists of four components:

[0161] 1) Input encoder: The input encoder takes part n as input and generates a geometric information vector g of the part n as output.

[0162] 2) Class Actor-Critic Agent: The class actor-critic agent represents the first strategy π1 mapping from parts to GCI. The class actor-critic agent consists of two modules, namely the class actor module and the class critic module. The input of this component is the set {g1,…,g N}, and its output is GRG, where the parts are nodes and the weights of the edges are GCI. During the training phase, the actor-critic agent learns the value of GCI with the help of the reward signal provided by the nesting environment in the RL framework, which improves the quality of the nesting. During the inference phase, the actor-critic agent predicts the value of GCI to construct the GRG.

[0163] 3) CVRP Solver: The CVRP Solver represents the second strategy π2 mapping from GCI to groups. This component treats the GRG constructed by the previous component as a CVRP problem and solves it using a meta-heuristic approach. The input of the CVRP Solver is the GRG, and the output is a subgraph of the GRG as a route. Each resulting GRG subgraph represents a group of parts to be nested together on the same sheet.

[0164] 4) Nesting environment: The nesting environment solves the layout problem for the parts of each GRG subgraph and returns the reward of Equation (6) to the class actor-critic agent during the training phase. During inference, the nesting environment only solves the layout problem for each group of parts without returning any reward.

[0165] Figure 4 shows a block diagram of the four components. In the second and third subsections, the first and second components will be described in detail, respectively. The third and fourth components will be described in the learning routine of the last subsection because the third and fourth components cannot be described separately.

[0166] B. Input code

[0167] The input of this algorithm is a set of parts to be arranged, where each part is described by its area and geometry. To ease the task of the first strategy π1, the geometry of each part, described as a rasterized image, is encoded as a geometric information vector. Inspired by representation learning, as a preprocessing step, an autoencoder is used to encode the rasterized image of each part into R d The vector g in n , where d = 256. The encoding of the parts will help the GCI learning task and optimize the required computing resources by reducing the dimensionality. The encoded geometric information vector g of part n n It will be further used as the feature of its GRG node in Subsection VI-C. A custom autoencoder based on the Inception Network

[57] is trained on a different dataset of more than 10,000 rasterized parts including some layout images. To evaluate the generalization of the encoding to new images, for a single part ( Figure 3 (a)) and layout ( Figure 3 (b)) The autoencoder is tested on new, unseen images.

[0168] C. Actor-Critic Architecture

[0169] Inspired by the concept of “learning with critics”

[58] , which inspired Sutton, Barto, and Anderson

[59] to outline the basic concepts of all modern actor-critic algorithms in the field of RL, this model consists of two modules: an actor-like module and a critic-like module ( Figure 5 ).

[0170] a) Class-Critic Module: In the class-critic module, the nodes of a subgraph of GRG pass through two consecutive GCNConv layers to perform message passing between the nodes. The rationale for using a subgraph instead of the entire GRG will become clear in the “Data Collection” step of Subsection VI-D. Both GCNConv layers initially take a random edge weight matrix representing the GCI. The output of the first GCNConv layer is then provided as input to the second GCNConv layer. Theoretically, in Eq. (8), the node geometry information vector matrix [g1,…,g N ] represents X, and the edge weight represents matrix and are learnable parameters.

[0171] To guide the improvement of the actor-like module, the output of the last GCNConv layer is averaged into a single value that estimates the reward of the environment. This value is compared with the true reward signal from the environment using the L1 loss function. The L1 loss is further used in the back-propagation using stochastic gradient descent (SGD) to update the parameters of the 2 GCNConv layers and the edge weight matrix. Each parameter of this edge weight matrix represents the GCI between its connected nodes (i.e., parts).

[0172] Class actor module: The class actor module consists of: a GCNConv layer that takes the nodes of GRG as input; and an adjacency matrix of (because it is assumed that all nodes are connected). The goal of the class actor module is to learn the forward mapping between the input graph nodes and the edge weights The forward map is a matrix of dimension N×N, where N is the number of nodes in the graph. After updating the edge weights of the critic class module, the output A of the actor class module is calculated. b Estimate the edge weights A learned by the class critic module b Then, the parameters of the GCNConv layer of the actor-like module are updated by SGD in the back-propagation.

[0173] In summary, the actor-like module learns the edge labeling task.

[0174] One might mistakenly think that the critic-like module does the entire task by learning edge weights, and that the actor-like module is redundant. In fact, this is partially true. The critic-like module learns the GCI by backpropagating the learnable parameters of the edge weight matrix. However, the critic-like module maps input nodes and the GCI to a reward function, and only learns the GCI by backpropagation, which is not possible at test time.

[0175] In short, it can be said that the critic-like module has no memory of the learned knowledge. The role of the actor-like module is to generalize the knowledge learned by the critic-like module.

[0176] a) Class Actor-Critic vs. Class Actor-Critic: The actor-critic paradigm and the class actor-critic paradigm are conceptually consistent, but different in their implementation. The critic in the actor-critic paradigm implicitly improves the actor by suggesting the direction of updated actions from the evaluation signal (e.g., v-value, q-value, a-value, or reward). On the other hand, the class critic in the class actor-critic paradigm explicitly learns good actions and teaches the actor these actions by directly estimating the reward signal. The actor and the critic have an "action then get criticism" dynamic: the actor first takes an action, and then the critic evaluates its performance. The dynamic between the class actor and the class critic is more precisely "get criticism and then learn": the class critic learns the explicit action by evaluating the action, and then the class actor learns the action.

[0177] D. Learning routines

[0178] The GCI of GRG is learned by simply placing the parts of GRG on a sheet and evaluating the layout quality in an RL framework. In this section, the learning process of GCI (Algorithm 1) and the corresponding learning process of GRG are described in detail.

[0179] a) Initialization of the environment: The first step is to initialize the nesting environment. In the RL context, the action space includes all sets of parts that can be nested together on the same sheet, that is, each action is a subgraph of the GRG. The state space only includes the initial state that is the set of all parts in the GRG. In other words, there is only one state as the entire GRG graph, and each action is a subgraph of the GRG whose parts the agent believes will fit together geometrically on the same sheet. The environment adopts the subgraph, nests its parts on the same sheet, and returns the quality of the resulting layout (Eq. (6)) to the agent as a "reward".

[0180] b) Initialize the replay buffer: In order to store the experience collected from the agent and environment interaction, the replay buffer is initialized for the GRG. Each entry in the replay buffer is a pair of binary mask and reward. The mask filters out all parts of the GRG that are not included in the subgraph. The replay buffer has a limited capacity, and once the replay buffer is full, the latest experience replaces the oldest experience. This allows off-policy training because the experience collected from the old policy is used to learn the new policy.

[0181] c) Data collection: In the data collection step ( Figure 6), the GCI between all parts of the GRG is inferred from the class actor module. Then, the GRG is regarded as a CVRP and solved. An analogy between CVRP and GRG is made in Figure 7: the black numbers in Figure 7(b) are the inverse of the GCI values, i.e. 1-GCI, and are similar to the distances between the points in Figure 7(a). Both need to be minimized. On the other hand, the red numbers in Figure 7(b) are the areas of the parts, which are similar to the capacities of the points in Figure 7(a). Both need to satisfy the capacity constraints. Finally, the available empty sheets for arranging the subgraphs of the GRG correspond to the available vehicles in their CVRP corresponding parts. Among several metaheuristic algorithms for processing CVRP, Google's OR Tools are used to solve the GRG into subgraphs. Then, each subgraph is passed to the environment to be arranged on the same sheet and the reward is returned. The masks and rewards of the subgraphs are then stored in the playback buffer.

[0182]

[0183] d) Update step: To update the actor-like module and the critic-like module, a batch of experiences is randomly sampled from the replay buffer. The mask of each element in the batch is used to restore the subgraph, and the reward is used to calculate the L1 loss and update the critic-like module. After the update of the critic-like module, the actor-like module uses the entire GRG to estimate the GCI. These GCIs are again compared with the edge weight matrix learned by the critic-like module in the L2 loss function to update the actor-like module ( Figure 6 ). Note that the critic-like module learns the GCI of the entire GRG by using only subgraphs from the GRG. This design choice is inspired by the fact that geometric compatibility should be evaluated only for clusters of parts to be sampled through reward signals. Therefore, by evaluating the quality of many subgraphs, the critic-like module can infer the geometric compatibility between all parts of the GRG. However, the critic-like module is a direct model that cannot generalize its learned knowledge of edge weights to unseen GRGs. On the other hand, the actor-like module is an inductive model that can easily predict the GCI of newly encountered GRGs.

[0184] e) Scaling to large GRGs: To limit computational resources, the GRG is restricted to include at most N parts. To scale the model for larger datasets containing N parts, where N>N, the large GRG is divided into smaller GRGs, each GRC has N parts ( Figure 8). The learning process described above is then applied to each GRG. At the beginning of training for each GRG, the edge weight matrix for the class critic module and a new playback buffer are initialized. The edge weight matrix of the class critic module is randomly initialized only for the first GRG. However, for all subsequent GRGs, the edge weight matrix learned previously is used for initialization. This can be considered a simple form of transfer learning. In the inference phase, only the class actor model is used to estimate the GCI in the test partition.

[0185] VII. Experiments and Results

[0186] Several experiments are performed to demonstrate the effectiveness of the proposed model. In Section VII-A, the impact of different choices of reward functions on a single GRG with N = 500 is studied. Next, in Section VII-B, the performance of the proposed model is compared with open source nesting software.

[0187] Nesting environment. For this experiment, the following virtual environment was developed: this virtual environment does not solve the layout problem but only considers the area of ​​the parts to calculate the reward of the currently nested parts. There are several reasons behind the choice of a virtual environment. First, compared with GNN, the real nesting environment is very slow in solving the layout problem, and its training time will increase dramatically due to the increase in data collection time. Second, since the proposed algorithm is not limited to a specific layout solver algorithm, the virtual environment can be regarded as a more general and abstract case of the real nesting environment. This design choice constitutes a proof of concept of the GCI and GRG concepts. It is impossible to implement the reward function (Eq. (6)) in a virtual environment because the reward is Area(BB K ) function, the Area(BB K ) is a nonlinear function of the geometry and area of ​​the laid-out parts (Eq. (2)), which can only be calculated after solving the layout problem. However, in the virtual environment, the layout problem is not solved and the bounding box area cannot be calculated. In order to approximately simulate the nonlinearity of the reward in (6), the reward function of the virtual environment is designed as a nonlinear function F of the utilization rate U. For the part to be laid out in the area s k Each subgraph (part group) formed by the CVRP on sheet k has a utilization rate U k and reward R k The calculation is as follows:

[0188]

[0189] For the choice of F, the impact of two different nonlinear functions has been studied in Subsection VII-A.

[0190] Dataset. This dataset includes 2500 CAD files representing different irregular complex geometric parts used in sheet metal production. The parts used are to be nested on sheets of size 3000mm×3000mm. The CAD files of the parts are converted into scalable vector graphics (SVG) format. In order to capture the size differences between the parts, the canvas is set to a constant reference system. Each part is placed at the upper left corner of the canvas and rasterized into a binary image of 128×128 pixels, as shown in Figure 11(b). The dataset is partitioned into a training set (2000 parts) and a test set of 500 parts. Two test partitions are further selected from the test set and used to evaluate the model. Test partition 1 contains 113 parts with regular shapes, while test partition 2 contains 124 parts with irregular shapes. Regular shapes mainly contain straight lines and smooth curves. On the other hand, irregular shapes have more complex concave and convex curves. The test partitions are selected based on manual inspection. All parts in the test partition are different from each other and different from the training partition.

[0191] A. The impact of different reward functions on model generalization.

[0192] In this set of experiments, only 500 parts are selected from the training set to train the model, and these parts can be grouped into a GRG. Two different nonlinear functions F are tried for reward calculation to test the generalization ability of the proposed model. The first model is trained using a sine function of utilization, as follows: R k = sin((U k -0.5)·n). The range of utilization is set to [-0.5, 0.5] to make the reward range between [-1, 1]. The second model uses a sigmoid function for the reward. This reward is also non-linear and represents different challenges for the agent. The reward function is expressed as follows: In this case, the range of the reward is [0,1].

[0193] Model a. In this model, the reward function is a nonlinear sine function. Figure 9(a) shows the edge weight error L2 during 500 training steps under the sine reward function. It can be noted that the L2 loss converges quickly to a relatively small value. Figure 9(b) shows the reward loss L1 that exhibits the same convergence behavior.

[0194] Model b. In this model, a Sigmoid reward function is used. Figure 10(a) shows the edge weight error L2 during 500 training steps. Similar to model a, the L2 loss and L1 loss converge quickly (see Figure 10(b)).

[0195] The estimated GCI for model a and model b on test partition 1 are shown in Figures 12(a) and 12(b). Qualitatively, it can be seen that these plots are similar except for a few cases.

[0196] A.GRG and existing nesting software

[0197] To the best of our knowledge, the nesting subproblem of grouping the parts to be nested into clusters before doing the nesting itself has never been handled by any nesting software before. Almost all current nesting software try to solve two other nesting subproblems, namely, finding the best order of the parts to be nested and choosing the angle and xy position of each part on the sheet, i.e. the nesting / packing itself. After the nesting order of the parts is chosen, usually by a metaheuristic approach, and during the packing of the parts on the sheet, parts that do not fit on the current sheet are placed on a new sheet. This is how the clustering subproblem is solved in the nesting software without taking into account the suitability of the parts to be nested together.

[0198] To understand how our approach compares to other existing frameworks, we conducted the following experiments: DeepNestPort DeepNest-Port is essentially a SVGNest A port of a browser-based vector nesting tool from SVGNest to handle different input / output image formats. DeepNestPort was chosen as the baseline for comparison for three reasons: (1) DeepNestPort is open source software that is easily accessible for research purposes; (2) SVGNest claims to have similar performance to commercial software; and (3) other learning frameworks for solving the layout problem operate only on a small number of regular parts and are not suitable for operating on a large number of regular and irregular parts. It should be noted that only a simple virtual environment was used in the training of the entire model, which is expected to have a negative impact on the results. Training using one of the existing nesting software as an environment is not possible because they are very slow and therefore not suitable for RL training that requires a large number of samples to learn.

[0199] Baseline and Models. In this experiment, all parts in each test split are first fed to the DeepNestPort framework for nesting. The performance of this experiment is considered as the baseline. Then, the performance of the four models is reported:

[0200] Model a and model b were trained twice for 500 parts and 2000 parts. For each model, the GCI values ​​of the parts were estimated and the GCI values ​​were divided using CVRP. Then, each group of parts corresponding to a specific route in the CVRP problem was nested on the sheet using DeepNestPort. The main difference is that DeepNestPort assigns parts to each sheet independently, while in this case, parts are assigned according to their GCI.

[0201] Metrics. Two metrics are used to evaluate the performance of the model: Time (T) and Wasted Material (WM). Time (in seconds) is calculated by measuring the nesting time of all used sheets after the recommended grouping operation. Wasted material (expressed as a percentage) is the ratio of the sum of the waste areas of all used sheets to the total area of ​​the sheet. The wasted area of ​​a sheet is considered only as the unused area within the rectangular bounding box that encloses all the parts nested on the sheet. The unenclosed area can be reused for nesting new parts. Therefore it is not considered as waste.

[0202] Results and Discussion The results are reported in Table I. In the case of using DeepNestPort, a total of four sheets and two sheets with a sheet size of 3000 mm × 3000 mm each are required to nest the required 113 parts and 124 parts in test partitions 1 and 2, respectively (for layout visualization, refer to Appendix VIII). Test partitions 1 and 2 were nested with an average time of 600 seconds and 761 seconds, and the material waste percentage in this case was 13% and 20% of the total area of ​​the sheets used, respectively. In the case of using GCI, a total of 4 sheets are recommended for test partition 1 and a total of 2 sheets are recommended for test partition 2.

[0203] From the table, first notice that model a generalizes better than model b in all combinations. This empirically suggests that sinusoidal rewards are more suitable than sigmoid rewards. Second, both models perform better than the baseline for test partition 2, where model b causes only 7% material waste when trained on 2000 parts, almost one-third of DeepNestPort. Model b still achieves considerable reduction when trained on only 500 parts. The difference in the two results can be explained by the number of parts during training. However, both models a and b experience a slight degradation in performance for test partition 1. It is hypothesized that this effect can be mitigated by designing a more realistic nesting environment or adjusting the choice of reward function.

[0204] Overall, the framework consistently achieves shorter operation times than the classical nesting approach in all combinations. Regarding test partition 1, the operation time was reduced by 48% on average, and regarding test partition 2, the operation time was reduced by 30% on average. The longer time in test partition 2 is attributed to the irregular shapes of the parts. This stems from the fact that implicitly solving the grouping problem during the layout problem leads to a large number of combinations that the nesting environment should consider. In contrast, the proposed process divides the parts to be nested into small groups with more compatible parts, which in turn can be directly nested on the sheet in fewer combinations.

[0205] Limitations. One limitation of this work is that no nesting environment was used during training. However, this is a promising result as the model can achieve considerable reductions in time and scrap for irregular parts. It is believed that using a realistic nesting environment during training would further reduce material waste. However, the design of such an environment is beyond the scope of this work and will be considered in future work.

[0206] Another limitation of the proposed framework is that the use of metaheuristics to solve the CVRP problem may lead to suboptimal results or even no results at all within an acceptable time frame. It is planned to address this issue in future work by replacing the metaheuristics with plug-and-play learning-based methods such as

[60] . Overall, these results validate the hypothesis that a different flowsheet design, in which the grouping problem is solved before the layout problem, can significantly speed up the nesting process and reduce the amount of scrap generated.

[0207]

[0208] The above table (Table I) shows the comparison between GRG and DeepNestPort. Test partition 1 contains regular parts, while test partition 2 contains irregular parts. The best number is marked in bold.

[0209] VIII. Conclusions on Figures 1 to 14

[0210] In this work, the conventional nesting process is revisited by solving the grouping problem of parts directly before solving the placement problem. To achieve this, two new concepts, namely GRG and GCI, are introduced and formally defined. An RL-based framework for estimating the GCI of parts in an unsupervised manner is proposed. The framework consists of two GNNs trained in an actor-critic manner, where the actor estimates the GCI and the critic judges its quality by fitting the reward function of the nesting environment. The proposed framework is tested for two virtual environments with different nonlinear reward functions. Comparisons with open source nesting software are performed to demonstrate the performance of the proposed grouping and nesting process. The effectiveness of the framework is demonstrated by achieving a threefold improvement in waste and a 34% improvement in operation time for test partitions with irregular shapes. Similarly, for additional test partitions with regular parts, the model achieves a 50% time reduction with minimal sacrifice in waste. In future work, it is planned to complete the process by developing an intelligent nesting algorithm for the layout problem.

[0211] Fig.15 The process of determining the parameter GCI (Geometric Compatibility Index) for all pairs of parts in the training phase 100 of the method according to the invention for selecting parts to be placed on a sheet is schematically shown. In a first step 102, the geometric features of each part are encoded in a corresponding geometric information vector, wherein, for each part, one of the geometric features is the projected area of ​​the part in the plan view (wherein the first step 102 can also be performed independently of the training phase). In a second step 104, a graph is generated, wherein the geometric features of each part are assigned to one node of the graph at a time. All nodes in the graph are connected in pairs by one edge at a time (wherein the second step 104 can also be performed independently of the training phase). In a third step 106, an estimate of the GCI is set, wherein each estimate of the GCI is respectively assigned to a pair of parts. In addition, first edge weights are assigned to the edges of the graph, wherein each first edge weight depends on the corresponding GCI assigned to the edge (i.e., the corresponding pair of parts). In a fourth step 108, the parts are assigned to the sheet. The assignment is performed by determining a subgraph of the graph passing through the node by means of an optimization method. The nodes passed by the respective subgraph represent parts to be placed on the sheet, wherein the sum of the projected areas of the parts on each subgraph is at most equal to the respective sheet size. The optimization method is in particular a method for solving the capacity constrained vehicle routing problem (CVRP).

[0212] In a fifth step 110, a reward function value for each subgraph is determined as a measure of how much surface area of ​​the corresponding sheet is covered by the parts on each subgraph after the parts are arranged on the corresponding sheet. In a sixth step 112, a fitness value for each subgraph is determined in the form of an arithmetic mean (particularly) of the output values ​​of the corresponding subgraph, wherein the output values ​​take into account the estimated value of the GCI. The output values ​​are calculated in particular by the class critic module using a graph convolutional neural network. In a seventh step 114, a reward loss is determined for each subgraph, wherein the reward loss depends on the reward function value and the fitness value. In an eighth step 116, the GCI is re-determined for each subgraph so that the reward loss is reduced on each subgraph. In a ninth step 118, steps 110 to 116 are repeated until the reward loss meets the first termination criterion. In particular, the value of the GCI in the class critic module is determined in this way. In a tenth step 120, a second edge weight of the edge of the graph is estimated. The tenth step 120 is performed in particular by the class actor module. Furthermore, an edge weight loss at the edge of the graph is determined, wherein the edge weight loss depends on the second edge weight and the GCI at the corresponding edge determined in the ninth step 118 .

[0213] In an eleventh step 122, step 120 is repeated, wherein estimating the second edge weight is performed so that the edge weight loss is minimized until the edge weight loss satisfies the second termination criterion. In a twelfth step 124, the value of GCI is obtained from the process according to step 122. Thereafter, steps 106 to 124 are repeated, wherein the estimated value of GCI in step 106 is set to the second edge weight last determined in step 124, until the estimated value of GCI satisfies the third termination criterion.

[0214] References

[0215] [1] M. Arenales, R. Morabito, and H. Yanasse, “Special issue: Cutting and packing problems,” Pesquisa Operacional, Vol. 19, No. 2, pp. 107–299, 1999.

[0216] [2] H. Dyckhoff, “A typology of cutting and packing problems”, European Journal of Operational Research, Vol. 44, No. 2, pp. 145–159, 1990.

[0217] [3] G. H. Haubner and H. Schumann, “An improved typology of cutting and packing problems”, European journal of operational research, Vol. 183, No. 3, pp. 1109–1130, 2007.

[0218] [4] L. Faina, “A survey on the cutting and packing problems”, Bolletino dell’Unione Matematica Italiana, Vol. 13, No. 4, pp. 567–572, 2020.

[0219] [5] C. Chen, S.-M. Lee and Q. Shen, “An analytical model for the containerloading problem”, European Journal of operational research, vol. 80, no. 1, pp. 68–76, 1995.

[0220] [6] A. Lodi and M. Monaci, “Integer linear programming models for 2-staged two-dimensional knapsack problems”, Mathematical Programming, vol. 94, no. 2, pp. 257–278, 2003.

[0221] [7] A. Lodi, S. Martello, and D. Vigo, “Models and bounds for two-dimensional level packing problems”, Journal of Combinatorial Optimization, vol. 8, no. 3, pp. 363–379, 2004.

[0222] [8] J. Puchinger and G. R. Raidl, “Models and algorithms for three-stage two-dimensional bin packing”, European Journal of Operational Research, Vol. 183, No. 3, pp. 1304–1327, 2007.

[0223] [9] F. Furini and E. Malaguti, “Models for the two-dimensional two-stage cutting stock problem with multiple stock size,” Computers & Operations Research, Vol. 40, No. 8, pp. 1953–1962, 2013.

[0224]

[10] E. Burke, G. Kendall, and G. Whitwell, “A new placement heuristic for the orthogonal stock-cutting problem,” Operations Research, vol. 52, no. 4, pp. 655–671, 2004.

[0225]

[11] S. Imahori and M. Yagiura, “The best-fit heuristic for the rectangular strip packing problem: An efficient implementation and the worst-case approximation ratio,” Computers & Operations Research, vol. 37, no. 2, pp. 325–333, 2010.

[0226]

[12] J. Verstichel, P. De Causmaecker, and G. V. Berghe, “An improved bestfituristic for the orthogonal strip packing problem,” International Transactions in Operational Research, Vol. 20, No. 5, pp. 711–730, 2013.

[0227]

[13] E.¨Ozcan, Z. Kai and J.H. Drake, “Bidirectional best-fit heuristic considering compound placement for two dimensional orthogonal rect-angular strip packing,” Expert Systems with Applications, Vol. 40, No. 10, pp. 4035–4043, 2013.

[0228]

[14] E. Hopper and BC Turton, “A review of the application of meta-heuristic algorithms to 2d strip packing problems”, Artificial Intelligence Review, vol. 16, no. 4, pp. 257–300, 2001.

[0229]

[15] ——, “An empirical investigation of meta-heuristic and heuristical go-rithms for a 2d packing problem”, European Journal of Operational Research, Vol. 128, No. 1, pp. 34–57, 2001.

[0230]

[16] S. C. Leung, D. Zhang, and K. M. Sim, “A two-stage intelligent search algorithm for the two-dimensional strip packing problem,” European Journal of Operational Research, vol. 215, no. 1, pp. 57–69, 2011.

[0231]

[17] K. and F. Glover, “Metaheuristics,” Encyclopedia of operations research and management science, volume 62, pages 960–970, 2013.

[0232]

[18] E. Hopper and B. Turton, “Application of genetic algorithms to packing problems—a review”, Soft Computing in Engineering Design and Manufacturing, pp. 279–288, 1998.

[0233]

[19] P. Poshyanonda and C. H. Dagli, “Genetic neuro-nester,” Journal of Intelligent Manufacturing, vol. 15, no. 2, pp. 201–218, 2004.

[0234]

[20] R. Labib and R. Assadi, “Modified multi-layered perceptron applied to packing and covering problems”, Neural Computing and Applications, vol. 16, no. 2, pp. 173–186, 2007.

[0235]

[21] H. Hu,

[0236]

[22] A. Laterre, Y. Fu, MK Jabri, A.-S. Cohen, D. Kas, K. Hajjar, TSDahl, A. Kerkeni, and K. Beguir, “Ranked reward: Enabling self-play reinforcement learning for combinatorial optimization”, arXiv preprint arXiv:1807.01672, 2018.

[0237]

[23] O. Kundu, S. Dutta, and S. Kumar, “Deep-pack: A vision-based 2d online binpacking algorithm with deep reinforcement learning,” in 2019 28th IEEE International Conference on Robot and Human Interactive Communication (RO-MAN), IEEE, 2019, pp. 1–7.

[0238]

[24] H. Zhao, Q. She, C. Zhu, Y. Yang, and K. Xu, “Online 3d bin packing with constrained deep reinforcement learning,” in Proceedings of the AAAIConference on Artificial Intelligence, vol. 35, no. 1, 2021, pp. 741–749.

[0239]

[25] R. Hu, J. Xu, B. Chen, M. Gong, H. Zhang, and H. Huang, “Tap-net: transport-and-pack using reinforcement learning,” ACM Transactions on Graphics (TOG), vol. 39, no. 6, pp. 1–15, 2020.

[0240]

[26] R. Verma, A. Singhal, H. Khadilkar, A. Basumatary, S. Nayak, HVSingh, S. Kumar, and R. Sinha, “A generalized reinforcement learning algorithm for online3d bin-packing”, arXiv preprint arXiv:2007.00463, 2020.

[0241]

[27] Y. Wu, W. Li, M. Goh, and R. De Souza, “Three-dimensional bin packing problem with variable bin height,” European journal of operational research, vol. 202, no. 2, pp. 347–355, 2010.

[0242]

[28] J. Cagan, K. Shimada, and S. Yin, “A survey of computational approaches to three-dimensional layout problems,” Computer-Aided Design, vol. 34, no. 8, pp. 597–611, 2002.

[0243]

[29] J. Michalek, R. Choudhary and P. Papalambros, “Architectural layout design optimization”, Engineering Optimization, Vol. 34, No. 5, pp. 461–484, 2002.

[0244]

[30] J.F. Oliveira, A. Neuenfeldt Jünior, E. Silva and M.A. Carravilla, “A survey on heuristics for the two-dimensional rectangular strip packing problem”, Pesquisa Operacional, vol. 36, pp. 197–226, 2016.

[0245]

[31] BS Baker, EG Coffman, Jr, and RL Rivest, “Orthogonal packings in two dimensions,” SSIAM Journal on computing, vol. 9, no. 4, pp. 846–855, 1980.

[0246]

[32] D. Zhang, L. Shi, S. C. Leung, and T. Wu, “A priority heuristic for the guillotine rectangular packing problem,” Information Processing Letters, vol. 116, no. 1, pp. 15–21, 2016.

[0247]

[33] H. Zhao, Q. She, C. Zhu, Y. Yang, and K. Xu, “Online 3d bin packing with constrained deep reinforcement learning”, arXiv preprint arXiv:2006.14978, 2020.

[0248]

[34] B. Guo, J. Hu, F. Wu, and Q. Peng, “Automatic layout of 2d free-form shapes based on geometric similarity feature searching and fuzzy matching,” Journal of Manufacturing Systems, vol. 56, pp. 37–49, 2020.

[0249]

[35] Y. Yang, B. Liu, H. Li, X. Li, G. Wang and S. Li, "A nesting optimization method based on digital contour similarity matching for additive manufacturing", Journal of Intelligent Manufacturing, pp. 1 to 23, 2022.

[0250]

[36] J. Zewicz, P. Hawryluk, and R. Walkowiak, “Using a tabu search approach for solving the two-dimensional irregular cutting problem,” Annals of Operations Research, vol. 41, no. 4, pp. 313–325, 1993.

[0251]

[37] JF Oliveira, AMGomes and JS Ferreira, “Topos — a new construc-tive algorithm for nesting problems”, OR-Spektrum, vol. 22, no. 2, pp. 263–284, 2000.

[0252]

[38] AK Sato, Td C Martins, and Md S G Tsuzuki, “A pairwise exact placement algorithm for the irregular nesting problem,” International Journal of Computer Integrated Manufacturing, vol. 29, no. 11, pp. 1177–1189, 2016.

[0253]

[39] J.Liu, GPOng, and X.Chen, "Graphsage-based traffic speed fore-casting for segment network with sparse data", IEEE Transactions on IntelligentTransportation Systems, 2020.

[0254]

[40] B. Perozzi, R. Al-Rfou, and S. Skiena, “Deepwalk: Online learning of social representations,” in Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining, 2014, pp. 701–710.

[0255]

[41] J. Kim, T. Kim, S. Kim, and C.D. Yoo, “Edge-labeling graph neural network for few-shot learning,” in Proceedings of the IEEE / CVF con-ference on computer vision and pattern recognition, 2019, pp. 11–20.

[0256]

[42] TNKipf and M. Welling, "Semi-supervised classification with graphconvolutional networks", arXiv preprint arXiv:1609.02907, 2016.

[0257]

[43] G. Laporte, “The vehicle routing problem: An overview of exact and approximate algorithms”, European journal of operational research, vol. 59, no. 3, pp. 345–358, 1992.

[0258]

[44] J.-F. Cordeau, M. Gendreau, A. Hertz, G. Laporte, and J.-S. Sormany, “New heuristics for the vehicle routing problem,” Logistics systems: design and optimization, pp. 279–297, 2005.

[0259]

[45] I. Grondman, L. Busoniu, G. A. Lopes, and R. Babuska, “A survey of actor-critic reinforcement learning: Standard and natural policy gradients,” IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews), vol. 42, no. 6, pp. 1291–1307, 2012.

[0260]

[46] C. Niedzwiedz, I. Elhanany, Z. Liu, and S. Livingston, “A consolidated actor-critic model with function approximation for high-dimensional pomdps,” in AAAI 2008 workshop for advancement in POMDP solvers, 2008, pp. 37–42.

[0261]

[47] X.-S. Wang, Y.-H. Cheng, and J.-Q. Yi, “A fuzzy actor–critic reinforcement learning network,” Information Sciences, vol. 177, no. 18, pp. 3764–3781, 2007.

[0262]

[48] ​​Y.-H. Cheng, J.-Q. Yi, and D.-B. Zhao, “Application of actor-critic learning to adaptive state space construction,” in Proceedings of 2004 International Conference on Machine Learning and Cybernetics (IEEE Cat. No. 04EX826), vol. 5. IEEE, 2004, pp. 2985–2990.

[0263]

[49] ICPaschalidis, K.Li and RMEstanjini, "An actor-critic method using least squares temporal difference learning", in Proceedings of the 48h IEEEconference on decision and control (CDC) held jointly with 2009 28th ChineseControl Conference, IEEE, 2009, pp. 2564 to 2569.

[0264]

[50] H.R. Berenji and D. Vengerov, “A convergent actor-critic-based frlalgo-rithm with application to power management of wireless transmitters,” IEEE Transactions on Fuzzy Systems, vol. 11, no. 4, pp. 478–485, 2003.

[0265]

[51] V. R. Konda and J. N. Tsitsiklis, “Onactor-critic algorithms,” SIAM journal on Control and Optimization, vol. 42, no. 4, pp. 1143–1166, 2003.

[0266]

[52] H. Kimura, “Natural gradient actor-critic algorithms using random rectangular coarse coding”, in 2008 SICE Annual Conference, IEEE, 2008, pp. 2027–2034.

[0267]

[53] S. Girgin and P. Preux, “Basis expansion in natural actor critic methods”, in European Workshop on Reinforcement Learning, Springer, 2008, pp. 110–123.

[0268]

[54] J. Park, J. Kim, and D. Kang, “An RLs-based natural actor-critic algorithm for locomotion of a two-linked robot arm,” in International Conference on Computational and Information Science, Springer, 2005, pp. 65–72.

[0269]

[55] S. Bhatnagar, R. S. Sutton, M. Ghavamzadeh, and M. Lee, "Natural gradient actor-critic algorithms", Automatica, 2007.

[0270]

[56] T. Morimura, E. Uchibe, J. Yoshimoto and K. Doya, "A generalized naturalactor-critic algorithm", Advances in neural information processing systems, Volume 22, 2009.

[0271]

[57] C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, and A. Rabinovich, “Going deeper with convolutions,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2015, pp. 1–9.

[0272]

[58] B. Widrow, N. K. Gupta, and S. Maitra, “Punish / reward: Learning with acritic in adaptive threshold systems,” IEEE Transactions on Systems, Man, and Cybernetics, 5, 455–465, 1973.

[0273]

[59] AG Barto, RS Sutton, and CW Anderson, “Neuronlike adaptive elements that can solve difficult learning control problems,” IEEE transactions on systems, man, and cybernetics, 5, 834–846, 1983.

[0274]

[60] M. Nazari, A. Oroojlooy, L. Snyder, and M. Takáč, “Reinforcement learning for solving the vehicle routing problem”, Advances in neural information processing systems, vol. 31, 2018.

Claims

1. A method for selecting parts to be placed on a sheet using a computer, the method comprising the steps of: I. Encoding the geometric features of each part in a corresponding geometric information vector, wherein, for each part, one of the geometric features is the projected area of ​​the part in the plane view; II. generating a graph in which the geometrical features of each part are assigned once to one node of the graph, in which all nodes are connected pairwise by one edge at a time, The method comprises the following additional steps in the inference phase: XIV. Estimating parameters GCI for all pairs of parts, where each GCI is a measure of how well the pair of parts can be placed on the same sheet; XV. assigning each GCI to a corresponding edge of the graph, the corresponding edge passing through two nodes representing the parts to which the corresponding GCI is assigned; XVI. Determining weights of the edges of the graph, wherein each edge weight depends on the GCI associated with the edge; XVII. Allocate the parts to the corresponding sheets by determining a subgraph of the graph passing through the nodes with the aid of an optimization method, wherein the nodes passed through by the corresponding subgraph represent the parts to be placed on the sheet, and wherein the sum of the projected areas of the parts on each subgraph is at most equal to the corresponding sheet size.

2. The method according to claim 1, wherein: The optimization method in step XVII is performed using a quantum computer.

3. A method according to any one of the preceding claims, wherein: One or more steps of the method are performed by a first neural network, preferably a graph neural network, particularly preferably a graph convolutional neural network.

4. A method according to any one of the preceding claims, wherein: Step XIV, step XV and / or step XVI are performed by a class actor module, wherein the class actor module preferably comprises the first neural network.

5. A method according to any one of the preceding claims, wherein: The allocation of the parts to the respective sheets according to step XVII is performed with the aid of an optimization method for solving the capacity-constrained vehicle routing problem CVRP.

6. A method according to any one of the preceding claims, wherein: The sheets have the same sheet size.

7. A method according to any one of the preceding claims, wherein: Determining the GCI for all pairs of the parts in a training phase performed before the inference phase comprises the following steps: III. Setting an estimated value of the GCI, wherein each estimated value of the GCI is assigned to a pair of the parts; IV. providing first edge weights to the edges of the graph, wherein each first edge weight depends on a corresponding GCI assigned to the edge; V. assigning the parts to the corresponding sheets according to step XVII by determining a subgraph of the graph passing through the node; VI. determining a reward function value for each sub-graph as a measure of how much surface area of ​​the corresponding sheet is covered by the part on each sub-graph after the part is arranged on the corresponding sheet; VII. determining a fitness value for each sub-graph in the form of an average of output values ​​of the corresponding sub-graph, wherein the output values ​​take into account the estimated value of the GCI, wherein the average is determined by taking into account all the output values ​​of the corresponding sub-graph; VIII. Re-determine the GCI for each subgraph so that the reward loss is reduced on each subgraph, wherein the reward loss depends on the reward function value and the fitness value; IX. Repeat steps VI to VIII until the reward loss meets the first termination criterion; X. Estimating a second edge weight of the edge of the graph, in particular estimating the second edge weight through steps XIV to XVI; XI. determining an edge weight loss at the edge of the graph, wherein the edge weight loss depends on the second edge weight and the GCI at the corresponding edge determined in step VIII; XII. Repeating step X and step XI, wherein the second edge weight is estimated so that the edge weight loss is minimized until the edge weight loss satisfies a second termination criterion; XIII. Repeat step III to step XII, wherein the estimated value of the GCI in step III is set to the second edge weight finally determined in step XII, until the estimated value of the GCI meets a third termination criterion.

8. The method according to claim 4 and claim 7, wherein: Step X, step XI and / or step XII are performed by the class actor module.

9. The method according to claim 7 or 8, wherein: Step III, step IV, step VII, step VIII and / or step IX are performed by a class critic module, wherein the class critic module in particular comprises a second neural network, wherein the class critic module preferably comprises a third neural network in addition to the second neural network, wherein the third neural network receives result data of the second neural network.

10. The method according to any one of claims 7 to 9, wherein: The re-determination of the GCI according to step VIII is performed using error back-propagation to minimize the reward loss of the corresponding subgraph.

11. The method according to any one of claims 7 to 10, wherein: The first termination criterion according to step IX is given by the reward loss being less than a first default value, and / or the second termination criterion according to step XII is given by the edge weight loss at the edge of the graph being less than a second default value, and / or the third termination criterion according to step XIII is given by the estimated value change of the GCI after undergoing steps III to XII being less than a third default value.

12. The method according to claim 1, wherein: In step XIV, the GCI is determined using a metaheuristic approach.

13. A method according to any one of the preceding claims, wherein: The parts are arranged on the sheet and subsequently produced.

Images