Three-dimensional thin-wall part layout algorithm for single-side infinite extension raw material plate

Abstract

The invention discloses a reinforcement learning-based layout algorithm for three-dimensional thin-walled parts of a unilateral infinite extension raw material plate. The method comprises the following steps: generating a two-dimensional expansion polygon by adopting a three-dimensional projection algorithm, a minimum convex hull algorithm and a Minkowski vector sum algorithm; polygon coding is a generalized graph, a selection graph is used as an action, current layout coding is a generalized graph used as a state, a placement area and a compact degree are used as action return, deep reinforcement learning iteration is used to obtain an action sequence, and finally, a placement sequence, an angle and a position of each polygon are obtained. According to the invention, the layout problem of the thin-wall three-dimensional characteristic part in machining can be solved; reinforcement learning fully explores an unknown state in a placement process, fully considers the influence of a part arrangement sequence on a final result, and can quickly give a better layout mode; in addition, the generalization ability is high by using deep learning to fit the reinforcement learning value function, and a better layout result can be obtained without retraining network parameters for a brand new layout problem.

Description

Technical Field

[0001] The present invention relates to the two-dimensional blanking problem and optimization field of a single-sided infinitely long raw material plate, and specifically to a method for obtaining a two-dimensional point cloud set of a three-dimensional part by using a projection algorithm, and obtaining a placement order, placement angle and placement position of the two-dimensional part by using iterative learning of reinforcement learning. Background Art

[0002] In the field of modern manufacturing, the three-dimensional sheet metal layout problem is a key engineering optimization task, which involves how to efficiently arrange and cut parts of the required shape with limited sheet metal resources. The complexity of this problem comes from the three-dimensional nature of the sheet metal and the need to consider many constraints during the manufacturing process, such as minimizing raw material waste, reducing production costs, and improving production efficiency. Therefore, solving the three-dimensional sheet metal layout problem has important theoretical and practical significance for promoting the digital transformation of the manufacturing industry and improving resource utilization efficiency.

[0003] Traditional sheet metal layout problems are mainly concentrated on one-dimensional linear layout and two-dimensional plane layout. However, with the advancement of manufacturing technology and the complexity of product structure, the demand for three-dimensional sheet metal is increasing. In the fields of aerospace, automobile manufacturing, construction, etc., the shapes and structures of many parts have complex three-dimensional features, so the traditional two-dimensional layout method can no longer meet the needs of actual production.

[0004] Among them, the patents related to the layout algorithm of irregular three-dimensional thin plate parts include: a knowledge-driven two-dimensional polygonal automatic layout method invented by Feng Yixiong et al. This method traverses all angles of all polygons, calculates the angle and position corresponding to the maximum fitness of each part, and arranges them. This method does not consider the influence of the order of polygon placement on the layout results in the actual layout process, and the calculation amount is huge and difficult to use in medium and large-scale projects. In addition, this method only considers two-dimensional layout, which is difficult to use in three-dimensional part layout projects; a two-dimensional rectangular part layout method based on an improved simulated annealing algorithm invented by Bo Leilei et al. This method can reduce the number of plates used, improve the utilization rate of plates, greatly reduce the layout plan, and reduce the material cost and labor cost of the product, but this method has a large amount of calculation and is difficult to converge slowly, and the generalization ability is poor. In addition, the existing patents have given little consideration to the layout problem of two-dimensional square single-sided infinite plates, and there is no algorithm that minimizes the square area occupied by the final layout after the polygons are placed. Summary of the invention

[0005] In response to the above problems, this patent proposes an irregular three-dimensional thin plate parts layout algorithm based on reinforcement learning. Through the projection algorithm and the two-dimensional convex hull algorithm, the three-dimensional parts are compressed and projected into two-dimensional polygons in a certain direction. The current layout is encoded as the reinforcement learning state, and the sub-polygon set after all polygons are rotated at a fixed angle is used as the action. The reinforcement learning model is constructed and iteratively optimized to obtain the layout result.

[0006] This patent invents a three-dimensional thin-walled parts layout algorithm for a single-sided infinitely extended raw material plate, which includes the following steps:

[0007] S1. Project the 3D part onto the 2D plane using the 3D projection algorithm. In actual production, the 3D thin-walled parts are usually cut to obtain the blank parts; use the minimum convex hull algorithm to calculate the minimum enveloping polygon of the 2D plane projection point set; use the polygon expansion algorithm to calculate the expansion polygon of the minimum enveloping polygon, leaving a machining allowance for subsequent finishing in actual production;

[0008] The projection algorithm three-dimensional part formats in S1 include point cloud format and set polygon vertex format. For the set polygon vertex format part, step S1 can use the PCA dimensionality reduction projection method. The PCA algorithm can ensure that the variance of the projection figure point set is the largest, thereby ensuring that the contour of the thin-walled part after dimensionality reduction projection is the main shape contour. The PCA algorithm specifically includes the following steps:

[0009] Step 1, read the polygon vertex set {p(x,y,z)} from the three-dimensional part;

[0010] Step 2: Calculate the center point coordinates

[0011] Step 3: Centralize the vertex set data set {p(x,y,z)} to get P center = {p(x,y,z)-p mean};

[0012] Step 4: Calculate the covariance matrix

[0013] Step 5: Calculate the covariance matrix eigenvalues ​​[λ1,λ2,λ3] and eigenvectors [v1,v2,v3], sort the eigenvalues, select the eigenvectors corresponding to the two largest eigenvalues ​​as the projection basis, and finally calculate the reduced dimensionality two-dimensional point set P project = {p′(P center,i ·v max1 ,P center,i ·v max2 )};

[0014] The processing method of point cloud format parts is the same as that of polygon vertex format, but the amount of calculation is relatively larger. In actual production, three-dimensional thin-walled parts are usually obtained by cutting. Therefore, the beneficial effect of using a three-dimensional projection algorithm is that it can effectively obtain the main structure contour of three-dimensional thin-walled parts, convert three-dimensional nesting problems into two-dimensional nesting problems, eliminate interference and reduce the overall calculation amount of the algorithm.

[0015] The minimum convex hull algorithm calculation in S1 can adopt Bykat algorithm, Akl-Toussaint algorithm, Jarvis matching algorithm and Graham-Andrew scanning algorithm. Among them, Graham-Andrew scanning algorithm is simple and easy to implement. Graham-Andrew scanning algorithm is preferably adopted. The implementation steps are as follows:

[0016] Step 1: traverse the input polygon vertex set P project Find the point P with the smallest ordinate project,miny As the starting point of the convex hull, that is, P ConvexHull [0] = P project,miny ;

[0017] Step 2: project All other points in P project,miny Sort polar angles for benchmarks;

[0018] Step 3, from P ConvexHull [0] Start scanning all points and use vector cross product to determine point P project [t] Whether it is inside the convex hull, if so, it is a convex hull point and is added to the convex hull point set, otherwise traverse to the next point;

[0019]

[0020] The beneficial effect of calculating convex polygons is that the number of contour polygon points can be further reduced, which greatly reduces the amount of calculation of subsequent nesting algorithms.

[0021] The polygon expansion algorithm in S1 can use the method of finding the Minkowski sum of the polygon and the fixed radius circle. The calculation can be done by using the convolution-based Minkowski vector and calculation, the specific method is as follows:

[0022] Step 1: Sort all vertices of polygons M and N to obtain a counterclockwise vertex sequence M = {m i}、N={n i};

[0023] Step 2, calculate the simplified convolution K of M and N, that is and A set of line segments, where m iWith n j is a convex vertex;

[0024] Step 3, extract non-overlapping, orientable loops L from K (if the normals of all line segments are consistently facing inward or outward, the loop L is considered orientable), and analyze their nested relationships for further filtering;

[0025] Step 4, check the intersections between the input polygons placed in the loop layout to filter out the boundaries, and the final result is the Minkowski vector sum;

[0026] Convert the circle to a polygon and calculate its convex hull {P ConvexHull} of the Minkowski sum to obtain the expanded polygon set {P offset}.

[0027] In actual production, it is often necessary to leave enough machining allowance for cutting blanks. Therefore, calculating the expansion polygon of the convex polygon is to fully consider the subsequent machining allowance and set the expansion circle radius according to the allowance size.

[0028] S2. Encode reinforcement learning actions and states, design state transition functions based on actions and states, and set reinforcement learning reward function R t (a t |s t ), build a deep reinforcement learning model: design two identical neural networks based on the reinforcement learning state: the value network Q(a t ,s t ) and the target network T(a t ,s t ), and adopt the ε-decay strategy to explore the interaction between the agent and the environment. t The agent chooses action a according to the strategy t After that, the layout is updated to get the new state s t+1 , and according to action a t and status t Calculate the reward R brought by this action t (a t |s t ), the state s t 、Action a t , return R t (a t |s t ) and the new state s t+1 Put it into the experience pool, and calculate the prediction score when the data in the experience pool reaches batch1 and the actual score U t :

[0029]

[0030] Ut =∑ i≤t γ i R(a i |s i )=R t (a t |s t )+γmax({T(a t ,s t+1 )})

[0031] , and train the updated value network Q(a t ,s t ) parameter epoch1, when the total number of rounds reaches epoch2, Q(a t ,s t ) parameter is assigned to T(a t ,s t ) Update the target network parameters, output the action sequence when the number of rounds reaches the total number of rounds epoch, and finally generate the placement angle, placement order and placement position of each polygon according to the action sequence;

[0032] The beneficial effects of using enhanced deep learning to solve the pattern-sorting problem are: it can fully explore the unknown areas in the search space, and the strategy function formulated using deep learning has good generalization ability. In theory, it can also solve new pattern-sorting problems after training is completed.

[0033] Among them, the encoding reinforcement learning action and state in S2, one encoding method is as follows:

[0034] Reinforce learning action encoding, extract the expansion polygon {P offset} and rotate 360° to generate the expanded polygon sub-polygon set {P i (α)}, each action selects the expanded polygon sub-polygon set {P i (α)}, that is, a t ∈{P i (α)}, the action and dilation subpolygon are represented as the following mapping:

[0035] a t (i,k)=P(i,kθ)

[0036] Where i and n are integers, and P(i,kθ) is the set of expanded polygon sub-polygons {P i The sub-polygon generated by rotating the i-th polygon in (α)} by an angle α=kθ;

[0037] The polygon corresponding to the action is regarded as an image I with d×d pixels. a (d cannot be so small that the two vertices of the action image collapse into one pixel), the pixel value Ia (c, r) Whether there is a polygon vertex in the corresponding pixel range, if I a (c,r)=1 indicates that the pixel is a polygon vertex, and the feature matrix is ​​designed and set As the final encoded action, the first feature indicates whether the pixel value is a vertex, and the second and third features are the exact values ​​of the vertex, namely:

[0038]

[0039] p j For polygon vertices, the final action is encoded as: a t =H a ;

[0040] Reinforcement learning state encoding, taking the current layout in the raw material board as the reinforcement learning state, and treating the layout as an image I with pixels w×h to further extract features s (Image I s with I a The actual distance represented by each pixel is the same, and each time only the rightmost vertex of the actual layout is taken to the left of a certain area to draw as image I s ), pixel value I s (c,r) represents whether there is a polygon vertex within the pixel range, setting the feature matrix Further description of the status characteristics:

[0041] Feature Matrix Use the number of polygon vertices in the current pixel I s (c, r), the exact x-coordinate and the exact y-coordinate of the vertex describe the state of the polygon vertex. The specific calculation method is as follows:

[0042]

[0043] p j is the polygon vertex in the raw material board layout, and finally the state encoding feature matrix, that is, s t =H s .

[0044] The beneficial effects of using a generalized image approach to encode actions and states are: it can unify all two-dimensional packing problems, and does not lose excessive information features due to the limited number of pixels in the image, which to a certain extent ensures that the strategy learned by the final reinforcement learning agent is applicable to the new packing problem encountered.

[0045] The reward function R in S2 t (a t |s t ), including R S,t R C,t and R SC,tThese three parts include:

[0046] R S,t is the current placement polygon area, which is used to evaluate the size of the polygon area after each action. Subjectively, we prefer to place the largest polygon possible first. S,t The calculation method is as follows:

[0047]

[0048] where p i,t are the two-dimensional coordinate vertices in the convex hull.

[0049] R C,t Used to evaluate the compactness of placement. This score is composed of the Minkowski vector sum of the raw material board and the expansion polygon of the current polygon, and the Minkowski vector sum of the current layout area and the expansion polygon of the current polygon. The specific calculation formula is as follows:

[0050]

[0051] R SC,t In the design of reward functions for tasks and planning goals, this patent considers infinite length of a single side. If the y direction of the raw material plate is fixed, the x direction can be infinitely extended. It is hoped that the distance between the rightmost vertex of the polygon and the origin in the final layout result is as short as possible. Based on this optimization goal, R SC,t It can be designed as the inverse of the product of the length added in the x direction after the current polygon is placed and the board width d. The formula is as follows:

[0052] R SC,t =-d×(max(P t (x))-max({P i,m (x)}))

[0053] Final reward function R t (a t |s t ) is calculated as follows:

[0054] R t (a t |s t )=w1R S,t +w2R C,t +w3R SC,t

[0055] w1, w2, and w3 are the weights of the three scores.

[0056] The value network Q(a t ,s t ) and the target network T(a t ,st ) neural network, exactly the same, the design can use the convolutional neural network CNN model, the input is the current state s t and action a t , the output is the prediction score and the actual score U t , these two networks are exactly the same, first use action graph a t For state diagram s t A convolution and pooling operation is performed, and the subsequent layer_n convolutional layers plus pooling layers and full_connected_n fully connected layers form the final network. This neural network design method fully considers the characteristics of action graphs and state graphs.

[0057] In the design of the neural network, the first layer uses the action graph to perform a convolution on the state graph, fully considering the relationship between the current layout and the polygons. The subsequent convolutional layers fully extract the features of the image and ultimately improve the total prediction score of the fully connected layer output.

[0058] Further, the three-dimensional thin-walled parts layout algorithm of the raw material plate is infinitely extended on one side, and the specific process of the "layout update" in step S2 is as follows:

[0059] Step 1, decoding action a t , obtain the current placement polygon P(i,k), use Minkowski and calculate the current polygon to be placed P(i,k) and the raw polygon P m The critical inscribed polygon IFP i , and calculate the circumscribed critical polygon {NFP i,j}.

[0060] Step 2: Use the inscribed critical polygon IFP i For all external critical {NFP i,j} Perform a subtraction operation to obtain the collision-free region CFR of the current polygon under the current layout i , the formula is as follows.

[0061]

[0062] Step 3: Use a heuristic placement strategy to complete the placement of the current polygon.

[0063] The heuristic placement strategy may be a bottom-left heuristic method BL and a lowest horizontal line placement strategy.

[0064] S3. Place the raw material board at the coordinate origin, transform the coordinates of each three-dimensional part in the projection direction to obtain the transformed part 1, and then transform the transformed part 1 according to the polygon placement angle and position calculated in S2 to obtain the three-dimensional part finally placed in the raw material board. Finally, place all the parts in a raw material board coordinate system and output the three-dimensional assembly after the final layout is completed.

[0065] The beneficial effects of the present invention are:

[0066] 1. The algorithm flow proposed in this patent is suitable for thin-walled three-dimensional feature parts in mechanical processing. The thin-walled three-dimensional feature parts are converted into two-dimensional polygons using the three-dimensional projection algorithm and the minimum convex hull algorithm.

[0067] 2. The algorithm proposed in this patent can solve the problem of arranging square raw material plates with infinite one side, and achieve the minimum square area occupied by the final layout.

[0068] 3. Fully consider the impact of different parts placement orders on the final layout results, making it easier to get the optimal solution. Utilizing the powerful learning ability of the reinforcement learning trial and error process, the unknown state in the placement process is fully explored, and it is not easy to fall into the local minimum point, which further improves the utilization rate of the raw material board.

[0069] 4. Strong generalization ability. Using deep learning network to learn the action value function that predicts the current state and the selected action can converge faster than traditional reinforcement learning. In theory, after completing the training with enough training data, it can give good sorting results for new problems even without training. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] Figure 1 The figure is a schematic diagram of the overall process of the algorithm of the present invention.

[0071] Figure 2 Generate a flow chart for 3D parts to 2D expanded polygons.

[0072] Figure 3 It is the Minkowski sum of the polygon and the circle, generating a schematic diagram of the expanded polygon.

[0073] Figure 4 It is the state transition diagram of reinforcement learning.

[0074] Figure 5 Arrange the reinforcement learning flowchart for polygons.

[0075] Figure 6 Score R for the compactness of polygon placement C,t Schematic diagram.

[0076] Figure 7 For R SC,t Scoring diagram.

[0077] Figure 8 Schematic diagram of the Q network and T network structure. DETAILED DESCRIPTION

[0078] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the accompanying drawings of the present invention.

[0079] The overall process of this method is as follows Figure 1 As shown, it includes the following three steps: generating an inflated polygon, obtaining data from the interaction between the agent and the environment in reinforcement learning to update the neural network model, and finally obtaining the placement angle sequence and placement position of the polygon and the three-dimensional parts layout. Specifically, it includes the following steps:

[0080] S1. Project the 3D part onto the 2D plane using the 3D projection algorithm, calculate the minimum enveloping polygon of the 2D plane projection point set using the minimum convex hull algorithm, and calculate the expanded polygon of the minimum enveloping polygon using the polygon expansion algorithm. The specific flow chart is as follows: Figure 2 ;

[0081] For three-dimensional thin-walled parts, it is difficult for the machine to determine the correct projection direction through data. In order to enable the machine to find the correct direction of thin-wall projection, the projection direction is defined as the direction with the largest variance of the projection points after the projection of the three-dimensional part vertices, so PCA can be used for dimensionality reduction projection. The three-dimensional part format of the projection algorithm in S1 includes point cloud format and set polygon vertex format. For the set polygon vertex format part, step S1 adopts the PCA dimensionality reduction projection method, which specifically includes the following steps:

[0082] Step 1, read the polygon vertex set {p(x,y,z)} from the three-dimensional part;

[0083] Step 2: Calculate the center point coordinates

[0084] Step 3: Centralize the vertex set data set {p(x,y,z)} to get P center = {p(x,y,z)-p mean};

[0085] Step 4: Calculate the covariance matrix

[0086] Step 5: Calculate the covariance matrix eigenvalues ​​[λ1,λ2,λ3] and eigenvectors [v1,v2,v3], sort the eigenvalues, select the eigenvectors corresponding to the two largest eigenvalues ​​as the projection basis, and finally calculate the reduced dimensionality two-dimensional point set P project = {p′(P center,i ·v max1 ,P center,i ·v max2)};

[0087] The point cloud format parts processing method is the same as the polygon vertex format, but the amount of calculation is relatively larger.

[0088] In order to obtain the contour of a two-dimensional point set and reduce the amount of subsequent calculations, it is necessary to first calculate the minimum convex hull of the point set. The minimum convex hull algorithm in S1 can be calculated using the Bykat algorithm, Akl-Toussaint algorithm, Jarvis matching algorithm, and Graham-Andrew scanning algorithm. The Graham-Andrew scanning algorithm is simple and easy to implement. The implementation steps are as follows:

[0089] Step 1: traverse the input polygon vertex set P project Find the point P with the smallest ordinate project,miny As the starting point of the convex hull, that is, P ConvexHull [0] = P project,miny ;

[0090] Step 2: project All other points in P project,miny Sort polar angles for benchmarks;

[0091] Step 3, from P ConvexHull [0] Start scanning all points and use vector cross product to determine point P project [t] Whether it is inside the convex hull, if so, it is a convex hull point and is added to the convex hull point set, otherwise traverse to the next point;

[0092]

[0093] In actual production, the blank parts cut out of thin walls often need to leave enough margin for subsequent part processing. This process can be imagined as a new shape obtained after the center of a circle with a fixed radius circles around the polygonal contour, such as Figure 3 As shown in the figure, this process can be regarded as the Minkowski sum of a polygon and a fixed radius circle, so the polygon expansion algorithm in S1 can use the method of finding the Minkowski sum of a polygon and a fixed radius circle. The calculation can be done by using the convolution-based Minkowski vector and calculation, the specific method is as follows:

[0094] Step 1: Sort all vertices of polygons M and N to obtain a counterclockwise vertex sequence M = {m i}、N={n i};

[0095] Step 2, calculate the simplified convolution K of M and N, that is and A set of line segments, where m i With n j is a convex vertex;

[0096] Step 3, extract non-overlapping, orientable loops L from K (if the normals of all line segments are consistently facing inward or outward, the loop L is considered orientable), and analyze their nested relationships for further filtering;

[0097] Step 4, check the intersections between the input polygons placed in the loop layout to filter out the boundaries, and the final result is the Minkowski vector sum;

[0098] Convert the circle to a polygon and calculate its convex hull {P ConvexHull} of the Minkowski sum to obtain the expanded polygon set {P offset}.

[0099] S2. Encode reinforcement learning actions and states, design state transition functions based on actions and states, and set reinforcement learning reward function R t (a t |s t ), build a deep reinforcement learning model: design two identical neural networks based on the reinforcement learning state: the value network Q(a t ,s t ) and the target network T(a t ,s t ), and uses the ε-decay strategy to explore the interaction between the agent and the environment. Using enhanced deep learning to solve the row-laying problem can fully explore the unknown areas in the search space, and the deep learning method to formulate the strategy function has good generalization ability. In theory, it can also solve new row-laying problems after training.

[0100] At the beginning of the training round, set the total number of reinforcement learning training rounds epoch = 10000, set the current round number epoch_now = 1, initialize the number of interactions between the current agent and the environment n = 1, initialize the environment to the raw material plate P0 without any part polygons placed, and initialize the greedy exploration coefficient ε t =1, the decreasing rate of greedy exploration coefficient during training is γ ε =0.9975, greedy exploration coefficient minimum value ε min =0.05. Set the value network Q(a t ,s t ) The learning rate lr = 0.005, set the cumulative reward function discount rate of reinforcement learning γ = 0.99, set the initialization experience pool pool = {}, and set the maximum amount of data in the experience pool to batch max =20000, set the value network Q update sampling number batch=512, the number of training times epoch1=100, and set the target network T update frequency epoch2=50.

[0101] According to the reinforcement learning value network Q(a t ,s t ), input current state s t and action a t , output action score Q(a t |s t ), according to the ε-greedy strategy, generate a random number Randnum between 0 and 1. If Randnum ≤ ε t Then a random action is taken, otherwise the action with the highest score is taken, i.e. a t = argmax({Q(s t , a t )}), in state s t The agent chooses action a according to the strategy t After that, update the layout state to enter the new state s t+1 , and determine whether the new state is a terminal state. If it is a terminal state, initialize the number of interactions between the agent and the environment and update the current round number, that is, n = 1, epoch_now = epoch_now + 1, and then according to action a t and status t Calculate the reward R brought by this action t (a t |s t ), the state s t 、Action a t , new state s t+1 and return R t (a t |s t ) into the experience pool, update the number of interactions between the current agent and the environment, the greedy exploration coefficient and the state, n = n + 1, ε = max (γ ε ε,ε min ),s t =s t+1 .

[0102] When the amount of data in the experience pool reaches the batch, randomly take out the batch group data in the experience pool and calculate the prediction score and the actual score U t :

[0103]

[0104] U t =∑ i≤t γ i R(a i |s i )=R t (a t |s t )+γmax({T(at ,s t )})

[0105] Obtain multiple sets of states, predicted scores, and actual scores, and use the gradient descent method to update the value network Q(a t ,s t ) parameter epoch1 times, when the current iteration number epoch_now can divide T network update frequency epoch2, Q(a t ,s t ) parameter is assigned to T(a t ,s t ) Update the target network parameters, and output the action sequence when the number of iterations epoch_now reaches epoch = 10000. Finally, the placement angle, placement order and placement position of each polygon are generated according to the action sequence. The state transition of reinforcement learning and the overall learning process are as follows: Figure 4 , Figure 5 As shown;

[0106] Using multiple sets of data from the experience pool to train the model can avoid the problem of convergence caused by random errors. Alternating between updating the value network and the target network fixes the parameters of a single model and greatly improves the convergence speed.

[0107] Among them, the encoding reinforcement learning action and state in S2, one encoding method is as follows:

[0108] Reinforce learning action encoding, extract the expansion polygon {P offset} and rotate 360° at an interval of θ = 10° to generate the expanded polygon sub-polygon set {P i (α)}, each action selects the expanded polygon sub-polygon set {P i (α)}, that is, a t ∈{P i (α)}, the action and dilation subpolygon are represented as the following mapping:

[0109] a t (i,k)=P(i,kθ)

[0110] Where i and n are integers, and P(i,kθ) is the set of expanded polygon sub-polygons {P i The sub-polygon generated by rotating the i-th polygon in (α)} by an angle α=kθ can include all polygons that may appear in the placement process by encoding the reinforcement learning actions in this way.

[0111] The polygon corresponding to the action is regarded as an image I with d×d pixels (where d cannot collapse the vertex of the minimum polygon into a single pixel, d=50) a , pixel value I a(c, r) Whether there is a polygon vertex in the corresponding pixel range, if I a (c,r)=1 indicates that the pixel is a polygon vertex, and the feature matrix is ​​designed and set As the final encoded action, the first feature indicates whether the pixel value is a vertex, and the second and third features are the exact values ​​of the vertex, namely:

[0112]

[0113] p j For polygon vertices, the final action is encoded as: a t =H a ;

[0114] Reinforcement learning state encoding, taking the current layout in the raw material board as the reinforcement learning state, and treating the layout as an image I with pixels w×h to further extract features s , if the height of the raw material board is h real =500mm, the width is infinite, then the image I is designed according to the action pixel s The specific calculation formula is:

[0115] h=h real / k a

[0116] where k a is the ratio of the actual distance to pixels of the action coding image.

[0117] Since the width of the raw material board is infinite, each time an image I is generated s When , only the rightmost vertex of the current layout is taken to the left of d real =500mm area, so image I s The height is w = w real / k a , if the motion picture coding has been designed and k a =1, then image I s The width and height w=h=500.

[0118] Pixel value I s (c,r) represents whether there is a polygon vertex within the pixel range, setting the feature matrix Further description of the status characteristics:

[0119] Feature Matrix Use the number of polygon vertices in the current pixel I s (c, r), the exact x-coordinate and the exact y-coordinate of the vertex describe the state of the polygon vertex. The specific calculation method is as follows:

[0120]

[0121] p j is the polygon vertex in the raw material board layout, and finally the state encoding feature matrix, that is, s t =H s Encoding reinforcement learning states in this way is simple and can include all states that appear in the nesting process. It has good generalization and different nesting problems (raw materials, parts) can use the same encoding method.

[0122] The reward function R in S2 t (a t |s t ), including R S,t R C,t and R SC,t These three parts include:

[0123] R S,t is the current placement polygon area, which is used to evaluate the size of the polygon area after each action. Subjectively, we prefer to place the largest polygon possible first. S,t The calculation method is as follows:

[0124]

[0125] where p i,t are the two-dimensional coordinate vertices in the convex hull.

[0126] R C,t Used to evaluate the compactness of placement. This score is composed of the Minkowski vector sum of the raw material board and the expanded polygon of the current polygon, and the Minkowski vector sum of the current layout area and the expanded polygon of the current polygon. Figure 6 , the specific calculation formula is as follows:

[0127]

[0128] R SC,t In the design of reward functions for tasks and planning goals, this patent considers infinite length of a single side. If the y direction of the raw material plate is fixed, the x direction can be infinitely extended. It is hoped that the distance between the rightmost vertex of the polygon and the origin in the final layout result is as short as possible. Based on this optimization goal, R SC,t It can be designed as the inverse of the product of the length added in the x direction after the current polygon is placed and the board width d. Figure 7 , R SC,t The calculation formula is as follows:

[0129] R SC,t =-d×(max(P t (x))-max({P i,m (x)}))

[0130] Final reward function R t (a t |s t ) is calculated as follows:

[0131] R t (a t |s t )=w1R S,t +w2R C,t +w3R SC,t

[0132] w1, w2, and w3 are the weights of the three scores. If w1, w2, and w3 are 0.4, 0.4, and 0.2 respectively, the reward function is finally: R t (a t |s t )=0.4R S,t +0.4R C,t +0.2R SC,t .

[0133] The value network Q(a t ,s t ) and the target network T(a t ,s t ) neural network, exactly the same, the design can use the convolutional neural network CNN model, the input is the current state s t Corresponding image H s and action a t Corresponding image H a , the output is the prediction score and the actual score U t , these two networks are exactly the same, and the specific structure of the neural network is as follows:

[0134] First, the action diagram H a As the convolution kernel, for the state graph H s Perform a convolution to obtain the first hidden layer hidden1;

[0135] After using 64 3×3 convolution kernels and the convolution layer of the LeakyReLU activation function to complete convolution twice on hidden1, a 2×2 maximum pooling layer with a stride of 2 is used to complete pooling to obtain the second hidden layer hidden2;

[0136] After using 128 3×3 convolution kernels and the convolution layer of the LeakyReLU activation function to complete convolution twice on hidden2, a 2×2 maximum pooling layer with a stride of 2 is used to complete pooling to obtain the third hidden layer hidden3;

[0137] After using 256 3×3 convolution kernels and the convolution layer of the LeakyReLU activation function to complete convolution twice on hidden3, a 2×2 maximum pooling layer with a stride of 2 is used to complete pooling to obtain the fourth hidden layer hidden4;

[0138] After using 512 3×3 convolution kernels and the convolution layer of the LeakyReLU activation function to complete convolution twice on hidden4, a 2×2 maximum pooling layer with a stride of 2 is used to complete pooling to obtain the fifth hidden layer hidden5;

[0139] After using 512 3×3 convolution kernels and the convolution layer of the LeakyReLU activation function to complete convolution twice on hidden5, a 2×2 maximum pooling layer with a stride of 2 is used to complete pooling to obtain the fifth hidden layer hidden6;

[0140] The sixth hidden layer hidden7 is obtained by using a fully connected layer with 4096 neurons and LeakyReLU activation function, the seventh hidden layer hidden8 is obtained by using a fully connected layer with 1024 neurons and LeakyReLU activation function, and finally the final output is obtained by using a fully connected layer with 1 neuron and LeakyReLU activation function. The neural network structure is as follows Figure 8 shown.

[0141] In the design of the neural network, the first layer uses the action graph to perform a convolution on the state graph, fully considering the relationship between the current layout and the polygons. The subsequent convolutional layers fully extract the features of the image and ultimately improve the total prediction score of the fully connected layer output.

[0142] Further, the three-dimensional thin-walled parts layout algorithm of the raw material plate is infinitely extended on one side, and the specific process of the "layout update" in step S2 is as follows:

[0143] Step 1, decoding action a t , obtain the current placement polygon P(i,k), use Minkowski and calculate the current polygon to be placed P(i,k) and the raw polygon P m The critical inscribed polygon IFP i , and calculate the circumscribed critical polygon {NFP i,j}.

[0144] Step 2: Use the inscribed critical polygon IFP i For all external critical {NFP i,j} Perform a subtraction operation to obtain the collision-free region CFR of the current polygon under the current layout i , the formula is as follows.

[0145]

[0146] Step 3: Use a heuristic placement strategy to complete the placement of the current polygon.

[0147] The heuristic placement strategy adopts the lowest horizontal line placement strategy in the x-direction. This strategy can further limit the expansion of parts in the x-direction and make the parts more compact in the x-direction.

[0148] S3. Place the raw material plate at the coordinate origin, and transform the coordinates of each three-dimensional part in the projection direction to obtain the transformed part P. trans1 , the specific calculation formula is as follows:

[0149] P trans1 =P center [v1 v2 v3]

[0150] Then according to the polygon placement angle θ calculated in S2 out and Position out , for the transformation part P trans1 Coordinate transformation is performed to obtain the three-dimensional parts after final placement in the raw material plate. The calculation method is as follows:

[0151]

[0152] Finally, all parts are placed in a raw material board coordinate system, and the final three-dimensional assembly layout is output.

[0153] Finally, it should be noted that the above are only preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or replace some of the technical features therein by equivalents. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A three-dimensional thin-walled parts layout algorithm for a single-sided infinitely extended raw material plate, characterized in that: The method comprises the following steps: S1. Project the three-dimensional part onto a two-dimensional plane using a three-dimensional projection algorithm, calculate the minimum enveloping polygon of the two-dimensional plane projection point set using a minimum convex hull algorithm, and calculate the expansion polygon of the minimum enveloping polygon using a polygon expansion algorithm; S2. Encode reinforcement learning actions and states, encode the expanded polygon into a generalized image, the reinforcement learning action is to select a generalized image from it, and encode the current layout in the raw material board into a generalized image as the reinforcement learning state; design a state transition function based on the action and state, and design a reinforcement learning reward function R by comprehensively considering the compactness of the polygon after it is placed and the size of the single polygon area. t (a t |s t ), build a deep reinforcement learning model: design two identical neural networks based on the reinforcement learning state: the value network Q(a t ,s t ) and the target network T(a t ,s t ), and adopt the ε-decay strategy to explore the interaction between the agent and the environment. t The agent chooses action a according to the strategy t After that, the layout is updated to get the new state s t+1 , and according to action a t and status t Calculate the reward R brought by this action t (a t |s t ), the state s t 、Action a t , return R t (a t |s t ) and the new state s t+1 Put it into the experience pool, and calculate the prediction score when the data in the experience pool reaches batch1 and the actual score U t : U t =∑ i≤t γ i R(a i |s i )=R t (a t |s t )+γmax({T(a t ,s t+1 )}), And train and update the value network Q(a t ,s t ) parameter epoch1, when the total number of rounds reaches epoch2, Q(a t ,s t ) parameter is assigned to T(a t ,s t ) Update the target network parameters, output the action sequence when the number of rounds reaches the total number of rounds epoch, and finally generate the placement angle, placement order and placement position of each polygon according to the action sequence; S3. Place the raw material board at the coordinate origin, transform the coordinates of each three-dimensional part in the projection direction to obtain the transformed part 1, and then transform the transformed part 1 according to the polygon placement angle and position calculated in S2 to obtain the final three-dimensional part position after placement in the raw material board. Finally, place all the parts in a raw material board coordinate system and output the three-dimensional assembly after the final layout is completed.

2. The three-dimensional thin-walled parts layout algorithm of the single-sided infinite extension raw material plate according to claim 1 is characterized in that The three-dimensional part formats in the projection algorithm described in step S1 include point cloud format and set polygon vertex format. For the set polygon vertex format part, step S1 can adopt the PCA dimensionality reduction projection method, which specifically includes the following steps: Step 1, read the polygon vertex set {p(x,y,z)} from the three-dimensional part; Step 2: Calculate the center point coordinates Step 3: Centralize the vertex set data set {p(x,y,z)} to get P center = {p(x,y,z)-p mean }; Step 4: Calculate the covariance matrix Step 5: Calculate the covariance matrix eigenvalues ​​[λ1,λ2,λ3] and eigenvectors [v1,v2,v3], sort the eigenvalues, select the eigenvectors corresponding to the two largest eigenvalues ​​as the projection basis, and finally calculate the reduced dimensionality two-dimensional point set P project = {p′(P center,i ·v max1 ,P center,i ·v max2 )}; The point cloud format parts processing method is the same as the polygon vertex format, but the amount of calculation is relatively larger.

3. The three-dimensional thin-walled parts layout algorithm of the single-sided infinite extension raw material plate according to claim 1 is characterized in that The minimum convex hull algorithm calculation described in step S1 can adopt the Bykat algorithm, Akl-Toussaint algorithm, Jarvis matching algorithm and Graham-Andrew scanning algorithm. As a preferred method, the Graham-Andrew scanning algorithm can be adopted. The specific implementation steps are as follows: Step 1: traverse the input polygon vertex set P project Find the point P with the smallest ordinate project,miny As the starting point of the convex hull, that is, P ConvexHull [0] = P project,miny ; Step 2: project All other points in P project,miny Sort polar angles for benchmarks; Step 3, from P ConvexHull [0] Start scanning all points and use vector cross product to determine point P project Whether [t] is inside the convex hull, if so, it is a convex hull point and is added to the convex hull point set, otherwise traverse the next point.

4. The three-dimensional thin-walled parts layout algorithm of the single-sided infinite extension raw material plate according to claim 1 is characterized in that The polygon expansion algorithm described in step S1 can use the method of finding the Minkowski sum of the polygon and the fixed radius circle. The Minkowski sum The calculation can be done by using the convolution-based Minkowski vector and calculation, the specific method is as follows: Step 1: Sort all vertices of polygons M and N to obtain a counterclockwise vertex sequence M = {m i }、N={n i }; Step 2, calculate the simplified convolution K of M and N, that is and A set of line segments, where m i With n j is a convex vertex; Step 3, extract non-overlapping, orientable loops L from K (if the normals of all line segments are consistently facing inward or outward, the loop L is considered orientable), and analyze their nested relationships for further filtering; Step 4, check the intersections between the input polygons placed in the loop layout to filter out the boundaries, and the final result is the Minkowski vector sum; Convert the circle to a polygon and calculate its convex hull {P ConvexHull } of the Minkowski sum to obtain the expanded polygon set {P offset }.

5. The three-dimensional thin-walled parts layout algorithm of the single-sided infinite extension raw material plate according to claim 1 is characterized in that In step S2, "encoding reinforcement learning actions and states", a method for encoding reinforcement learning actions and states is as follows: Reinforce learning action encoding, extract the expansion polygon {P offset } and rotate 360° to generate the expanded polygon sub-polygon set {P i (α)}, each action selects the expanded polygon sub-polygon set {P i (α)}, that is, a t ∈{P i (α)}, the action and dilation subpolygon are represented as the following mapping: the t (i,k)=P(i,kθ) Where i and n are integers, and P(i,kθ) is the set of expanded polygon sub-polygons {P i The sub-polygon generated by rotating the i-th polygon in (α)} by an angle α=kθ; The polygon corresponding to the action is regarded as an image I with d×d pixels. a , pixel value I a (c, r) Whether there is a polygon vertex in the corresponding pixel range, if I a (c,r)=1 indicates that the pixel is a polygon vertex, and the feature matrix is ​​designed and set As the final encoded action, the first feature indicates whether the pixel value is a vertex, and the second and third features are the exact values ​​of the vertex, namely: p j For polygon vertices, the final action is encoded as: a t =H a ; Reinforcement learning state encoding, taking the current layout in the raw material board as the reinforcement learning state, and treating the layout as an image I with pixels w×h to further extract features s , pixel value I s (c,r) represents whether there is a polygon vertex within the pixel range, setting the feature matrix Further description of the status characteristics: Feature Matrix Use the number of polygon vertices in the current pixel I s (c, r), the exact x-coordinate and the exact y-coordinate of the vertex describe the state of the polygon vertex. The specific calculation method is as follows: p j is the polygon vertex in the raw material board layout, and finally the state encoding feature matrix, that is, s t =H s .

6. The three-dimensional thin-walled parts layout algorithm of the single-sided infinite extension raw material plate according to claim 1 is characterized in that Step S2: the reward function R t (a t |s t ), including R S,t R C,t and R SC,t These three parts include: R S,t The area of ​​the current polygon; R C,t Used to evaluate the compactness of placement. This score is composed of the Minkowski vector sum of the raw material board and the expansion polygon of the current polygon, and the Minkowski vector sum of the current layout area and the expansion polygon of the current polygon. The specific calculation formula is as follows: R SC,t One of the reward function designs for the task and planning goals is to take the inverse of the product of the length of the current polygon in the x direction multiplied by the board width d as R SC,t ; Final reward function R t (a t |s t ) is calculated as follows: R t (a t |s t )=w1R S,t +w2R C,t +w3R SC,t w1, w2, and w3 are the weights of the three scores.

7. The three-dimensional thin-walled parts layout algorithm of the single-sided infinite extension raw material plate according to claim 1 is characterized in that Step S2 constructs a deep reinforcement learning model, using the DQN deep reinforcement learning model, where the value network Q(a t ,s t ) and the target network T(a t ,s t ) neural network, exactly the same, the design can use the convolutional neural network CNN model, the input is the current state s t and action a t , the output is the prediction score and the actual score U t , these two networks are exactly the same, first use action graph a t For state diagram s t A convolution and pooling operation is performed, and the subsequent layer_n convolutional layers plus pooling layers and full_connected_n fully connected layers form the final network.

8. The three-dimensional thin-walled parts layout algorithm of the single-sided infinite extension raw material plate according to claim 1 is characterized in that The specific process of "layout update" described in step S2 is as follows: Step 1, decoding action a t , obtain the current placement polygon P(i,k), use Minkowski and calculate the current polygon to be placed P(i,k) and the raw polygon P m The critical inscribed polygon IFP i , and calculate the circumscribed critical polygon {NFP i,j }; Step 2: Use the inscribed critical polygon IFP i For all external critical {NFP i,j } Perform a subtraction operation to obtain the collision-free region CFR of the current polygon under the current layout i , the formula is as follows: Step 3, using a heuristic placement strategy to complete the placement of the current polygon; The heuristic placement strategy may be the bottom-left heuristic method BL and the lowest horizontal line placement strategy, and the lowest horizontal line placement strategy is preferably adopted.

Images