Multi-raw-material-plate three-dimensional part layout algorithm

Abstract

The invention discloses a reinforcement learning-based layout algorithm for thin-wall three-dimensional parts of multiple raw material plates. The method comprises the following steps: completing the generation of a two-dimensional expansion polygon by adopting three-dimensional projection, a minimum convex hull algorithm and a Minkowski vector sum algorithm; and encoding the polygons as images, selecting the images as actions, encoding the layout of all the current raw material plates as images as states, designing action return in combination with the compact degree and the layout conditions of a single raw material plate and multiple raw material plates, and finally obtaining the placement sequence, angle and position of each polygon and the placed raw material plate by using deep enhanced iterative learning. The device is suitable for the discharging problem of one or more raw material plates; a reinforcement learning method is used for fully exploring the placing state, the influence of the part placing sequence and the placed raw material plates on the final result is fully considered, and a better and even optimal discharging mode can be quickly given; in addition, the generalization ability is high by using deep learning fitting reinforcement learning value functions, and the parameters are still practical for new problems.

Description

Technical Field

[0001] The present invention relates to the two-dimensional blanking problem and optimization field of multiple raw material plates, and specifically is a method for obtaining a two-dimensional point cloud set of three-dimensional parts by using a projection algorithm, and obtaining a two-dimensional part placement order, raw material plate placement, placement angle and placement position by using iterative learning of reinforcement learning. Background Art

[0002] In the field of modern manufacturing, the three-dimensional sheet metal arrangement problem of multiple raw material plates is a key engineering optimization task, which involves how to efficiently arrange and cut parts of the required shape with limited sheet metal resources. The complexity of this problem comes from the three-dimensional nature of the sheet metal and the need to consider many constraints during the manufacturing process, such as minimizing raw material waste, reducing production costs, and improving production efficiency. Therefore, solving the three-dimensional sheet metal arrangement problem of multiple raw material plates has important theoretical and practical significance for promoting the digital transformation of the manufacturing industry and improving resource utilization efficiency.

[0003] Traditional sheet metal nesting problems mainly focus on one-dimensional linear nesting and two-dimensional plane nesting. However, with the advancement of manufacturing technology and the complexity of product structure, the demand for multi-material three-dimensional sheet metal nesting is increasing. In the fields of aerospace, automobile manufacturing, construction, etc., the shapes and structures of many parts have complex three-dimensional features, so the traditional two-dimensional nesting method can no longer meet the needs of actual production.

[0004] Among them, the patents related to the layout algorithm of irregular three-dimensional thin plate parts are: a knowledge-driven two-dimensional polygonal automatic layout method invented by Feng Yixiong et al. This method traverses all angles of all polygons, calculates the angle and position corresponding to the maximum fitness of each part, and arranges them. This method does not consider the influence of the polygon placement order on the layout results in the actual layout process, and the calculation amount is huge and it is difficult to use in medium and large-scale projects. In addition, this method only considers two-dimensional layout, which is difficult to use in three-dimensional part layout projects; Gao Duo et al. invented the optimization method and system for the layout of two-dimensional heterogeneous plates, and the readable storage medium method, which uses a genetic algorithm-based method to complete the layout. This method still does not consider the influence of the placement order on the final result, and it is still difficult to use in three-dimensional layout projects. In addition, the existing patents have given little consideration to the two-dimensional layout problem of multiple raw material plates, which is difficult to meet the algorithm of placing multiple parts on one side of a square plate in actual applications. Summary of the invention

[0005] In response to the above problems, this patent proposes an irregular three-dimensional parts layout algorithm for multiple raw material plates based on reinforcement learning. Through the projection algorithm and the two-dimensional convex hull algorithm, the three-dimensional parts are compressed and projected into two-dimensional polygons in a certain direction. The current layout of the multiple raw material plates is encoded as the reinforcement learning state, and the sub-polygon set after all polygons are rotated at a fixed angle is used as the action. The reinforcement learning model is constructed and iteratively optimized to obtain the layout results.

[0006] This patent invents a three-dimensional parts layout algorithm for multiple raw material plates, which includes the following steps:

[0007] S1. Use the dimensionality reduction projection algorithm to project the three-dimensional part into a two-dimensional point set, use the minimum convex hull algorithm to calculate the minimum enveloping convex polygon of the two-dimensional point set, and use the polygon expansion algorithm to calculate the expansion polygon of the minimum enveloping convex polygon to leave enough processing allowance for subsequent part processing.

[0008] The beneficial effects of the dimensionality reduction projection algorithm and the minimum convex hull algorithm are: extracting the contour edge of the part and reducing the dimension from three dimensions to two dimensions, eliminating interference dimensions to reduce the amount of calculation, calculating the contour convex hull to further eliminate the influence of internal element points on the calculation, and simplifying the point set into a convex polygon point sequence to reduce the amount of calculation for the subsequent nesting algorithm.

[0009] The formats of the three-dimensional parts in S1 include point cloud format and set polygon vertex format. The dimension reduction projection algorithm uses the Rodriguez rotation formula to calculate the rotation matrix R of the plane normal vector to the z-axis of the coordinate system, and compresses the z coordinate to obtain a two-dimensional point set. If all vertex sets of the three-dimensional part are {p(x, y, z)}, the given projection plane is A: ax+by+cz+d=0, and the specific projection steps are as follows:

[0010] Step 1: According to the normal vector n of plane A A and z-axis direction to calculate the rotation axis, that is, v = n A ×z;

[0011] Step 3, calculate the normal vector n of A A The rotation angle to the z-axis is:

[0012]

[0013] Step 3, calculate the Rodriguez rotation matrix:

[0014] R=I+(1-cos(β))V×V+sin(β)V

[0015] Where V is the matrix form of the rotation axis v, namely:

[0016]

[0017] Step 4: traverse and calculate the coordinates of the rotated vertices, i.e., pR (x,y,z)=R×p(x,y,z);

[0018] Step 5: Eliminate the z coordinates of the rotated vertices to obtain the two-dimensional point set P project ={p′(x,y)};

[0019] The projection algorithm of point cloud format 3D parts is similar to that of polygon vertex set format, but the amount of calculation is relatively larger. The beneficial effect of using the Rodriguez format rotation formula is that the algorithm simplifies the amount of calculation of the projection algorithm. In addition, this method can manually specify the projection square, and for 3D parts with unclear thin-wall features, the projection direction can be manually specified, which improves the stability of the algorithm.

[0020] Furthermore, the minimum convex hull algorithm calculation in S1 can adopt the Bykat algorithm, Akl-Toussaint algorithm, Jarvis matching algorithm and Graham-Andrew scanning algorithm, among which the Graham-Andrew scanning algorithm is simple and easy to implement, and has a small amount of calculation. As a preferred method, the Graham-Andrew scanning algorithm can be adopted. The specific implementation steps are as follows:

[0021] Step 1: traverse the input polygon vertex set P project Find the point P with the smallest ordinate project,miny As the starting point of the convex hull, that is, P ConvexHull [0] = P project,miny ;

[0022] Step 2: project All other points in P project,miny Sort polar angles for benchmarks;

[0023] Step 3, from P ConvexHull [0] Start scanning all points and use vector cross product to determine point P project Whether [t] is inside the convex hull, if so, it is a convex hull point and is added to the convex hull point set, otherwise traverse the next point.

[0024]

[0025] The polygon expansion algorithm in step S1 can use the method of finding the Minkowski sum of the polygon and the fixed radius circle. The calculation can be done by using the convolution-based Minkowski vector and calculation, the specific method is as follows:

[0026] Step 1: Sort all vertices of polygons M and N to obtain a counterclockwise vertex sequence M = {m i}、N={n i};

[0027] Step 2, calculate the simplified convolution K of M and N, that is and A set of line segments, where m i With n j is a convex vertex;

[0028] Step 3, extract non-overlapping, orientable loops L from K (if the normals of all line segments are consistently facing inward or outward, the loop L is considered orientable), and analyze their nested relationships for further filtering;

[0029] Step 4, check the intersections between the input polygons placed in the loop layout to filter out the boundaries, and the final result is the Minkowski vector sum;

[0030] Convert the circle to a polygon and calculate its convex hull {P ConvexHull} of the Minkowski sum to obtain the expanded polygon set {P offset}.

[0031] In industrial production, the blank parts cut from the raw material plate often need to reserve machining allowance for subsequent processing. Therefore, setting the expansion radius according to the machining allowance given by the subsequent part processing technology and calculating the expansion polygon as the layout target can avoid the problem of insufficient machining allowance after cutting.

[0032] S2. Encode reinforcement learning actions and states, encode all expanded polygons into images, each selection of a polygon is a reinforcement learning action, encode the layout state of each raw material board into an image, and stack all raw material board encoding images to form a reinforcement learning state; design a state transition function based on actions and states, and design a reinforcement learning reward function R by comprehensively considering the density of a single raw material board and the overall layout of multiple raw material boards t (a t |s t ) ; Build a deep reinforcement learning model: design two identical neural networks based on the reinforcement learning state: the value network Q(a t ,s t ) and the target network T(a t ,s t ), and adopt the ε-decay strategy to explore the interaction between the agent and the environment. t The agent chooses action a according to the strategy t After that, the layout is updated to get the new state s t+1 , the state s t 、Action a t , new state s t+1 and return R t (a t |s t) into the experience pool. When the amount of data in the experience pool reaches the batch group, each time the agent interacts with the environment, a batch of data is randomly taken out and the prediction score is calculated. and the actual score U t The specific calculation formula is as follows:

[0033]

[0034] U t =∑ i≤t γ i R(a i |s i )=R t (a t |s t )+γmax({max({T(a i ,s t+1 )})})

[0035] And train and update Q(a t ,s t ) parameter epoch1 round, when the number of iterations reaches epoch2, Q(a t ,s t ) parameter is assigned to T(a t ,s t ) Update the target network parameters, output the action sequence when the number of iterations reaches epoch, and finally generate the placement angle, placement order, placement position and raw material board index of each polygon according to the action sequence.

[0036] The beneficial effects of using enhanced deep learning to solve the problem of multiple raw material plate layout are: it can fully explore the unknown areas in the search space, find the most suitable polygonal layout angles and the raw material plates to be placed therein, and the deep learning method is used to formulate strategy functions with good generalization capabilities. In theory, it can also solve new layout problems after training is completed.

[0037] Furthermore, in step S2, “encoding reinforcement learning actions and states”, a method for encoding reinforcement learning actions and states is as follows:

[0038] Reinforcement learning state encoding, taking the current layout of all raw material boards as reinforcement learning state, if there are N m Raw material board P m , then the current layout of each raw material board is regarded as an image I with pixels w×h s , pixel value I s (c,r) represents whether the pixel is inside the placed polygon, that is:

[0039]

[0040] Reinforce the learning state I by describing the current layout image of each raw material board s,j Stacked up as the final reinforcement learning state, that is, The final reinforcement learning state is encoded as

[0041] The current layout of multiple raw material plates is used as the state, and the layout is encoded in an image manner and stacked up as a whole as a reinforcement learning state. The beneficial effect is that the image encoding appropriately ignores features smaller than a single pixel, fully extracts the main features of the state, ensures the accuracy of the model, and improves the convergence speed of subsequent model training.

[0042] Reinforce learning action encoding, extract the expansion polygon {P offset} and rotate 360° to generate the expanded polygon sub-polygon set {P i (α)}, each action selects the expanded polygon sub-polygon set {P i (α)}, the action and dilation subpolygon are represented as the following mapping:

[0043] a t (i,k)=P(i,kθ)

[0044] Where i and n are integers, and P(i,kθ) is the set of expanded polygon sub-polygons {P i The sub-polygon generated by rotating the i-th polygon in (α)} by an angle α=kθ.

[0045] The expanded polygon P i (α) is considered as an image I with d×d pixels. a , pixel value I a (c, r) corresponds to whether the coordinates of the pixel point are within the polygon. If I a (c,r)=1 means that the pixel is inside the polygon. a If (c, r) = 0, the pixel is not within the polygon, and the action is finally encoded as a feature matrix.

[0046]

[0047] The beneficial effect of encoding reinforcement learning actions in this way is that it can unify all polygons in two-dimensional pattern-laying problems and has strong generalization ability.

[0048] The reward function R in S2 t (a t |s t ), including R S,t , R C,t , R SC,t and R A,tFour parts, including:

[0049] R S,t is the current placement polygon area, which is used to evaluate the size of the polygon area after each action. Subjectively, we prefer to place the largest polygon possible first. S,t The calculation method is as follows:

[0050]

[0051] where p i,t are the two-dimensional coordinate vertices in the convex hull.

[0052] R C,t Used to evaluate the compactness of placement. This score is composed of the Minkowski vector sum of the raw material board and the expansion polygon of the current polygon, and the Minkowski vector sum of the current layout area and the expansion polygon of the current polygon. The specific calculation formula is as follows:

[0053]

[0054] R SC,t In the design of the reward function for the task and planning goals, if a raw material board cannot be placed completely after the final layout in the patent, it is hoped that the rectangular area it occupies is as small as possible. Taking all raw material boards as rectangles as an example, this item is considered to be the opposite of the increase in the area in the x direction of the placed raw material board after each placement of the polygon. The calculation method is as follows:

[0055] R SC,t =-d×(max(P t (x))-max({P i,m (x)}))

[0056] Where P i,m (x) is the x-coordinate set of all polygons in the raw material board, P t (x) is the x-coordinate set of the current placement polygon. This method can be used to punish a part that has an unreasonable placement angle, causing the x-direction area of ​​the raw material board to increase too much after the current part is placed.

[0057] R A,t It is used to evaluate the overall layout of multiple raw material boards. This part is evaluated and calculated once after each placement of n3 (more than 5 times) polygons. The result of multiplying the standard deviation of the total area of ​​each raw material board polygon to the raw material board area by the increasing coefficient is used as R A,t , the specific calculation method is as follows:

[0058]

[0059] Where n is the current number of interactions between the reinforcement learning agent and the environment, e is a natural constant, and Si is the sum of the areas of all polygons placed on the i-th raw material board, S i,m is the area of ​​the ith raw material board. A,t It can ensure that the final layout result has a large variance in the placement area of ​​each raw material board, combined with R SC,t , tend to leave the raw material boards that cannot be placed completely empty as much as possible or place the polygons as close to one side as possible.

[0060] Final reward function R t (a t |s t ) is calculated as follows:

[0061] R t (a t |s t )=w1R S,t +w2R C,t +w3R SC,t +w4R A,t

[0062] Among them, w1, w2, w3, and w4 are the weights of the four scores.

[0063] In S2, a deep reinforcement learning model is constructed, using the DQN deep reinforcement learning model, in which the value network Q(a t ,s t ) and the target network T(a t ,s t ) neural network, exactly the same, the design can use the convolutional neural network CNN model, the input is the current state s t and action a t , the output is the prediction score and the actual score U t , these two networks are exactly the same, first use action graph a t For the state diagram s t N m A convolution and pooling operation is performed on the subgraphs to obtain the first hidden layer. Subsequently, layer_n convolutional layers and pooling layers with the same parameters are used to extract the first hidden layer information. Finally, full_connected_n fully connected layers with the same parameters are used to form the final network.

[0064] In the design of the neural network, the first layer uses the action graph to perform a convolution on the state graph, fully considering the relationship between the current layout and the polygon. The subsequent convolution layer fully extracts the features of the image and finally improves the output of the fully connected layer to predict the action.

[0065] The specific process of "layout update" in step S2 is as follows:

[0066] Step 1: Input the state and use the ε-decay strategy to randomly generate or calculate the action a with the highest predicted total score t And place the raw material board index, that is:

[0067]

[0068] Step 2, decoding action a t , obtain the current placement polygon P(i,k), use Minkowski and calculate the current polygon to be placed P(i,k) and the raw polygon P m [index]'s critical inscribed polygon IFP i , and calculate the circumscribed critical polygon {NFP i,j};

[0069] Step 3: Use the inscribed critical polygon IFP i For all external critical {NFP i,j} Perform a subtraction operation to obtain the collision-free region CFR of the current polygon under the current layout i , the formula is as follows:

[0070]

[0071] Step 4, using a heuristic placement strategy to complete the placement of the current polygon;

[0072] The heuristic placement strategy may be a bottom-left heuristic method BL, a lowest horizontal line placement strategy, and a four-corner priority placement strategy, and the bottom-left heuristic method BL is preferably used.

[0073] S3. Place multiple raw material plates at the coordinate origins of different assembly files respectively, transform the coordinates of each three-dimensional part in the projection direction to obtain transformed part 1, and then transform the coordinates of transformed part 1 according to the polygon placement angle and position calculated in S2 to obtain the three-dimensional part position after final placement in the raw material plate. Finally, place all parts in the coordinate system of their corresponding raw material plates, and output the three-dimensional assembly of all raw material plate layouts after the final layout is completed.

[0074] The beneficial effects of the present invention are:

[0075] 1. The algorithm flow proposed in this patent is suitable for thin-walled three-dimensional feature parts in mechanical processing. The thin-walled three-dimensional feature parts are converted into two-dimensional polygons using the three-dimensional projection algorithm and the minimum convex hull algorithm.

[0076] 2. The algorithm proposed in this patent is applicable to the arrangement problem of multiple raw material boards, and different optimization goals can be achieved by setting different reinforcement learning rewards. The reinforcement learning reward function designed in this patent fully considers the total area and release of parts placed between multiple raw material boards. If there is a board that cannot be completely placed in the end, it can ensure that the remaining area is the largest and continuous.

[0077] 3. Fully consider the impact of different parts placement orders on the final layout results, making it easier to get the optimal solution. Utilizing the powerful learning ability of the reinforcement learning trial and error process, the unknown state in the placement process is fully explored, and it is not easy to fall into the local minimum point, which further improves the utilization rate of the raw material board.

[0078] 4. Strong generalization ability. Using deep learning network to learn the action value function that predicts the current state and the selected action can converge faster than traditional reinforcement learning. Based on the method of predicting action scores based on deep learning models, theoretically, after using enough training data to complete training, even without training, it can give good sorting results for new problems. BRIEF DESCRIPTION OF THE DRAWINGS

[0079] Figure 1 The figure is a schematic diagram of the overall process of the algorithm of the present invention.

[0080] Figure 2 Generate a flow chart for 3D parts to 2D expanded polygons.

[0081] Figure 3 It is the Minkowski sum of the polygon and the circle, generating a schematic diagram of the expanded polygon.

[0082] Figure 4 It is the state transition diagram of reinforcement learning.

[0083] Figure 5 Reinforcement learning flow chart for polygon alignment of multiple raw material boards.

[0084] Figure 6 Score R for the compactness of polygon placement C,t Schematic diagram.

[0085] Figure 7 For R SC,t Scoring diagram.

[0086] Figure 8 Schematic diagram of the Q network and T network structure. DETAILED DESCRIPTION

[0087] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the accompanying drawings of the present invention.

[0088] The overall process of this method is as follows Figure 1As shown, it includes the following three steps: generating inflated polygons, optimizing and updating the neural network model by interacting with the intelligent agent and the environment in coding reinforcement, and finally obtaining the placement angle sequence and placement position of the polygons and the three-dimensional parts layout. Specifically, it includes the following steps:

[0089] S1. Use the dimensionality reduction projection algorithm to project the three-dimensional parts into a two-dimensional point set. In the factory, the three-dimensional thin-walled parts are usually cut from the raw material plate to obtain the blank parts. The dimensionality reduction projection algorithm can effectively obtain the main structure outline of the three-dimensional thin-walled parts, and convert the three-dimensional layout problem into a two-dimensional layout problem; use the minimum convex hull algorithm to calculate the minimum enveloping convex polygon of the two-dimensional point set, and further simplify the two-dimensional point set into the vertices of the enveloping polygon to greatly reduce the calculation amount of the subsequent algorithm. Use the polygon expansion algorithm to calculate the expansion polygon of the minimum enveloping convex polygon to leave enough processing allowance for subsequent parts processing. The specific process of this process is as follows Figure 2 .

[0090] For three-dimensional parts, it is usually difficult for machines to determine the cutting direction plane in the raw material plate, so the projection plane is usually given by the designer. The Rodriguez rotation formula is simple to calculate and can quickly calculate the rotation matrix when a vector is rotated to be collinear with another vector. Therefore, the Rodriguez rotation formula is used to calculate the rotation matrix R of the plane normal vector to the z-axis of the coordinate system, and the z coordinate is compressed to obtain a two-dimensional point set. If the set of all vertices of the three-dimensional part is {p(x,y,z)}, the given projection plane is A: ax+by+cz+d=0, and the specific projection steps are as follows:

[0091] Step 1: According to the normal vector n of plane A A and z-axis direction to calculate the rotation axis, that is, v = n A ×z;

[0092] Step 2, calculate the normal vector n of A A The rotation angle to the z-axis is:

[0093]

[0094] Step 3, calculate the Rodriguez rotation matrix:

[0095] R=I+(1-cos(β))V×V+sin(β)V

[0096] Where V is the matrix form of the rotation axis v, namely:

[0097]

[0098] Step 4: traverse and calculate the coordinates of the rotated vertices, i.e., p R (x,y,z)=R×p(x,y,z);

[0099] Step 5: Eliminate the z coordinates of the rotated vertices to obtain the two-dimensional point set P project ={p′(x,y)};

[0100] The projection algorithm of 3D parts in point cloud format is similar to that in polygon vertex set format, but the amount of calculation is relatively larger.

[0101] In order to obtain the outer contour of the projected point set, the minimum convex hull algorithm can be used to further reduce the amount of algorithm calculation. The minimum convex hull algorithm can use the Bykat algorithm, the Akl-Toussaint algorithm, the Jarvis matching algorithm and the Graham-Andrew scanning algorithm. Among them, the Graham-Andrew scanning algorithm is simple and easy to implement, and the amount of calculation is small. As a preferred option, the Graham-Andrew scanning algorithm can be used. The specific implementation steps are as follows:

[0102] Step 1: traverse the input polygon vertex set P project Find the point P with the smallest ordinate project,miny As the starting point of the convex hull, that is, P ConvexHull [0] = P project,miny ;

[0103] Step 2: project All other points in P project,miny Sort polar angles for benchmarks;

[0104] Step 3, from P ConvexHull [0] Start scanning all points and use vector cross product to determine point P project Whether [t] is inside the convex hull, if so, it is a convex hull point and is added to the convex hull point set, otherwise traverse the next point.

[0105]

[0106] In actual production, the blank parts cut out of thin walls often need to leave enough margin for subsequent part processing. This process can be imagined as a new shape obtained after the center of a circle with a fixed radius circles around the polygonal contour, such as Figure 3 As shown in the figure, this process can be regarded as the Minkowski sum of a polygon and a fixed radius circle. Therefore, the polygon expansion algorithm in S1 can be calculated by the method of finding the Minkowski sum of a polygon and a fixed radius circle. The Minkowski sum The calculation can be done by using the convolution-based Minkowski vector and calculation, the specific method is as follows:

[0107] Step 1: Sort all vertices of polygons M and N to obtain a counterclockwise vertex sequence M = {m i}、N={n i};

[0108] Step 2, calculate the simplified convolution K of M and N, that is and A set of line segments, where m i With n j is a convex vertex;

[0109] Step 3, extract non-overlapping, orientable loops L from K (if the normals of all line segments are consistently facing inward or outward, the loop L is considered orientable), and analyze their nested relationships for further filtering;

[0110] Step 4, check the intersections between the input polygons placed in the loop layout to filter out the boundaries, and the final result is the Minkowski vector sum;

[0111] Convert the circle to a polygon and calculate its convex hull {P ConvexHull} of the Minkowski sum to obtain the expanded polygon set {P offset}.

[0112] S2. Encode reinforcement learning actions and states, encode all expanded polygons into images, each selection of a polygon is a reinforcement learning action, encode the layout state of each raw material board into an image, and stack all raw material board encoding images to form a reinforcement learning state; design a state transition function based on actions and states, and design a reinforcement learning reward function R by comprehensively considering the density of a single raw material board and the overall layout of multiple raw material boards t (a t |s t ) ; Build a deep reinforcement learning model: design two identical neural networks based on the reinforcement learning state: the value network Q(a t ,s t ) and the target network T(a t ,s t ), and adopts the ε-decay strategy to explore the interaction between the agent and the environment.

[0113] Using multiple sets of data from the experience pool to train the model can avoid the problem of slow convergence caused by random errors. Alternating and rotating updates of the value network and the target network fixes the parameters of a single model and greatly improves the convergence speed.

[0114] At the beginning of the training round, set the total number of reinforcement learning training rounds epoch = 20000, set the current round number epoch_now = 1, initialize the number of interactions between the current agent and the environment n = 1, and initialize the environment to all raw material plates without any part polygons {P m}, initialize the greedy exploration coefficient ε t =1, the decreasing rate of greedy exploration coefficient during training is γ ε =0.9975, greedy exploration coefficient minimum value ε min = 0.1. Set the value network Q(at ,s t ) Learning rate lr = 0.005, set the cumulative reward function discount rate of reinforcement learning γ = 0.99, set the initialization experience pool pool = {}, and set the maximum data size of the experience pool to batch max =10000, set the number of samples for each learning sampling batch = 512, set the number of update rounds of the value network Q epoch1 = 100, and set the update frequency of the target network T epoch2 = 50.

[0115] According to the reinforcement learning value network Q(a t ,s t ), input current state s t and action a t , output action score Q(a t |s t ), according to the ε-greedy strategy, generate a random number Randnum between 0 and 1. If Randnum ≤ ε t Then take a random action, otherwise take the action with the highest score. If the total number of raw material boards in the case is N m =4, then the value network Q(a t |s t )The output dimension is N m = 4 dimensions, each dimension represents the predicted score of the current action placed in the corresponding raw material board, and finally the dimension with the highest score is selected as the raw material board index where the polygon is placed, i.e. a t ,index=argmax ai,index ({Q(s t , a i )}).

[0116] In status t The agent chooses action a according to the strategy t After placing the raw material board index index, update the layout state to enter the new state s t+1 , and determine whether the new state is a terminal state. If it is a terminal state, initialize the number of interactions between the agent and the environment and update the current round number, that is, n = 1, epoch_now = epoch_now + 1, and then according to action a t and status t Calculate the reward R brought by this action t (a t |s t ), the state s t 、Action a t , new state s t+1 and return R t (a t |s t) into the experience pool, update the number of interactions between the current agent and the environment, the greedy exploration coefficient and the state, n = n + 1, ε = max (γ ε ε,ε min ),s t =s t+1 .

[0117] When the amount of data in the experience pool reaches a batch, the agent randomly takes out a batch of data from the experience pool each time it interacts with the environment and calculates the prediction score. and the actual score U t :

[0118]

[0119] U t =∑ i≤t γ i R(a i |s i )=R t (a t |s t )+γmax({max({T(a i ,s t+1 )})})

[0120] Obtain multiple sets of states, actions, predicted scores, and actual scores, and use the gradient descent method to update the value network Q(a t ,s t ) parameter epoch1, when the number of iterations epoch_now is an integer multiple of epoch2, Q(a t ,s t ) parameter is assigned to T(a t ,s t ) Update the target network parameters, and output the action sequence when the iteration number epoch_now reaches epoch = 20000 times. Finally, generate the placement angle, placement order, placement position and raw material board index of each polygon according to the action sequence. Strengthen the learning state transition and the overall learning process as follows Figure 4 , Figure 5 As shown;

[0121] The use of enhanced deep learning to solve the problem of multiple raw material plate layout can fully explore the unknown areas in the search space, find the most suitable polygonal layout angle and the raw material plate to be placed therein, and the use of deep learning to formulate strategy functions has good generalization ability. In theory, it can also solve new layout problems after training is completed.

[0122] In step S2, the reinforcement learning action and state are encoded. As a preferred method for encoding the reinforcement learning action and state, the following is used:

[0123] Reinforcement learning state encoding, taking the current layout of all raw material boards as reinforcement learning state, if there are N m = 4 same raw material boards P m (length and width are a = 2000 mm, b = 2000 mm), then the current layout of each raw material board is regarded as an image I with pixels w×h s (where w = 500, d = 500), pixel value I s (c,r) represents whether the pixel is inside the placed polygon, that is:

[0124]

[0125] Reinforce the learning state I by describing the current layout image of each raw material board s,j Stacked up as the final reinforcement learning state, that is, The final reinforcement learning state is

[0126] The current layout of multiple raw material plates is taken as the state, encoded in the form of images, and stacked up as a whole as the reinforcement learning state. The image encoding appropriately ignores the features smaller than a single pixel and fully extracts the main features of the state, which ensures the accuracy of the model while improving the convergence speed of subsequent model training.

[0127] Reinforce learning action encoding, extract the expansion polygon {P offset} and rotate 360° to generate the expanded polygon sub-polygon set {P i (α)}, each action selects the expanded polygon sub-polygon set {P i (α)}, the action and dilation subpolygon are represented as the following mapping:

[0128] a t (i,k)=P(i,kθ)

[0129] Where i and n are integers, and P(i,kθ) is the set of expanded polygon sub-polygons {P i The sub-polygon generated by rotating the i-th polygon in (α)} by an angle α=kθ.

[0130] The expanded polygon P i (α) is regarded as an image I with d×d pixels (d depends on the longest side of the largest part and has the same point distance as the state diagram, here d=50) a , pixel value I a (c, r) corresponds to whether the coordinates of the pixel point are within the polygon. If I a (c,r)=1 means that the pixel is inside the polygon. aIf (c, r) = 0, the pixel is not within the polygon, and the action is finally encoded as a feature matrix.

[0131]

[0132] This method encodes reinforcement learning actions and can unify all polygons in two-dimensional pattern-laying problems, with strong generalization.

[0133] The reward function R in S2 t (a t |s t ), including R S,t , R C,t , R SC,t and R A,t Four parts, including:

[0134] R S,t is the current placement polygon area, which is used to evaluate the size of the polygon area after each action. Subjectively, we prefer to place the largest polygon possible first. S,t The calculation method is as follows:

[0135]

[0136] where p i,t are the two-dimensional coordinate vertices in the convex hull.

[0137] R C,t Used to evaluate the compactness of placement. This score is composed of the Minkowski vector sum of the raw material board and the expanded polygon of the current polygon, and the Minkowski vector sum of the current layout area and the expanded polygon of the current polygon. Figure 6 , the specific calculation formula is as follows:

[0138]

[0139] R SC,t In the design of reward functions for tasks and planning goals, if a raw material board cannot be placed completely after the final arrangement in the patent, it is hoped that the rectangular area it occupies is as small as possible. Taking all raw material boards as rectangles as an example, consider this item as the opposite of the increase in the area in the x direction of the raw material board after each placement of the polygon, such as Figure 7 , calculated as follows:

[0140] R SC,t =-d×(max(P t (x))-max({P i,m (x)}))

[0141] Where P i,m (x) is the x-coordinate set of all polygons in the raw material board, Pt (x) is the x-coordinate set of the current placement polygon. This method can be used to punish a part that has an unreasonable placement angle, causing the x-direction area of ​​the raw material board to increase too much after the current part is placed.

[0142] R A,t It is used to evaluate the overall layout of multiple raw material boards. This part is evaluated and calculated once after placing n3 (more than 5 times) polygons. The result of multiplying the standard deviation of the total area of ​​each raw material board by the increasing coefficient is used as R A,t , the specific calculation method is as follows:

[0143]

[0144] Where n is the current number of interactions between the reinforcement learning agent and the environment, e is a natural constant, and S i is the sum of the areas of all polygons placed on the i-th raw material board, S i,m is the area of ​​the ith raw material board. A,t It can ensure that the final layout result has a large variance in the placement area of ​​each raw material board, combined with R SC,t , tend to leave the raw material boards that cannot be placed completely empty as much as possible or place the polygons as close to one side as possible.

[0145] Final reward function R t (a t |s t ) is calculated as follows:

[0146] R t (a t |s t )=w1R S,t +w2R C,t +w3R SC,t +w4R A,t

[0147] Among them, w1, w2, w3, and w4 are the weights of the four scores. In the placement task, the most important requirement is usually a high degree of compactness. The values ​​of w1, w2, w3, and w4 can be 0.1, 0.5, 0.25, and 0.25 respectively.

[0148] The deep reinforcement learning model is constructed as described in S2, and the DQN deep reinforcement learning model is adopted, in which the value network Q(a t ,s t ) and the target network T(a t ,s t ) The neural network is exactly the same. The design can use the convolutional neural network CNN model, and the input is the current state s t and action a t , the output is the prediction scores of multiple raw material boards and the actual score U t , these two networks are exactly the same, first use action graph a t For the state diagram s t N m The subgraphs are convolved and pooled once to get the first hidden layer. Subsequently, layer_n convolutional layers and pooling layers with the same parameters are used to extract the information of the first hidden layer. Finally, full_connected_n fully connected layers with the same parameters are used to form the final network. The specific structure of the neural network is as follows:

[0149] The action diagram H a As the convolution kernel, for the state graph H s All sub-graphs in the convolution are convolved once to obtain the first hidden layer hidden1;

[0150] After using 64 3×3 convolution kernels and the convolution layer of the LeakyReLU activation function to complete convolution twice on hidden1, a 2×2 maximum pooling layer with a stride of 2 is used to complete pooling to obtain the second hidden layer hidden2;

[0151] After using 128 3×3 convolution kernels and the convolution layer of the LeakyReLU activation function to complete convolution twice on hidden2, a 2×2 maximum pooling layer with a stride of 2 is used to complete pooling to obtain the third hidden layer hidden3;

[0152] After using 256 3×3 convolution kernels and the convolution layer of the LeakyReLU activation function to complete convolution twice on hidden3, a 2×2 maximum pooling layer with a stride of 2 is used to complete pooling to obtain the fourth hidden layer hidden4;

[0153] After using 512 3×3 convolution kernels and the convolution layer of the LeakyReLU activation function to complete convolution twice on hidden4, a 2×2 maximum pooling layer with a stride of 2 is used to complete pooling to obtain the fifth hidden layer hidden5;

[0154] After using 512 3×3 convolution kernels and the convolution layer of the LeakyReLU activation function to complete convolution twice on hidden5, a 2×2 maximum pooling layer with a stride of 2 is used to complete pooling to obtain the fifth hidden layer hidden6;

[0155] The sixth hidden layer hidden7 is obtained by using a fully connected layer with 4096 neurons and LeakyReLU activation function, the seventh hidden layer hidden8 is obtained by using a fully connected layer with 1024 neurons and LeakyReLU activation function, and finally the final output is obtained by using a fully connected layer with 1 neuron and LeakyReLU activation function. The neural network structure is as follows Figure 8 shown.

[0156] In the design of the neural network, the first layer uses the action graph to perform a convolution on the state graph, fully considering the relationship between the current layout and the polygons. The subsequent convolutional layers fully extract the features of the image and ultimately improve the total prediction score of the fully connected layer output.

[0157] The specific process of layout update in step S2 is as follows:

[0158] Step 1: Input the state and use the ε-decay strategy to randomly generate or calculate the action a with the highest predicted total score t And place the raw material board index, that is:

[0159]

[0160] Step 2, decoding action a t , obtain the current placement polygon P(i,k), use Minkowski and calculate the current polygon to be placed P(i,k) and the raw polygon P m [index]'s critical inscribed polygon IFP i , and calculate the circumscribed critical polygon {NFP i,j};

[0161] Step 3: Use the inscribed critical polygon IFP i For all external critical {NFP i,j} Perform a subtraction operation to obtain the collision-free region CFR of the current polygon under the current layout i , the formula is as follows:

[0162]

[0163] Step 4, using a heuristic placement strategy to complete the placement of the current polygon;

[0164] The heuristic placement strategies may be the bottom-left heuristic method BL, the lowest horizontal line placement strategy, and the four-corner priority placement strategy. The simplest and most implementable bottom-left heuristic method BL is preferably used.

[0165] S3. Place multiple raw material plates at the coordinate origins of different assembly files respectively, transform the coordinates of each three-dimensional part in the projection direction to obtain transformed part 1, and then transform the coordinates of transformed part 1 according to the polygon placement angle and position calculated in S2 to obtain the three-dimensional part position after final placement in the raw material plate. Finally, place all parts in the coordinate system of their corresponding raw material plates, and output the three-dimensional assembly of all raw material plate layouts after the final layout is completed.

[0166] Finally, it should be noted that the above are only preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or replace some of the technical features therein by equivalents. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A three-dimensional parts layout algorithm for multiple raw material plates, characterized in that: include: S1. Use the dimensionality reduction projection algorithm to project the three-dimensional part into a two-dimensional point set, use the minimum convex hull algorithm to calculate the minimum enveloping convex polygon of the two-dimensional point set, and use the polygon expansion algorithm to calculate the expansion polygon of the minimum enveloping convex polygon; S2. Encode reinforcement learning actions and states, encode all expanded polygons into images, each selection of a polygon is a reinforcement learning action, encode the layout state of each raw material board into an image, and stack all raw material board encoding images to form a reinforcement learning state; design a state transition function based on actions and states, and design a reinforcement learning reward function R by comprehensively considering the density of a single raw material board and the overall layout of multiple raw material boards t (a t |s t ) ; Build a deep reinforcement learning model: design two identical neural networks based on the reinforcement learning state: the value network Q(a t ,s t ) and the target network T(a t ,s t ), and adopt the ε-decay strategy to explore the interaction between the agent and the environment. t The agent chooses action a according to the strategy t After that, the layout is updated to get the new state s t+1 , the state s t 、Action a t , new state t+1 and return R t (a t |s t ) into the experience pool. When the amount of data in the experience pool reaches the batch group, each time the agent interacts with the environment, a batch of data is randomly taken out and the prediction score is calculated. and the actual score U t The specific calculation formula is as follows: U t =∑ i≤t c i R(a i |s i )=R t (a t |s t )+γmax({max({T(a i ,s t+1 )})}) And train and update Q(a t ,s t ) parameter epoch1 round, when the number of iterations reaches epoch2, Q(a t ,s t ) parameter is assigned to T(a t ,s t ) Update the target network parameters, output the action sequence when the number of iterations reaches epoch, and finally generate the placement angle, placement order, placement position and raw material board index of each polygon according to the action sequence; S3. Place multiple raw material plates at the coordinate origins of different assembly files respectively, transform the coordinates of each three-dimensional part in the projection direction to obtain transformed part 1, and then transform the coordinates of transformed part 1 according to the polygon placement angle and position calculated in S2 to obtain the three-dimensional part position after final placement in the raw material plate. Finally, place all parts in the coordinate system of their corresponding raw material plates, and output the three-dimensional assembly of all raw material plate layouts after the final layout is completed.

2. A three-dimensional parts layout algorithm for multiple raw material plates according to claim 1, characterized in that: The formats of the three-dimensional parts in step S1 include point cloud format and set polygon vertex format. The dimension reduction projection algorithm uses the Rodriguez rotation formula to calculate the rotation matrix R of the plane normal vector to the z-axis of the coordinate system, and compresses the z coordinate to obtain a two-dimensional point set. If all vertex sets of the three-dimensional part are {p(x, y, z)}, the given projection plane is A: ax+by+cz+d=0, and the specific projection steps are as follows: Step 1: According to the normal vector n of plane A A and z-axis direction to calculate the rotation axis, that is, v = n A ×z; Step 2, calculate the normal vector n of A A The rotation angle to the z-axis is: Step 3, calculate the Rodriguez rotation matrix: R=I+(1-cos(β))V×V+sin(β)V Where V is the matrix form of the rotation axis v, namely: Step 4: traverse and calculate the coordinates of the rotated vertices, i.e., p R (x,y,z)=R×p(x,y,z); Step 5: Eliminate the z coordinates of the rotated vertices to obtain the two-dimensional point set P project ={p′(x,y)}; The projection algorithm of 3D parts in point cloud format is similar to that in polygon vertex set format, but the amount of calculation is relatively larger.

3. The three-dimensional parts layout algorithm of multiple raw material plates according to claim 1 is characterized in that The minimum convex hull algorithm calculation described in step S1 can adopt the Bykat algorithm, Akl-Toussaint algorithm, Jarvis matching algorithm and Graham-Andrew scanning algorithm. As a preferred method, the Graham-Andrew scanning algorithm can be adopted. The specific implementation steps are as follows: Step 1: traverse the input polygon vertex set P project Find the point P with the smallest ordinate project,miny As the starting point of the convex hull, that is, P ConvexHull [0] = P project,miny ; Step 2: project All other points in P project,miny Sort polar angles for benchmarks; Step 3, from P ConvexHull [0] Start scanning all points and use vector cross product to determine point P project Whether [t] is inside the convex hull, if so, it is a convex hull point and is added to the convex hull point set, otherwise traverse the next point.

4. The three-dimensional parts layout algorithm of multiple raw material plates according to claim 1 is characterized in that The polygon expansion algorithm described in step S1 can use the method of finding the Minkowski sum of the polygon and the fixed radius circle. The Minkowski sum The calculation can be done by using the convolution-based Minkowski vector and calculation, the specific method is as follows: Step 1: Sort all vertices of polygons M and N to obtain a counterclockwise vertex sequence M = {m i }、N={n i }; Step 2, calculate the simplified convolution K of M and N, that is and A set of line segments, where m i With n j is a convex vertex; Step 3, extract non-overlapping, orientable loops L from K (if the normals of all line segments are consistently facing inward or outward, the loop L is considered orientable), and analyze their nested relationships for further filtering; Step 4, check the intersections between the input polygons placed in the loop layout to filter out the boundaries, and the final result is the Minkowski vector sum; Convert the circle to a polygon and calculate its convex hull {P ConvexHull } of the Minkowski sum to obtain the expanded polygon set {P offset }.

5. The three-dimensional parts layout algorithm of multiple raw material plates according to claim 1 is characterized in that In step S2, "encoding reinforcement learning actions and states", a method for encoding reinforcement learning actions and states is as follows: Reinforcement learning state encoding, taking the current layout of all raw material boards as reinforcement learning state, if there are N m Raw material board P m , then the current layout of each raw material board is regarded as an image I with pixels w×h s , pixel value I s (c,r) represents whether the pixel is inside the placed polygon, that is: Reinforce the learning state I by describing the current layout image of each raw material board s,j Stacked up as the final reinforcement learning state, that is, The final reinforcement learning state is encoded as Reinforce learning action encoding, extract the expansion polygon {P offset } and rotate 360° to generate the expanded polygon sub-polygon set {P i (α)}, each action selects the expanded polygon sub-polygon set {P i (α)}, the action and dilation subpolygon are represented as the following mapping: the t (i,k)=P(i,kθ) Where i and n are integers, and P(i,kθ) is the set of expanded polygon sub-polygons {P i The sub-polygon generated by rotating the i-th polygon in (α)} by an angle α=kθ; The expanded polygon P i (α) is considered as an image I with d×d pixels. a , pixel value I a (c, r) corresponds to whether the coordinates of the pixel point are within the polygon. If I a (c,r)=1 means that the pixel is inside the polygon. a If (c, r) = 0, the pixel is not within the polygon, and the action is finally encoded as a feature matrix.

6. The three-dimensional thin-walled parts layout algorithm of the single-sided infinite extension raw material plate according to claim 1 is characterized in that Step S2: the reward function R t (a t |s t ), including R S,t , R C,t , R SC,t and R A,t Four parts, including: R S,t The area of ​​the current polygon; R C,t Used to evaluate the compactness of placement. This score is composed of the Minkowski vector sum of the raw material board and the expansion polygon of the current polygon, and the Minkowski vector sum of the current layout area and the expansion polygon of the current polygon. The specific calculation formula is as follows: R SC,t For the design of reward functions for tasks and planning goals, take all raw material boards as rectangles as an example, consider this item as the opposite of the increase in the x-direction area of ​​the placed raw material board after each placement of the polygon, and the calculation method is as follows: R SC,t =-d×(max(P t (x))-max({P i,m (x)})) Where P i,m (x) is the x-coordinate set of all polygons in the raw material board, P t (x) is the x-coordinate set of the current placement polygon; R A,t It is used to evaluate the overall layout of multiple raw material boards. This part is evaluated and calculated once after placing n3 (more than 5 times) polygons. The result of multiplying the standard deviation of the total area of ​​each raw material board by the increasing coefficient is used as R A,t , the specific calculation method is as follows: Where n is the current number of interactions between the reinforcement learning agent and the environment, e is a natural constant, and S i is the sum of the areas of all polygons placed on the i-th raw material board, S i,m is the area of ​​the i-th raw material board; Final reward function R t (a t |s t ) is calculated as follows: R t (a t |s t )=w1R S,t +w2R C,t +w3R SC,t +w4R A,t Among them, w1, w2, w3, and w4 are the weights of the four scores.

7. The three-dimensional thin-walled parts layout algorithm of the single-sided infinite extension raw material plate according to claim 1 is characterized in that Step S2 constructs a deep reinforcement learning model, using the DQN deep reinforcement learning model, where the value network Q(a t ,s t ) and the target network T(a t ,s t ) neural network, exactly the same, the design can use the convolutional neural network CNN model, the input is the current state s t and action a t , the output is the prediction score and the actual score U t , these two networks are exactly the same, first use action graph a t For the state diagram s t N m A convolution and pooling operation is performed on the subgraphs to obtain the first hidden layer. Subsequently, layer_n convolutional layers and pooling layers with the same parameters are used to extract the first hidden layer information. Finally, full_connected_n fully connected layers with the same parameters are used to form the final network.

8. The three-dimensional thin-walled parts layout algorithm of the single-sided infinite extension raw material plate according to claim 1 is characterized in that The specific process of "layout update" described in step S2 is as follows: Step 1: Input the state and use the ε-decay strategy to randomly generate or calculate the action a with the highest predicted total score t And place the raw material board index, that is: Step 2, decoding action a t , obtain the current placement polygon P(i,k), use Minkowski and calculate the current polygon to be placed P(i,k) and the raw polygon P m [index]'s critical inscribed polygon IFP i , and calculate the circumscribed critical polygon {NFP i,j }; Step 3: Use the inscribed critical polygon IFP i For all external critical {NFP i,j } Perform a subtraction operation to obtain the collision-free region CFR of the current polygon under the current layout i , the formula is as follows: Step 4, using a heuristic placement strategy to complete the placement of the current polygon; The heuristic placement strategy may be a bottom-left heuristic method BL, a lowest horizontal line placement strategy, and a four-corner priority placement strategy, and the bottom-left heuristic method BL is preferably used.

Images