A multi-material plate three-dimensional part nesting method

Abstract

The application discloses a kind of based on reinforcement learning's multi-raw material board thin-walled three-dimensional part layout algorithm.The application includes: using three-dimensional projection, minimum convex hull algorithm and Minkowski vector sum algorithm to complete the generation of two-dimensional inflation polygon;Polygon is encoded as image, select image as action, current all raw material board layout is encoded as image as state, combine compactness, single raw material board and multiple raw material board layout design action reward, use deep reinforcement iteration learning, finally obtain the placing order, angle, position and raw material board of each polygon.The application is applicable to single or multiple raw material board layout problem;Using the method of reinforcement learning fully explores the placing state, fully considers the influence of part placing order and raw material board on the final result, can quickly give better or even optimal layout mode;In addition, using deep learning to fit reinforcement learning value function has strong generalization ability, and the parameter is still practical for new problem.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of two-dimensional layout problem and optimization of multi-material plate, and particularly relates to a method for obtaining a two-dimensional point cloud set of a three-dimensional part by using a projection algorithm, and obtaining a two-dimensional part placement sequence, a material plate, a placement angle and a placement position by using reinforcement learning iterative learning. BACKGROUND

[0002] In the modern manufacturing field, the three-dimensional plate layout problem of multi-material plate is a key engineering optimization task, which involves how to efficiently arrange and cut the required shape of the part under the limited plate resources. The complexity of this problem comes from the three-dimensional nature of the plate and the need to consider many constraints in the manufacturing process, such as minimizing material waste, reducing production cost, improving production efficiency, etc. Therefore, solving the three-dimensional plate layout problem of multi-material plate has important theoretical and practical significance for promoting the digital transformation of manufacturing industry and improving resource utilization efficiency.

[0003] Traditional plate layout problems mainly focus on one-dimensional linear layout and two-dimensional plane, but with the advancement of manufacturing technology and the complexity of product structure, the demand for multi-material three-dimensional plate layout is increasing. In the fields of aerospace, automobile manufacturing, construction, etc., many parts have complex three-dimensional characteristics, so the traditional two-dimensional layout method cannot meet the needs of actual production.

[0004] Among them, there are patents related to irregular three-dimensional thin plate part layout algorithms: a two-dimensional polygon cloth automatic layout method based on knowledge-driven invented by Feng Yixiong et al. This method traverses all the angles of all polygons, calculates the angle and position corresponding to the maximum fitness of each part and lays out. This method does not consider the influence of polygon placement order on the layout result in the actual layout process, and the calculation amount is huge and difficult to use in medium and large-scale projects. Moreover, this method only considers two-dimensional layout and is difficult to apply in three-dimensional part layout projects; the optimization method and system for two-dimensional irregular plate layout and readable storage medium method invented by Gao Duo, which uses a genetic algorithm-based method to complete the layout. This method still does not consider the influence of placement order on the final result, and is still difficult to use in three-dimensional layout projects. In addition, the current patents have less consideration for the two-dimensional layout problem of multi-material plate, and it is difficult to meet the algorithm of placing multiple parts on one side of a square plate in actual application. SUMMARY

[0005] To solve the above problems, the patent proposes a multi-raw plate irregular three-dimensional part nesting algorithm based on reinforcement learning. The three-dimensional parts are compressed and projected into two-dimensional polygons in a certain direction through projection algorithm and two-dimensional convex hull algorithm. The current layout of the multi-raw plate is encoded as the reinforcement learning state, and the set of sub-polygons obtained by rotating all polygons at a fixed angle is used as the action. The reinforcement learning model is constructed and iteratively optimized to obtain the nesting result.

[0006] The patent invents a multi-raw plate three-dimensional part nesting algorithm, which includes the following steps:

[0007] S1. Adopt dimension reduction projection algorithm to project three-dimensional parts into two-dimensional point set, use minimum convex hull algorithm to calculate the minimum envelope convex polygon of two-dimensional point set, use polygon dilation algorithm to calculate the dilation polygon of minimum envelope convex polygon, and leave enough machining allowance for subsequent part machining.

[0008] The dimension reduction projection algorithm and the minimum convex hull algorithm have the following beneficial effects: extracting the contour edge of the part and reducing the dimension from three-dimensional to two-dimensional, eliminating the interference dimension to reduce the calculation amount, calculating the contour convex hull to further eliminate the influence of internal element points on the calculation, and simplifying the point set to a convex polygon point sequence to reduce the calculation amount for the subsequent nesting algorithm.

[0009] In S1, the format of the three-dimensional part includes point cloud format and set polygon vertex format, the dimension reduction projection algorithm uses the Rodrigues rotation formula to calculate the rotation matrix R of the plane normal vector to the z-axis of the coordinate system, and obtains the two-dimensional point set by compressing the z coordinate. If the three-dimensional part vertex set is {p(x,y,z)}, the given projection plane is A: ax+by+cz+d=0, the specific projection steps are as follows:

[0010] Step 1, calculate the rotation axis according to the plane A normal vector n A and the direction of z-axis, that is, v=n A ×z;

[0011] Step 3, calculate the rotation angle of A normal vector n A to z-axis, that is:

[0012]

[0013] Step 3, calculate the Rodrigues rotation matrix:

[0014] R=I+(1-cos(β))V×V+sin(β)V

[0015] where V is the matrix form of the rotation axis v, that is:

[0016]

[0017] Step 4, traverse and calculate the vertex coordinates after rotation, that is, pR (x, y, z) = R × p(x, y, z);

[0018] Step 5: Eliminate the z coordinates of the rotated vertices to obtain the two-dimensional point set P project ={p′(x,y)};

[0019] The projection algorithm for 3D parts in point cloud format is similar to that in polygon vertex set format, but the computational complexity is relatively higher. The benefit of using the Rodriguez rotation formula for projection is that it simplifies the computational complexity of the projection algorithm. Furthermore, this method allows for the user to specify the projection square, which allows for the user to specify the projection direction for 3D parts with thin-walled features and less pronounced features, thus improving the algorithm's stability.

[0020] Furthermore, the minimum convex hull algorithm calculation in S1 can adopt the Bykat algorithm, Akl-Toussaint algorithm, Jarvis matching algorithm and Graham-Andrew scanning algorithm. Among them, the Graham-Andrew scanning algorithm is simple and easy to implement, and has a small amount of calculation. As a preferred method, the Graham-Andrew scanning algorithm can be adopted. The specific implementation steps are as follows:

[0021] Step 1: Traverse the input polygon vertex set P project Find the point P with the smallest ordinate project,miny As the starting point of the convex hull, that is, P ConvexHull [0] = P project,miny ;

[0022] Step 2: P project All other points in the project,miny Sort by polar angle for benchmark;

[0023] Step 3, from P ConvexHull [0] Start scanning all points and use vector cross product to determine point P project Is [t] inside the convex hull? If so, it is a convex hull point and is added to the convex hull point set. Otherwise, traverse to the next point.

[0024]

[0025] The polygon expansion algorithm in step S1 can use the method of finding the Minkowski sum of the polygon and the fixed radius circle. The calculation can be done by Minkowski vector sum calculation based on convolution, the specific method is as follows:

[0026] Step 1: Sort all vertices of polygons M and N to obtain a counterclockwise vertex sequence M = {m i}、N={n i};

[0027] Step 2, calculate the simplified convolution K of M and N, i.e. and the line segment set, where m i and n j are convex vertices;

[0028] Step 3, extract non-overlapping, orientable cycles L from K (a cycle L is considered to be orientable if the normals of all line segments in the cycle are consistently pointing inwards or outwards), and analyze their nesting relationship for further filtering;

[0029] Step 4, check the intersection between the input polygons placed in the cycle layout to filter out the boundary, and the final result is the Minkowski vector sum;

[0030] Convert the circle to a polygon, and calculate its Minkowski sum with all convex hulls {P ConvexHull} to obtain the inflated polygon set {P offset}.

[0031] In industrial production, the part blanks cut from raw material plates often need to reserve machining allowance for subsequent processing. Therefore, setting the inflation radius according to the machining allowance given by the subsequent part processing process and calculating the inflated polygon as the layout target can avoid the problem of insufficient machining allowance after cutting.

[0032] S2. Encode reinforcement learning actions and states, encode all inflated polygons as images, and select a polygon each time as a reinforcement learning action. Encode the layout state of each raw material plate as an image, and stack all raw material plate encoding images to form a reinforcement learning state. Design a state transition function according to the action and state, and design a reinforcement learning reward function R t (a t |s t ) considering the tightness of single raw material plate layout and the overall layout between multiple raw material plates; build a deep reinforcement learning model: design two identical neural networks according to the reinforcement learning state: value network Q(a t ,s t ) and target network T(a t ,s t ), and use the ε-decay strategy to explore the interaction between the agent and the environment. After the agent selects action a t according to the strategy in state s t , update the layout to get new state s t+1 , and state s t , action a t , new state s t+1 , and reward R t (a t |s t) into the experience pool, when the amount of data in the experience pool reaches the batch group, the agent interacts with the environment every time, and batch data is randomly taken out to calculate the prediction score and the actual score U t The specific calculation formula is as follows:

[0033]

[0034] U t =Σ i≤t γ i R(a i |s i )=R t (a t |s t )+γmax({max({T(a i ,s t+1 )})})

[0035] and train and update the Q(a t ,s t ) parameter epoch1 rounds, when the iteration number reaches epoch2 times, the Q(a t ,s t ) parameter is assigned to the T(a t ,s t ) update target network parameter, when the iteration number reaches epoch times, the action sequence is output, and finally the placement angle, placement order, placement position and raw material board index of each polygon are generated according to the action sequence.

[0036] The beneficial effects of using reinforcement deep learning to solve the multi-raw material plate layout problem are that the unknown areas in the search space can be fully explored to find the most suitable polygon layout angle and its raw material plate, and the strategy function formulated by using deep learning has good generalization ability. In theory, after training is completed, it can also solve new layout problems.

[0037] Further, the "encoding reinforcement learning actions and states" in step S2 includes the following encoding method of reinforcement learning actions and states:

[0038] The reinforcement learning state encoding takes the current layout of all raw material plates as the reinforcement learning state, and if there are N m raw material plates P m , the current layout of each raw material plate is regarded as an image I s with a pixel size of w x h, and the pixel value I s (c, r) represents whether the pixel is inside the placed polygon, that is:

[0039]

[0040] The current layout image of each raw material plate describes the reinforcement learning state I s,j Stacked as the final reinforcement learning state, that is, The final reinforcement learning state is encoded as

[0041] The current layout of multiple raw material plates is taken as a state, the layout is encoded in the form of an image, and the whole is stacked as a reinforcement learning state, which has the beneficial effect that the image encoding appropriately ignores features smaller than a single pixel point, fully extracts the main features of the state, ensures the accuracy of the model, and improves the convergence speed of subsequent model training.

[0042] The reinforcement learning action is encoded, all elements in the inflated polygon {P offset} are taken out and rotated by an angle interval θ to generate an inflated polygon sub-polygon set {P i (α)}, and each action selects an element in the inflated polygon sub-polygon set {P i (α)}, and the action and the inflated sub-polygon are represented as the following mapping:

[0043] a t (i,k)=P(i,kθ)

[0044] where i, n are integers, and P(i,kθ) is the sub-polygon generated after rotating the i-th polygon in the inflated polygon sub-polygon set {P i (α)} by an angle α=kθ.

[0045] The inflated polygon P i (α) can be regarded as an image I a with pixels of dxd, and the pixel value I a (c,r) indicates whether the corresponding pixel coordinates are inside the polygon, if I a (c,r)=1, it means that the pixel is inside the polygon, if I a (c,r)=0, it means that the pixel is not inside the polygon, and finally the action is encoded as a feature matrix

[0046]

[0047] This way of encoding reinforcement learning actions has the beneficial effect of being able to unify all polygons in two-dimensional layout problems, with strong generalization.

[0048] Where the reward function R t (a t |s t ) in S2 includes R S,t , R C,t , R SC,t , and R A,tFour parts, wherein:

[0049] R S,t The current polygon area is used to evaluate the size of the polygon area after each action, and subjectively we prefer to place the largest polygon first, R S,t The calculation method is as follows:

[0050]

[0051] Where p i,t The two-dimensional coordinate vertex in the convex hull.

[0052] R C,t Used to evaluate the compactness of the layout, the score is composed of two parts: the Minkowski vector sum of the raw material plate and the inflated polygon of the current polygon, and the Minkowski vector sum of the current layout area and the inflated polygon of the current polygon, and the specific calculation formula is as follows:

[0053]

[0054] R SC,t The reward function design for the task and planning target, if a raw material plate cannot be placed completely after the final layout in the patent, it is hoped that it occupies a small rectangular area as much as possible, taking all raw material plates as rectangles as an example, considering this item as the opposite number of the increase in x-direction area of the placed raw material plate after placing the polygon each time, the calculation method is as follows:

[0055] R SC,t =-d x (max(P t (x))-max({P i,m (x)}))

[0056] Where P i,m (x) is the x-coordinate set of all polygons in the placed raw material plate, P t (x) is the x-coordinate set of the current polygon placed, this method can punish a part for unreasonable placement angle, which makes the x-direction area of the raw material plate increase too much after placing the part.

[0057] R A,t Used to evaluate the overall layout of the raw material plate, this part is evaluated and calculated once every n3(5 times or more) times of polygon placement, and the result of the standard deviation of the total area of the polygon placed in each raw material plate multiplied by the incremental coefficient is used as R A,t , the specific calculation method is as follows:

[0058]

[0059] Where n is the current interaction times of the reinforcement learning agent and the environment, e is a natural constant, and Si S is the sum of all polygon areas placed for the i-th raw material plate i,m S i is the area of the i-th raw material plate. R A,t The final layout result can guarantee that the area variance of each raw material plate is large, combined with R SC,t , tend to leave the raw material plate that cannot be placed completely as empty as possible or place the polygon as close to one side as possible.

[0060] Final return function R t (a t |s t ) is calculated as follows:

[0061] R t (a t |s t ) = w1R S,t + w2R C,t + w3R SC,t + w4R A,t

[0062] where w1, w2, w3, and w4 are the weights of the four scores.

[0063] where S2 constructs a deep reinforcement learning model, adopts a DQN deep reinforcement learning model, and uses the same value network Q(a t ,s t ) and target network T(a t ,s t ) neural networks. The design can adopt a convolutional neural network CNN model, the input is the current state s t and action a t , and the output is the predicted score and the actual score U t . The two networks are exactly the same, first use the action graph a t to perform convolution and pooling operations on the N m subgraphs in the state graph s t to obtain the first hidden layer, and then use layer_n convolution layers and pool layers with the same parameters to extract the first hidden layer information, and finally use full_connected_n full connection layers with the same parameters to form the final network.

[0064] In the design of the neural network, the first layer uses the action graph to perform one convolution on the state graph, fully considering the relationship between the current layout and the polygon, and the subsequent convolution layers fully extract the features of the image to finally improve the output of the full connection layer to predict the action.

[0065] where the specific process of "layout update" in step S2 is as follows:

[0066] Step 1, input state, randomly generate or calculate the action a with the highest total score by using the epsilon-decay strategy t and the index of the raw material board, namely:

[0067]

[0068] Step 2, decode action a t , obtain the current placement polygon P(i, k), and calculate the critical inscribed polygon IFP of the current to-be-placed polygon P(i, k) and the raw material polygon P m [index] using the Minkowski sum i , and calculate the circumscribed critical polygon {NFP i,j} of all the placed polygons under the current layout at the same time.

[0069] Step 3, use the inscribed critical polygon IFP i to subtract all the circumscribed critical polygons {NFP i,j} to obtain the collision-free region CFR of the current polygon under the current layout i , the formula is as follows:

[0070]

[0071] Step 4, complete the placement of the current polygon by using a heuristic placement strategy.

[0072] The heuristic placement strategy can be a bottom-left (BL) heuristic method, a lowest horizontal line placement strategy, and a four-corner priority placement strategy, and the bottom-left heuristic method BL is preferably used.

[0073] S3. Place the plurality of raw material boards at different coordinate origins of different assembly files, and perform coordinate transformation on each three-dimensional part according to the projection direction to obtain a transformed part 1, and then perform coordinate transformation on the transformed part 1 according to the polygon placement angle and position calculated in S2 to obtain the final placement position of the three-dimensional part in the raw material board, and finally place all the parts in the coordinate system of the corresponding raw material board, and output the three-dimensional assembly body of all the raw material boards after the final nesting is completed.

[0074] The beneficial effects of the present application are:

[0075] 1. The algorithm process proposed in the patent is suitable for thin-walled three-dimensional feature parts in mechanical processing. The three-dimensional projection algorithm and the minimum convex hull algorithm are used to convert the thin-walled three-dimensional feature parts into two-dimensional polygons.

[0076] 2、The algorithm of the patent is suitable for the discharging problem of multiple raw material plates, and different optimization goals can be achieved by setting different reinforcement learning rewards. The reinforcement learning reward function designed in the patent fully considers the total area of the placed parts between multiple raw material plates and the release, and if there is a plate that cannot be fully placed at the end, it can ensure that the remaining area is the largest and continuous.

[0077] 3、Fully consider the influence of different part placement order on the final layout result, it is easier to get the optimal solution. Make full use of the powerful learning ability in the reinforcement learning trial and error process, fully explore the unknown state in the placement process, and it is not easy to fall into local minimum point, further improve the utilization rate of raw material plate.

[0078] 4、It has strong generalization ability. Using deep learning network to learn and predict the action value function of the current state and the selected action, compared with traditional reinforcement learning, it can converge faster. Based on the deep learning model to predict the action score, theoretically, after using enough training data to complete the training, for new problems, even without training, it can give a good layout result. BRIEF DESCRIPTION OF DRAWINGS

[0079] Figure 1 It is the overall flowchart of the algorithm of the application.

[0080] Figure 2 It is a flowchart for generating a two-dimensional inflated polygon from a three-dimensional part.

[0081] Figure 3 It is the Minkowski sum of polygon and circle, and the schematic diagram of generating inflated polygon.

[0082] Figure 4 It is the state transition diagram of reinforcement learning.

[0083] Figure 5 It is the reinforcement learning flowchart for arranging multiple raw material plates.

[0084] Figure 6 It is the score R of the compactness of polygon placement C,t Schematic diagram.

[0085] Figure 7 It is the score R SC,t Schematic diagram.

[0086] Figure 8 It is the structure schematic diagram of Q network and T network. DETAILED DESCRIPTION

[0087] The technical solutions in the embodiments of the application will be described clearly and completely below with reference to the drawings of the application.

[0088] The overall flowchart of the method is as follows Figure 1As shown, the method comprises three steps of generating an inflation polygon, updating a neural network model of the interaction between the agent and the environment in the reinforcement learning, and finally obtaining the placement angle sequence and placement position of the polygon and the three-dimensional part layout, which specifically comprises the following steps:

[0089] S1. Adopt a dimension reduction projection algorithm to project a three-dimensional part into a two-dimensional point set. Three-dimensional thin-walled parts in a factory are usually cut from raw material plates to obtain blank parts. Using a dimension reduction projection algorithm can effectively obtain the main structure profile of a three-dimensional thin-walled part, converting the three-dimensional layout problem into a two-dimensional layout problem. The minimum convex hull algorithm is used to calculate the minimum envelope convex polygon of the two-dimensional point set, which further simplifies the two-dimensional point set into the envelope polygon vertex, greatly reducing the computational amount of the subsequent algorithm. The polygon inflation algorithm is used to calculate the inflation polygon of the minimum envelope convex polygon, leaving sufficient machining allowance for subsequent part machining. The specific process of this process is as follows Figure 2 .

[0090] For three-dimensional parts, it is usually difficult for a machine to determine the cutting direction plane in the raw material plate, so the projection plane is usually given by a designer. The Rodrigues rotation formula is simple and can quickly calculate the rotation matrix when rotating a vector to be collinear with another vector. Therefore, the Rodrigues rotation formula is used to calculate the rotation matrix R of the plane normal vector to the z-axis, and the z-coordinate is compressed to obtain a two-dimensional point set. If the three-dimensional part vertex set is {p(x, y, z)}, the given projection plane is A: ax+by+cz+d=0, and the specific projection steps are as follows:

[0091] Step 1, according to the plane A normal vector n A and the z-axis direction to calculate the rotation axis, that is, v=n A ×z;

[0092] Step 2, calculate the rotation angle of A normal vector n A to the z-axis, that is:

[0093]

[0094] Step 3, calculate the Rodrigues rotation matrix:

[0095] R=I+(1-cos(β))V×V+sin(β)V

[0096] where V is the matrix form of the rotation axis v, that is:

[0097]

[0098] Step 4, traverse the vertex coordinates after rotation, that is, p R (x, y, z)=R×p(x, y, z);

[0099] Step 5, the z coordinate of the vertex after the rotation is removed to obtain a two-dimensional point set P project = {p'(x, y)};

[0100] The projection algorithm of the point cloud format three-dimensional part is similar to the vertex format of the set polygon, but the calculation amount is relatively larger.

[0101] In order to obtain the peripheral contour of the point set after projection, the minimum convex hull algorithm can be used for calculation, which further reduces the calculation amount of the algorithm. The minimum convex hull algorithm can use Bykat algorithm, Akl-Toussaint algorithm, Jarvis matching algorithm and Graham-Andrew scanning algorithm. Among them, the Graham-Andrew scanning algorithm is simple to implement and has smaller calculation amount, so the Graham-Andrew scanning algorithm is preferably used. The specific implementation steps are as follows:

[0102] Step 1, traverse the input polygon vertex set P project Find the point P with the smallest longitudinal coordinate project,miny as the starting point of the convex hull, that is, P ConvexHull [0] = P project,miny ;

[0103] Step 2, sort all the points in P project with P project,miny as the reference point according to the polar angle;

[0104] Step 3, start from P ConvexHull [0] and scan all the points, use vector cross product to determine whether the point P project [t] is inside the convex hull. If yes, it is a convex hull point and is added to the convex hull point set. Otherwise, the next point is traversed.

[0105]

[0106] In actual production, the part blank cut out by the thin wall often needs to leave enough allowance for subsequent part processing. This process can be imagined as a new figure obtained after the center of a fixed radius circle revolves around the polygon contour. As shown in Figure 3 This process can be regarded as the Minkowski sum of the polygon and the fixed radius circle. Therefore, the polygon inflation algorithm in S1 can be calculated by the method of Minkowski sum of the polygon and the fixed radius circle. The Minkowski sum can be calculated by the convolution-based Minkowski vector sum. The specific method is as follows:

[0107] Step 1, sort all the vertices of the polygons M and N to obtain the counterclockwise vertex sequence M = {m i}, N = {n i};

[0108] Step 2, calculate the simplified convolution K of M and N, that is and the line segment set, where m i and n j are convex vertices;

[0109] Step 3, extract non-overlapping, orientable cycles L from K (consider the cycle L as oriented if the normals of all line segments are consistently pointing inwards or outwards), and analyze their nesting relationship for further filtering;

[0110] Step 4, check the intersection between the input polygons placed in the cycle layout and filter out the boundaries, and the final result is the Minkowski vector sum;

[0111] Convert the circle to a polygon, and calculate its Minkowski sum with all convex hulls {P ConvexHull} to obtain the inflated polygon set {P offset}.

[0112] S2. Encode reinforcement learning actions and states, encode all inflated polygons into images, and select a polygon each time as a reinforcement learning action. Encode the layout state of each raw material plate into an image, and stack all raw material plate encoding images to form a reinforcement learning state. Design a state transition function according to the action and state, and design a reinforcement learning reward function R t (a t |s t ) considering the tightness of single raw material plate layout and the overall layout of multiple raw material plates. Build a deep reinforcement learning model: design two identical neural networks according to the reinforcement learning state: value network Q(a t ,s t ) and target network T(a t ,s t ), and use the ε-decay strategy to explore and let the agent interact with the environment.

[0113] Using experience pool multiple data to train the model can avoid the problem of slow convergence speed caused by random errors, and updating the value network and the target network alternately fixes the parameters of a single model, greatly improving the convergence speed,

[0114] At the beginning of the training round, set the total number of reinforcement learning training rounds epoch = 20000, set the current round number epoch_now = 1, initialize the number of current agent and environment interactions n = 1, initialize the environment as all raw material plates {P m} without placing any part polygons, initialize the greedy exploration coefficient ε t = 1, the greedy exploration coefficient decreases by γ ε = 0.9975 during the training process, and the minimum value of the greedy exploration coefficient ε min = 0.1. Set the value network Q(at ,s t ) learning rate lr = 0.005, setting reinforcement learning cumulative reward function discount rate γ = 0.99, setting initialization experience pool pool = {}, setting experience pool maximum data volume batch max = 10000, setting learning sample number batch = 512 each time, setting value network Q update number of rounds epoch1 = 100, setting target network T update frequency epoch2 = 50.

[0115] According to the reinforcement learning value network Q(a t ,s t ), input the current state s t and action a t , output action score Q(a t |s t ), according to the ε-greedy strategy, generate a random number Randnum between 0 and 1, if Randnum ≤ ε t , then random action, otherwise use the action with the highest score, if the total number of raw material plates N m = 4 in the case, the value network Q(a t |s t ) output dimension is N m = 4 dimensions, each dimension represents the predicted score of placing the current action in the corresponding raw material plate, and the highest score dimension is finally selected as the raw material plate index of the polygon placement, that is

[0116] In state s t , the agent selects action a t and raw material plate index index according to the strategy, updates the layout state to enter new state s t+1 , and judges whether the new state is a terminal state, if it is a terminal state, initializes the number of agent and environment interactions and updates the current round number, that is n = 1, epoch_now = epoch_now + 1, then calculates the reward R t (a t |s t ) brought by action a t and state s t , puts state s t+1 , action a t , new state s t and reward R t (a ε |s min ) into the experience pool pool, updates the current number of agent and environment interactions, greedy exploration coefficient and state, n = n + 1, ε = max(γ t ε,ε t+1 ), st = s t+1 .

[0117] When the amount of data in the experience pool reaches batch, the agent randomly takes batch data from the experience pool pool each time the environment interacts, and calculates the predicted score and the actual score U t :

[0118]

[0119] U t =∑ i≤t γ i R(a i |s i ) = R t (a t |s t ) + γmax({max({T(a i ,s t+1 )})})

[0120] Obtain multiple sets of states, actions, predicted scores and actual scores, update the value network Q(a t ,s t ) parameters epoch1 times using gradient descent method, and assign the Q(a t ,s t ) parameters to the T(a t ,s t ) update target network parameters when the iteration round number epoch_now is an integer multiple of epoch2, output the action sequence when the iteration round number epoch_now reaches epoch = 20000 times, and finally generate the placement angle, placement order, placement position and raw material board index of each polygon according to the action sequence. The reinforcement learning state transition and overall learning process are shown in Figure 4 、 Figure 5 ;

[0121] The multi-raw material plate layout problem is solved by using reinforcement deep learning, which can fully explore the unknown area in the search space, find the most suitable polygon layout angle and its raw material plate, and has good generalization ability by using deep learning to develop a strategy function. In theory, after training is completed, it can also solve new layout problems.

[0122] Among them, the encoding of reinforcement learning actions and states in step S2 is as follows:

[0123] The reinforcement learning state encoding takes the current layout of all raw material plates as the reinforcement learning state. If there are N m = 4 identical raw material plates P mIf the size of the raw material plate is (length a = 2000mm, width b = 2000mm), the current layout of each raw material plate is regarded as an image I with w x h pixels s (where w = 500, d = 500), the pixel value I s (c, r) represents whether the pixel is inside the placed polygon, that is:

[0124]

[0125] The current layout image of each raw material plate is described as the reinforcement learning state I s,j Stacked as the final reinforcement learning state, that is The final reinforcement learning state is

[0126] The current layout of multiple raw material plates is regarded as a state, encoded in the form of an image, and stacked as a whole as a reinforcement learning state. The image encoding appropriately ignores features smaller than a single pixel, sufficiently extracts the main features of the state, ensures the accuracy of the model, and improves the convergence speed of the subsequent model training.

[0127] The reinforcement learning action is encoded, and all elements in the inflated polygon {P offset} are taken out and traversed at a rotation angle interval θ to generate an inflated polygon sub-polygon set {P i (α)} by rotating 360°. Each action is to select an element in the inflated polygon sub-polygon set {P i (α)}. The action and the inflated sub-polygon are represented as the following mapping:

[0128] a t (i, k) = P(i, kθ)

[0129] where i, n are integers, and P(i, kθ) is the sub-polygon generated after rotating the i-th polygon in the inflated polygon sub-polygon set {P i (α)} by an angle α = kθ.

[0130] The inflated polygon P i (α) can be regarded as an image I with d x d pixels (d depends on the longest side of the largest part and has the same point distance as the state image, here d = 50) a , and the pixel value I a (c, r) represents whether the pixel point coordinates are inside the polygon. If I a (c, r) = 1, it means that the pixel is inside the polygon. If I a (c, r) = 0, it means that the pixel is not inside the polygon. Finally, the action is encoded as a feature matrix

[0131]

[0132] This way of encoding reinforcement learning actions can unify all polygons in two-dimensional layout problems and has strong generalization.

[0133] Where S2 is the reward function R t (a t |s t ), including R S,t , R C,t , R SC,t and R A,t four parts, where:

[0134] R S,t is the area of the current polygon placed, used to evaluate the size of the placed polygon area after each action, and we subjectively prefer to place the largest polygon first, R S,t The calculation method is as follows:

[0135]

[0136] Where p i,t is the two-dimensional coordinate vertex in the convex hull.

[0137] R C,t is used to evaluate the degree of compactness of the placement, which is composed of two parts: the Minkowski vector sum of the raw material plate and the current polygon, and the Minkowski vector sum of the current layout area and the current polygon. Figure 6 The specific calculation formula is as follows:

[0138]

[0139] R SC,t The reward function is designed for the task and planning goal, and if a raw material plate cannot be placed completely after the final layout in the patent, it is hoped that it occupies a small rectangular area as much as possible. Taking all raw material plates as rectangles as an example, consider this item as the opposite number of the increase in the x-direction area of the raw material plate after placing the polygon each time, such as Figure 7 The calculation method is as follows:

[0140] R SC,t =-d x (max (P t (x))-max ({P i,m (x)})

[0141] Where P i,m (x) is the x-coordinate set of all polygons in the placed raw material plate, and P t (x) is the x-coordinate set of the current placed polygon. This method can punish a part for unreasonable placement angle, which makes the x-direction area of the raw material plate increase too much after the current part is placed.

[0142] R A,t For evaluating the overall layout of the multi-material plate, the part is evaluated and calculated once after placing n3(5 times or more) polygons, and the result of multiplying the standard deviation of the total area of the polygons placed on each material plate by the incremental coefficient is taken as R A,t The specific calculation method is as follows:

[0143]

[0144] Where n is the current interaction times of the reinforcement learning agent and the environment, e is a natural constant, S i is the sum of the areas of all polygons placed on the i-th material plate, S i,m is the area of the i-th material plate. R A,t can ensure that the final layout result has a large variance in the area ratio of each material plate, combined with R SC,t , it tends to leave as much empty space as possible or place polygons as close to one side as possible for material plates that cannot be placed completely.

[0145] The final reward function R t (a t |s t ) has the following calculation formula:

[0146] R t (a t |s t )=w1R S,t +w2R C,t +w3R SC,t +w4R A,t

[0147] Where w1, w2, w3, w4 are the weights of the four scores, and the most important requirement in the placement task is high tightness, so w1, w2, w3, w4 are usually 0.1, 0.5, 0.25, 0.25.

[0148] The construction of the deep reinforcement learning model is described in S2, which adopts a DQN deep reinforcement learning model, where the value network Q(a t ,s t ) and the target network T(a t ,s t ) neural networks are exactly the same, and the design can adopt a convolutional neural network CNN model, with the input being the current state s t and the action a t , and the output being the predicted score and the actual score U t of the multiple material plates. The two networks are exactly the same, and the action graph a t is used to process the state graph s t N mOne convolution and pooling operation is performed on each subgraph to obtain the first hidden layer, and the same parameter layer_n convolution layer and pool layer are used to extract the first hidden layer information, and finally the full_connected_n same parameter layer full connection layer is used to form the final network. The specific structure of the neural network is as follows:

[0149] The action graph H a As a convolution kernel, the state graph H s All subgraphs in the action graph H

[0150] After two times of convolution of hidden1 by using a convolution layer with 64 3*3 convolution kernels and LeakyReLU activation function, a maximum pooling layer with 2*2 and a stride of 2 is used to complete the pooling to obtain the second hidden layer hidden2.

[0151] After two times of convolution of hidden2 by using a convolution layer with 128 3*3 convolution kernels and LeakyReLU activation function, a maximum pooling layer with 2*2 and a stride of 2 is used to complete the pooling to obtain the third hidden layer hidden3.

[0152] After two times of convolution of hidden3 by using a convolution layer with 256 3*3 convolution kernels and LeakyReLU activation function, a maximum pooling layer with 2*2 and a stride of 2 is used to complete the pooling to obtain the fourth hidden layer hidden4.

[0153] After two times of convolution of hidden4 by using a convolution layer with 512 3*3 convolution kernels and LeakyReLU activation function, a maximum pooling layer with 2*2 and a stride of 2 is used to complete the pooling to obtain the fifth hidden layer hidden5.

[0154] After two times of convolution of hidden5 by using a convolution layer with 512 3*3 convolution kernels and LeakyReLU activation function, a maximum pooling layer with 2*2 and a stride of 2 is used to complete the pooling to obtain the fifth hidden layer hidden6.

[0155] A full connection layer with 4096 neurons and LeakyReLU activation function is used to obtain the sixth hidden layer hidden7, a full connection layer with 1024 neurons and LeakyReLU activation function is used to obtain the seventh hidden layer hidden8, and finally a full connection layer with 1 neuron and LeakyReLU activation function is used to obtain the final output. The network structure of the neural network is as shown in Figure 8 .

[0156] In the neural network design, the first layer uses the action graph to perform a convolution on the state graph, fully considering the relationship between the current layout and polygons. The subsequent convolutional layer fully extracts the image features and ultimately improves the total prediction score of the fully connected layer output.

[0157] The specific process of layout update in step S2 is as follows:

[0158] Step 1: Input the state and use the ε-decay strategy to randomly generate or calculate the action a with the highest predicted total score t And place the raw material board index, that is:

[0159]

[0160] Step 2, decoding action a t , get the current placement polygon P(i,k), use Minkowski and calculate the current polygon to be placed P(i,k) and the raw polygon P m The critical inscribed polygon IFP of [index] i , and calculate the circumscribed critical polygon {NFP i,j};

[0161] Step 3: Use the inscribed critical polygon IFP i For all external critical {NFP i,j} Perform a subtraction operation to obtain the collision-free region CFR of the current polygon under the current layout i , the formula is as follows:

[0162]

[0163] Step 4: Use a heuristic placement strategy to complete the placement of the current polygon;

[0164] The heuristic placement strategies may include the bottom-left heuristic method BL, the lowest horizontal line placement strategy, and the four-corner priority placement strategy. The simplest and easiest to implement bottom-left heuristic method BL is preferably used.

[0165] S3. Place multiple raw material plates at the coordinate origins of different assembly files respectively, transform the coordinates of each three-dimensional part in the projection direction to obtain transformed part 1, and then transform the coordinates of transformed part 1 according to the polygon placement angle and position calculated in S2 to obtain the final three-dimensional part position after placement in the raw material plate. Finally, place all parts in the coordinate system of their corresponding raw material plates, and output the three-dimensional assembly layout of all raw material plates after the final layout is completed.

[0166] Finally, it should be noted that the above is only the preferred embodiment of the present application, and is not intended to limit the present application, although the foregoing embodiments of the present application are described in detail, for those skilled in the art, it still can be modified to the technical solution recorded in the foregoing embodiments, or equivalent replacement of some technical features. Any modification, equivalent replacement, improvement, etc. within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A three-dimensional parts layout method for multiple raw material plates, characterized in that: include: S1. Use the dimensionality reduction projection algorithm to project the three-dimensional part into a two-dimensional point set, use the minimum convex hull algorithm to calculate the minimum enveloping convex polygon of the two-dimensional point set, and use the polygon expansion algorithm to calculate the expansion polygon of the minimum enveloping convex polygon; S2. Encode reinforcement learning actions and states. Encode all expanded polygons into images. Each polygon selection is a reinforcement learning action. Encode the layout state of each raw material board into an image, and stack all the raw material board encoding images to form a reinforcement learning state. Design a state transition function based on actions and states. Design a reinforcement learning reward function R by comprehensively considering the density of the single raw material board layout and the overall layout of multiple raw material boards. t (a t |s t ); Build a deep reinforcement learning model: design two identical neural networks based on the reinforcement learning state: the value network Q(a t ,s t ) and target network T(a t ,s t ), and adopt the ε-decay strategy to explore the interaction between the agent and the environment. t The agent chooses action a according to the strategy t After that, the layout is updated to get the new state s t+1 , change the state s t 、Action a t , new state s t+1 and return R t (a t |s t ) into the experience pool. When the amount of data in the experience pool reaches the batch group, each time the agent interacts with the environment, it randomly takes out batch data and calculates the prediction score and the actual score U t The specific calculation formula is as follows: And train and update Q(a t ,s t ) parameter epoch1 round, when the number of iterations reaches epoch2, Q(a t ,s t ) parameter is assigned to T(a t ,s t ) Update the target network parameters, output the action sequence when the number of iterations reaches epoch, and finally generate the placement angle, placement order, placement position and raw material board index of each polygon based on the action sequence; S3. Place multiple raw material plates at the coordinate origins of different assembly files, perform coordinate transformation on each 3D part in the projection direction to obtain transformed part 1, then perform coordinate transformation on transformed part 1 according to the polygon placement angle and position calculated in S2 to obtain the final 3D part position in the raw material plate. Finally, place all parts in the coordinate system of their corresponding raw material plates, and output the final 3D assembly with all raw material plates arranged after final nesting. Step S2 of "encoding reinforcement learning actions and states" includes: Reinforcement learning state encoding, taking the current layout of all raw material boards as reinforcement learning state, if there are N m Raw material board P m , then the current layout of each raw material board is regarded as an image I with pixels w×h s , pixel value I s (c, r) represents whether the pixel is inside the placed polygon, that is: Reinforce the learning state I by describing the current layout image of each raw material board s,j Stacked up as the final reinforcement learning state, that is The final reinforcement learning state is encoded as Reinforce learning action encoding, extract the expansion polygon {P offset } and rotate 360° to generate the expanded polygon sub-polygon set {P i (α)}, each action selects the expanded polygon sub-polygon set {P i (α)}, the action and the dilation subpolygon are represented as the following mapping: the t (i,k)=P(i,kθ) Where i and n are integers, and P(i,kθ) is the set of expanded polygon sub-polygons {P i The sub-polygon generated by rotating the i-th polygon in (α)} by angle α = kθ; The expanded polygon P i (α) is considered as an image I with d×d pixels. a , pixel value I a (c, r) corresponds to whether the pixel coordinates are within the polygon. If I a (c, r) = 1 means that the pixel is inside the polygon. a If (c, r) = 0, the pixel is not within the polygon, and the action is finally encoded as a feature matrix. Right now:

2. A three-dimensional parts layout method for multiple raw material plates according to claim 1, characterized in that: The formats of the three-dimensional part in step S1 include point cloud format and polygon vertex set format. The dimensionality reduction projection algorithm uses the Rodriguez rotation formula to calculate the rotation matrix R of the plane normal vector to the z-axis of the coordinate system, and compresses the z coordinate to obtain a two-dimensional point set. If the set of all vertices of the three-dimensional part is {p(x, y, z)}, the given projection plane is A: ax+by+cz+d=0, and the specific projection steps are as follows: Step 1, according to the normal vector n of plane A A Calculate the rotation axis in the z-axis direction, that is, v = n A ×z; Step 2, calculate the normal vector n of A A The rotation angle about the z axis is: Step 3, calculate the Rodriguez rotation matrix: R=I+(1-cos(β))V×V+sin(β)V Where V is the matrix form of the rotation axis v, namely: Step 4, traverse and calculate the coordinates of the rotated vertices, i.e. p R (x, y, z) = R × p(x, y, z); Step 5: Eliminate the z coordinates of the rotated vertices to obtain the two-dimensional point set P project ={p′(x,y)}; The projection algorithm for 3D parts in point cloud format is similar to that in polygon vertex set format, except that the amount of calculation is relatively larger.

3. A three-dimensional parts layout method for multiple raw material plates according to claim 1, characterized in that The minimum convex hull algorithm calculation described in step S1 adopts the Graham-Andrew scanning algorithm, and the specific implementation steps are as follows: Step 1: traverse the input polygon vertex set P project Find the point P with the smallest ordinate project,miny As the starting point of the convex hull, that is, P ConvexHull [0] = P project,miny ; Step 2: P project All other points in the project,miny Sort by polar angle for benchmark; Step 3, from P ConvexHull [0] Start scanning all points and use vector cross product to determine point P project Is [t] inside the convex hull? If so, it is a convex hull point and is added to the convex hull point set. Otherwise, traverse to the next point, i.e.:

4. A three-dimensional parts layout method for multiple raw material plates according to claim 1, characterized in that The polygon expansion algorithm described in step S1 can use the method of finding the Minkowski sum of the polygon and the fixed radius circle. The calculation can be done by Minkowski vector sum calculation based on convolution, the specific method is as follows: Step 1: Sort all vertices of polygons M and N to obtain a counterclockwise vertex sequence M = {m i }、N={n i }; Step 2, calculate the simplified convolution K of M and N, that is and A set of line segments, where m i With n j is a convex vertex; Step 3: Extract non-overlapping, orientable loops L from K and analyze their nested relationships for further filtering. If the normals of all line segments are consistently facing inward or outward, the loop L is considered orientable. Step 4: Check the intersections between the input polygons placed in the loop layout to filter out the boundaries. The final result is the Minkowski vector sum. Convert the circle to a polygon and calculate its convex hull with all the polygons. ConvexHull }'s Minkowski sum to obtain the expanded polygon set {P offset }.

5. The method for arranging three-dimensional parts from multiple raw material plates according to claim 1, characterized in that The reward function R in step S2 t (a t |s t ), including R S,t 、R C,t 、R SC,t and R A,t Four parts, including: R S,t The area of ​​the current polygon; R C,t Used to evaluate the compactness of placement. This score is composed of the Minkowski vector sum of the raw material board and the expanded polygon of the current polygon, and the Minkowski vector sum of the current layout area and the expanded polygon of the current polygon. The specific calculation formula is as follows: R SC,t The reward function design for the task and planning goals is based on the assumption that all raw material plates are rectangles. This term is the inverse of the increase in the x-direction area of ​​the raw material plate after each polygon is placed. The calculation method is as follows: R SC,t =-d×(max(P t (x))-max({P i,m (x)})) Among them, P i,m (x) is the x-coordinate set of all polygons in the raw material board, P t (x) is the x-coordinate set of the current placed polygon; R A,t Used to evaluate the overall layout of multiple raw material boards. This part is evaluated and calculated once every n3 polygons are placed, where n3 is not less than 5. The result of multiplying the standard deviation of the total area of ​​each raw material board by the increasing coefficient is used as R A,t , the specific calculation method is as follows: Where n is the number of interactions between the reinforcement learning agent and the environment, e is a natural constant, and S i is the sum of the areas of all polygons placed on the i-th raw material board, S i,m is the area of ​​the i-th raw material plate; Final reward function R t (a t |s t ) is calculated as follows: R t (a t |s t )=w1R S,t +w2R C,t +w3R SC,t +w4R A,t Among them, w1, w2, w3, and w4 are the weights of the four scores.

6. A three-dimensional parts layout method for multiple raw material plates according to claim 1, characterized in that Step S2 constructs a deep reinforcement learning model, using the DQN deep reinforcement learning model, where the value network Q(a t ,s t ) and target network T(a t ,s t ) neural network, exactly the same, the design can use the convolutional neural network CNN model, the input is the current state s t and action a t , the output is the prediction score and the actual score U t , these two networks are exactly the same, first use action graph a t For the state diagram s t N in m A convolution and pooling operation is performed on each subgraph to obtain the first hidden layer. Subsequently, layer_n convolution layers and pooling layers with the same parameters are used to extract the first hidden layer information. Finally, full_connected_n fully connected layers with the same parameters are used to form the final network.

7. The method for arranging three-dimensional parts from multiple raw material plates according to claim 1, characterized in that The specific process of "layout update" in step S2 is as follows: Step 1: Input the state and use the ε-decay strategy to randomly generate or calculate the action a with the highest predicted total score t And place the raw material board index, that is: Step 2, decoding action a t , get the current placement polygon P(i,k), use Minkowski and calculate the current polygon to be placed P(i,k) and the raw polygon P m The critical inscribed polygon IFP of [index] i , and calculate the circumscribed critical polygon {NFP i,j }; Step 3: Use the inscribed critical polygon IFP i For all external critical {NFP i,j } Perform a subtraction operation to obtain the collision-free region CFR of the current polygon under the current layout i , the formula is as follows: Step 4: Use a heuristic placement strategy to complete the placement of the current polygon; The heuristic placement strategy adopts the lower left corner heuristic method BL.

8. The method for three-dimensional parts layout using multiple raw material plates according to claim 3, wherein the "minimum convex hull algorithm" can also use the Bykat algorithm, the Akl-Toussaint algorithm, and the Jarvis matching algorithm.

9. A three-dimensional parts layout method for multiple raw material plates according to claim 7, characterized in that The "heuristic placement strategy" described in step 4 of "Layout Update" can also use the lowest horizontal line placement strategy and the four-corner priority placement strategy.

Images