Three-dimensional thin-walled parts layout algorithm based on unilateral infinite extension raw material plate

Abstract

The present invention discloses a three-dimensional thin-walled parts layout algorithm for unilateral infinite extension raw material plates based on reinforcement learning. The present invention comprises: using a three-dimensional projection algorithm, a minimum convex hull algorithm and a Minkowski vector sum algorithm to generate a two-dimensional expansion polygon; the polygon is encoded as a generalized graph, the selected graph is an action, the current layout is encoded as a generalized graph as a state, the placement area and compactness are used as action rewards, and deep reinforcement learning is used to iteratively obtain an action sequence, and finally the placement order, angle and position of each polygon are obtained. The present invention can solve the layout problem of thin-walled three-dimensional feature parts in mechanical processing; reinforcement learning fully explores the unknown states of the placement process, fully considers the influence of the parts arrangement order on the final result, and can quickly give a better layout method; in addition, the use of deep learning to fit the reinforcement learning value function has a strong generalization ability, and for a completely new layout problem, better layout results can be obtained without retraining the network parameters.

Description

Technical Field

[0001] The present invention relates to the two-dimensional blanking problem and optimization field of single-sided infinitely long raw material plates, and specifically to a method for obtaining a two-dimensional point cloud set of three-dimensional parts using a projection algorithm, and obtaining the placement order, placement angle, and placement position of the two-dimensional parts using iterative learning using reinforcement learning. Background Art

[0002] In modern manufacturing, the 3D sheet metal nesting problem is a critical engineering optimization task. It involves efficiently arranging and cutting parts of desired shapes using limited sheet metal resources. The complexity of this problem stems from the 3D nature of sheet metal and the numerous constraints that must be considered during the manufacturing process, such as minimizing raw material waste, reducing production costs, and improving production efficiency. Therefore, solving the 3D sheet metal nesting problem has important theoretical and practical implications for promoting the digital transformation of the manufacturing industry and improving resource utilization efficiency.

[0003] Traditional sheet metal layout problems primarily focus on one-dimensional linear layout and two-dimensional planar layout. However, with advancements in manufacturing technology and the increasing complexity of product structures, the demand for three-dimensional sheet metal is increasing. In fields such as aerospace, automotive manufacturing, and construction, many parts have complex three-dimensional shapes and structures. Therefore, traditional two-dimensional layout methods are no longer able to meet the needs of actual production.

[0004] Among the patents for irregular three-dimensional thin plate parts layout algorithms, there are: a knowledge-driven two-dimensional polygonal automatic layout method invented by Feng Yixiong et al. This method traverses all angles of all polygons, calculates the angle and position corresponding to the maximum fitness of each part, and then arranges them. This method does not consider the impact of the polygon placement order on the layout results during the actual layout process, and the huge amount of calculation makes it difficult to use in medium- and large-scale projects. In addition, this method only considers two-dimensional layout, which is difficult to use in three-dimensional parts layout projects; a two-dimensional rectangular parts layout method based on an improved simulated annealing algorithm invented by Bo Leilei et al. This method can reduce the number of plates used, improve plate utilization, significantly reduce the layout plan, and reduce the material and labor costs of the product. However, this method has a large amount of calculation and is difficult to converge slowly, and has poor generalization ability. In addition, existing patents have given little consideration to the problem of two-dimensional square single-sided infinite plate layout, and there is no algorithm that can minimize the square area occupied by the final layout after the polygons are placed. Summary of the Invention

[0005] To address the above problems, this patent proposes an irregular three-dimensional thin plate parts layout algorithm based on reinforcement learning. Through the projection algorithm and the two-dimensional convex hull algorithm, the three-dimensional parts are compressed and projected into two-dimensional polygons in a certain direction. The current layout is encoded as the reinforcement learning state, and the sub-polygon set after all polygons are rotated at a fixed angle is used as the action. The reinforcement learning model is constructed and iteratively optimized to obtain the layout result.

[0006] This patent invents a three-dimensional thin-walled parts layout algorithm for a single-sided infinitely extended raw material plate, which includes the following steps:

[0007] S1. Project the 3D part onto a 2D plane using a 3D projection algorithm. In actual production, 3D thin-walled parts are typically cut to obtain a blank. A minimum convex hull algorithm is used to calculate the minimum enveloping polygon of the projected point set on the 2D plane. A polygon expansion algorithm is used to calculate the expanded polygon of the minimum enveloping polygon, leaving a machining allowance for subsequent finishing in actual production.

[0008] The three-dimensional part formats of the projection algorithm in S1 include point cloud format and polygon vertex set format. For the polygon vertex set format part, step S1 adopts the PCA dimensionality reduction projection method. The PCA algorithm can ensure the maximum variance of the projected graphic point set, thereby ensuring that the contour of the thin-walled part after dimensionality reduction projection is the main shape contour. The PCA algorithm specifically includes the following steps:

[0009] Step 1.1: Read the polygon vertex set from the 3D part ;

[0010] Step 1.2, calculate the center point coordinates ;

[0011] Step 1.3, Centralize the Vertex Set Dataset ,get ;

[0012] Step 1.4, calculate the covariance matrix ;

[0013] Step 1.5, calculate the eigenvalues ​​of the covariance matrix and eigenvectors , and sort the eigenvalues, select the eigenvectors corresponding to the two largest eigenvalues ​​as the projection basis, and finally calculate the two-dimensional point set after dimensionality reduction ;

[0014] The processing method for parts in point cloud format is the same as that in polygon vertex collection format, but the computational complexity is relatively higher. In actual production, 3D thin-walled parts are usually obtained by cutting the blank parts. Therefore, the beneficial effect of using a 3D projection algorithm is that it can effectively obtain the main structural outline of the 3D thin-walled parts, convert the 3D nesting problem into a 2D nesting problem, eliminate interference and reduce the overall computational complexity of the algorithm.

[0015] The minimum convex hull algorithm in S1 uses the most easily implemented Graham-Andrew scanning algorithm. The implementation steps are as follows:

[0016] Step 2.1, traverse the input polygon vertex set , find the point with the smallest vertical coordinate As the starting point of the convex hull, that is ;

[0017] Step 2.2, All other points in Sort by polar angle for benchmark;

[0018] Step 2.3, from Start scanning all points and use vector cross product to determine the points Is it inside the convex hull? If so, it is a convex hull point and is added to the convex hull point set. Otherwise, traverse to the next point.

[0019]

[0020] In addition, the Bykat algorithm, Akl-Toussaint algorithm and Jarvis matching algorithm can be used to calculate the minimum convex hull of the target polygon. The beneficial effect of calculating the convex polygon is that it can further reduce the number of contour polygon points, greatly reducing the computational complexity of the subsequent packing algorithm.

[0021] The polygon expansion algorithm in S1 uses the method of finding the Minkowski sum of the polygon and the fixed radius circle. , the calculation adopts the convolution-based Minkowski vector and calculation, the specific method is as follows:

[0022] Step 3.1, sort all vertices of polygons M and N to get a counterclockwise vertex sequence 、 ;

[0023] Step 3.2, calculate the simplified convolution K of M and N, that is and A set of line segments, where and is a convex vertex;

[0024] Step 3.3: Extract non-overlapping, orientable loops L from K (if the normals of all line segments are consistently facing inward or outward, the loop L is considered orientable), and analyze their nesting relationships for further filtering;

[0025] Step 3.4, check the intersections between the input polygons placed in the loop layout, filter out the boundaries, and the final result is the Minkowski vector sum;

[0026] Convert the circle to a polygon and calculate its convex hull with all polygons respectively. Minkowski sum of the dilated polygon set .

[0027] In actual production, it is often necessary to leave sufficient machining allowance for cutting blanks. Therefore, calculating the expansion polygon of a convex polygon is to fully consider the subsequent machining allowance and set the expansion circle radius according to the allowance size.

[0028] S2. Encode reinforcement learning actions and states, design state transition functions based on actions and states, and set reinforcement learning reward functions , build a deep reinforcement learning model: design two identical neural networks based on the reinforcement learning state: value network and target network , and adopt - Decay strategy exploration allows the agent to interact with the environment and The agent chooses actions according to the strategy After that, update the layout to get the new state , and according to the action and status Calculate the reward of this action , the state ,action , return and the new state Put it into the experience pool, and calculate the prediction score when the data in the experience pool reaches batch1 and actual score :

[0029]

[0030]

[0031] , and train the updated value network Parameter epoch1, when the total number of rounds reaches epoch2 Parameter assignment Update the target network parameters, output the action sequence when the number of rounds reaches the total number of epochs, and finally generate the placement angle, placement order and placement position of each polygon based on the action sequence;

[0032] The beneficial effects of using enhanced deep learning to solve the nesting problem are: it can fully explore the unknown areas in the search space, and the strategy function formulated by deep learning has good generalization ability. In theory, it can also solve new nesting problems after training is completed.

[0033] The specific method of encoding reinforcement learning actions and states in S2 is as follows:

[0034] Reinforcement learning action encoding, extracting the expanded polygon All elements in the array are spaced by rotation angle , traverse the rotation Generates a set of inflated polygon sub-polygons Each action selects the expanded polygon sub-polygon set One element in , the action and dilation subpolygon are represented as the following mapping:

[0035]

[0036] Where i and n are integers. The set of polygon sub-polygons for the inflated polygon The rotation angle of the i-th polygon The sub-polygon generated later;

[0037] The polygon corresponding to the action is regarded as a pixel Image (d cannot be so small that the two vertices of the action image collapse into one pixel), the pixel value Whether there is a polygon vertex within the corresponding pixel range, if Indicates that the pixel is a polygon vertex and the feature matrix is ​​designed and set As the final encoded action, the first feature indicates whether the pixel value is a vertex, and the second and third features are the exact values ​​of the vertex, namely:

[0038]

[0039] For polygon vertices, the final action is encoded as: ;

[0040] Reinforcement learning state encoding, taking the current layout of the raw material board as the reinforcement learning state, and treating the layout as pixels to further extract features Image (image and The actual distance represented by each pixel is the same, and each time only the rightmost vertex of the actual layout is drawn to a certain area to the left as the image ), pixel value Indicates whether there is a polygon vertex within the pixel range and sets the feature matrix Further description of the status characteristics:

[0041] Feature Matrix Use the number of polygon vertices in the current pixel , the exact x-coordinate and the exact y-coordinate of the vertex describe the state of the polygon vertex. The specific calculation method is as follows:

[0042]

[0043] For the polygon vertices in the raw material board layout, the state encoding feature matrix is ​​finally .

[0044] The beneficial effect of using a generalized image approach to encode actions and states is that it can unify all two-dimensional packing problems without losing excessive information features due to the limited number of pixels in the image. To a certain extent, it ensures that the strategy learned by the final reinforcement learning agent is applicable to the new packing problem.

[0045] The reward function in S2 ,include and These three parts include:

[0046] The current polygon area is used to evaluate the size of the polygon area after each action. Subjectively, we prefer to place the largest polygon possible first. The calculation method is as follows:

[0047]

[0048] in are the two-dimensional coordinate vertices in the convex hull.

[0049] Used to evaluate the compactness of placement. This score is composed of the Minkowski vector sum of the raw material board and the expanded polygon of the current polygon, and the Minkowski vector sum of the current layout area and the expanded polygon of the current polygon. The specific calculation formula is as follows:

[0050]

[0051] It is a reward function designed for tasks and planning goals. In this patent, we consider that one side is infinitely long. If the y direction of the raw material plate is fixed, the x direction can be infinitely extended. We hope that the distance between the rightmost vertex of the polygon and the origin in the final layout result is as short as possible. Based on this optimization goal, The design is the inverse of the product of the length added in the x direction after the current polygon is placed and the board width d. The formula is as follows:

[0052]

[0053] Final reward function The calculation formula is as follows:

[0054]

[0055] are the weights of the three scores.

[0056] The value network described in S2 and target network Neural network, exactly the same, using convolutional neural network CNN model, the input is the current state and actions , the output is the prediction score and actual score , these two networks are exactly the same, first use the action graph State diagram Perform a convolution and pooling operation, and then Layer convolution layer plus pooling layer and The final network is composed of layers of fully connected layers. The neural network design method fully considers the characteristics of action graphs and state graphs.

[0057] In the neural network design, the first layer uses the action graph to perform a convolution on the state graph, fully considering the relationship between the current layout and polygons. The subsequent convolutional layer fully extracts the image features and ultimately improves the total prediction score of the fully connected layer output.

[0058] Furthermore, the three-dimensional thin-walled parts layout algorithm of the raw material plate with unilateral infinite extension is characterized in that the specific process of the "layout update" in step S2 is as follows:

[0059] Step 4.1, decoding action , get the current placement polygon , use Minkowski and calculate the current polygon to be placed With raw polygons The critical inscribed polygon of , and calculate the current polygon to be placed The circumscribed critical polygon of all placed polygons in the current layout .

[0060] Step 4.2, use the inscribed critical polygon For all external critical Perform a subtraction operation to obtain the collision-free area of ​​the current polygon under the current layout , the formula is as follows.

[0061]

[0062] Step 4.3, use the heuristic placement strategy to complete the placement of the current polygon.

[0063] Among them, the heuristic placement strategy adopts the lowest horizontal line placement strategy.

[0064] S3. Place the raw material plate at the coordinate origin and transform each 3D part in the projection direction to obtain transformed part 1. Then, according to the polygon placement angle and position calculated in S2, transform the transformed part 1 to obtain the final 3D part placed on the raw material plate. Finally, place all parts in a raw material plate coordinate system and output the final 3D assembly layout after layout completion.

[0065] The beneficial effects of the present invention are:

[0066] 1. The algorithm flow proposed in this patent is applicable to thin-walled three-dimensional feature parts in mechanical processing. It uses a three-dimensional projection algorithm and a minimum convex hull algorithm to convert thin-walled three-dimensional feature parts into two-dimensional polygons.

[0067] 2. The algorithm proposed in this patent can solve the problem of arranging square raw material plates with infinite one side, and achieve the minimum square area occupied by the final layout.

[0068] 3. Fully consider the impact of different parts placement orders on the final layout results, making it easier to obtain the optimal solution. Leveraging the powerful learning capabilities of reinforcement learning during trial and error, we fully explore the unknown states of the placement process, making it less likely to fall into local minimums, and further improving the utilization rate of raw material boards.

[0069] 4. Strong generalization ability. Using a deep learning network to learn the action-value function that predicts the current state and the selected action, it can converge faster than traditional reinforcement learning. In theory, after training with sufficient training data, it can produce good sorting results for new problems even without further training. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] Figure 1 The figure is a simplified diagram of the overall process of the algorithm of the present invention.

[0071] Figure 2 Generate a flow chart for converting 3D parts to 2D expanded polygons.

[0072] Figure 3 It is the Minkowski sum of the polygon and the circle, generating a diagram of the expanded polygon.

[0073] Figure 4 It is the state transition diagram of reinforcement learning.

[0074] Figure 5 Arrange reinforcement learning flowchart for polygons.

[0075] Figure 6 Score how compactly polygons are arranged Schematic diagram.

[0076] Figure 7 for Scoring diagram.

[0077] Figure 8 Schematic diagram of the Q network and T network structure. DETAILED DESCRIPTION

[0078] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the accompanying drawings of the present invention.

[0079] The overall process of this method is as follows Figure 1 As shown in the figure, it includes three steps: generating inflated polygons, acquiring data from the interaction between the agent and the environment in reinforcement learning to update the neural network model, and finally obtaining the placement angle sequence and placement position of the polygons and the three-dimensional part layout. Specifically, it includes the following steps:

[0080] S1. Use the 3D projection algorithm to project the 3D part onto a 2D plane. Use the minimum convex hull algorithm to calculate the minimum enveloping polygon of the 2D plane projection point set. Use the polygon expansion algorithm to calculate the expanded polygon of the minimum enveloping polygon. The specific flow chart is as follows: Figure 2 ;

[0081] For thin-walled 3D parts, it is difficult for the machine to determine the correct projection direction based on the data. To enable the machine to find the correct direction for thin-wall projection, the projection direction is defined as the direction with the largest variance of the projection points after the 3D part vertices are projected. Therefore, PCA is used for dimensionality reduction projection. The 3D part formats in the projection algorithm S1 include point cloud format and polygon vertex collection format. For polygon vertex collection format parts, step S1 uses the PCA dimensionality reduction projection method, which specifically includes the following steps:

[0082] Step 1.1: Read the polygon vertex set from the 3D part ;

[0083] Step 1.2, calculate the center point coordinates ;

[0084] Step 1.3, Centralize the Vertex Set Dataset ,get ;

[0085] Step 1.4, calculate the covariance matrix ;

[0086] Step 1.5, calculate the eigenvalues ​​of the covariance matrix and eigenvectors , and sort the eigenvalues, select the eigenvectors corresponding to the two largest eigenvalues ​​as the projection basis, and finally calculate the two-dimensional point set after dimensionality reduction ;

[0087] The method for processing parts in point cloud format is the same as that in polygon vertex collection format, except that the amount of calculation is relatively larger.

[0088] In order to obtain the contour of a two-dimensional point set and reduce the amount of subsequent calculations, it is necessary to first calculate the minimum convex hull of the point set. The Graham-Andrew scanning algorithm, which is the easiest to implement, is preferably used for the minimum convex hull algorithm in S1. The specific implementation steps are as follows:

[0089] Step 2.1, traverse the input polygon vertex set , find the point with the smallest vertical coordinate As the starting point of the convex hull, that is ;

[0090] Step 2.2, All other points in Sort by polar angle for benchmark;

[0091] Step 2.3, from Start scanning all points and use vector cross product to determine the points Is it inside the convex hull? If so, it is a convex hull point and is added to the convex hull point set. Otherwise, traverse to the next point.

[0092]

[0093] In actual production, the blanks of parts cut out from thin walls often need to leave enough margin for subsequent part processing. This process can be imagined as a new shape obtained after the center of a circle with a fixed radius circles around the polygonal contour, such as Figure 3 As shown, the process can be regarded as the Minkowski sum of a polygon and a fixed radius circle, so the polygon expansion algorithm in S1 adopts the method of finding the Minkowski sum of a polygon and a fixed radius circle. , the calculation adopts the convolution-based Minkowski vector and calculation, the specific method is as follows:

[0094] Step 3.1, sort all vertices of polygons M and N to get a counterclockwise vertex sequence 、 ;

[0095] Step 3.2, calculate the simplified convolution K of M and N, that is and A set of line segments, where and is a convex vertex;

[0096] Step 3.3: Extract non-overlapping, orientable loops L from K (if the normals of all line segments are consistently facing inward or outward, the loop L is considered orientable), and analyze their nesting relationships for further filtering;

[0097] Step 3.4, check the intersections between the input polygons placed in the loop layout, filter out the boundaries, and the final result is the Minkowski vector sum;

[0098] Convert the circle to a polygon and calculate its convex hull with all polygons respectively. The Minkowski sum of the dilated polygonal set is obtained .

[0099] S2. Encode reinforcement learning actions and states, design state transition functions based on actions and states, and set reinforcement learning reward functions , build a deep reinforcement learning model: design two identical neural networks based on the reinforcement learning state: value network and target network , and adopt - Decay strategy exploration allows the agent to interact with the environment. Using reinforcement deep learning to solve the nesting problem allows for full exploration of unknown regions in the search space. Furthermore, the deep learning approach to developing a policy function has excellent generalization capabilities, theoretically enabling the solution of new nesting problems even after training.

[0100] At the beginning of the training round, set the total number of reinforcement learning training rounds epoch = 10000, set the current round number epoch_now = 1, initialize the number of interactions between the current agent and the environment n = 1, and initialize the environment to a raw material plate without any part polygons placed , initialize the greedy exploration coefficient During the training process, the greedy exploration coefficient decreases at a rate of , greedy exploration coefficient minimum . Setting up the value network Learning rate , set the discount rate of the reinforcement learning cumulative reward function , set the initial experience pool , set the maximum amount of data in the experience pool to bacth max=20000, set the value network Q update sampling number batch=512, the number of training times epoch1=100, and the target network T update frequency epoch2=50.

[0101] According to the reinforcement learning value network , enter the current state and actions , output action score ,according to Strategy, generate Random number ,like Then take random action, otherwise take the action with the largest score, that is , in state The agent chooses actions according to the strategy After that, update the layout state to enter the new state , and determine whether the new state is the terminal state. If it is the terminal state, initialize the number of interactions between the agent and the environment and update the current round number, that is, n=1, epoch_now=epoch_now+1, and then according to the action and status Calculate the reward of this action , the state ,action , new status and returns Put it into the experience pool, update the number of interactions between the current agent and the environment, greedy exploration coefficient and state, 、 、 .

[0102] When the amount of data in the experience pool reaches the batch, randomly take out the experience pool Batch group data and calculate the prediction score and actual score :

[0103]

[0104]

[0105] Obtain multiple sets of states, predicted scores, and actual scores, and use gradient descent to update the value network Parameter epoch1, when the current iteration number epoch_now can divide T network update frequency epoch2 Parameter assignment Update the target network parameters, and output the action sequence when the number of iterations epoch_now reaches epoch=10000. Finally, generate the placement angle, placement order, and placement position of each polygon based on the action sequence. The reinforcement learning state transition and the overall learning process are as follows: Figure 4 、 Figure 5 As shown;

[0106] Using multiple sets of data from the experience pool to train the model can avoid the problem of full convergence caused by random errors. Alternating updates of the value network and the target network fix the parameters of a single model and greatly improve the convergence speed.

[0107] The specific method of encoding reinforcement learning actions and states in S2 is as follows:

[0108] Reinforcement learning action encoding, extracting the expanded polygon All elements in the array are spaced by rotation angle , traverse the rotation Generates a set of inflated polygon sub-polygons Each action selects the expanded polygon sub-polygon set One element in , the action and dilation subpolygon are represented as the following mapping:

[0109]

[0110] Where i and n are integers. The set of polygon sub-polygons for the inflated polygon The rotation angle of the i-th polygon The sub-polygons generated later can include all possible polygons that may appear during the placement process by encoding reinforcement learning actions in this way.

[0111] The polygon corresponding to the action is regarded as a pixel (where d cannot collapse the vertex of the minimum polygon into a single pixel point, ) , pixel value Whether there is a polygon vertex within the corresponding pixel range, if Indicates that the pixel is a polygon vertex and the feature matrix is ​​designed and set As the final encoded action, the first feature indicates whether the pixel value is a vertex, and the second and third features are the exact values ​​of the vertex, namely:

[0112]

[0113] For polygon vertices, the final action is encoded as: ;

[0114] Reinforcement learning state encoding, taking the current layout of the raw material board as the reinforcement learning state, and treating the layout as pixels to further extract features Image , if the height of the raw material plate is , the width is unlimited, then the image is designed based on the action pixels The specific calculation formula is:

[0115]

[0116] in is the ratio of the actual distance to pixels of the action coding image.

[0117] Since the raw material board has unlimited width, each time the image is generated When , only the left vertex of the rightmost vertex of the current layout is taken area, so the image Height is , if motion picture coding has been designed, and , then the image Width and Height .

[0118] Pixel value Indicates whether there is a polygon vertex within the pixel range and sets the feature matrix Further description of the status characteristics:

[0119] Feature Matrix Use the number of polygon vertices in the current pixel , the exact x-coordinate and the exact y-coordinate of the vertex describe the state of the polygon vertex. The specific calculation method is as follows:

[0120]

[0121] For the polygon vertices in the raw material board layout, the state encoding feature matrix is ​​finally Encoding reinforcement learning states in this way is simple and can include all states that appear in the nesting process. It has good generalization and different nesting problems (raw materials, parts) can use the same encoding method.

[0122] The reward function in S2 ,include and These three parts include:

[0123] The current polygon area is used to evaluate the size of the polygon area after each action. Subjectively, we prefer to place the largest polygon possible first. The calculation method is as follows:

[0124]

[0125] in are the two-dimensional coordinate vertices in the convex hull.

[0126] Used to evaluate the compactness of placement. The score is composed of the Minkowski vector sum of the raw material board and the expanded polygon of the current polygon, and the Minkowski vector sum of the current layout area and the expanded polygon of the current polygon. Figure 6 , the specific calculation formula is as follows:

[0127]

[0128] It is a reward function designed for tasks and planning goals. In this patent, we consider that one side is infinitely long. If the y direction of the raw material plate is fixed, the x direction can be infinitely extended. We hope that the distance between the rightmost vertex of the polygon and the origin in the final layout result is as short as possible. Based on this optimization goal, The design is the inverse of the product of the length added in the x direction after the current polygon is placed and the board width d. Figure 7 , The calculation formula is as follows:

[0129]

[0130] Final reward function The calculation formula is as follows:

[0131]

[0132] is the weight of the three scores, take If they are 0.4, 0.4, and 0.2 respectively, the reward function is finally: .

[0133] The value network described in S2 and target network Neural network, exactly the same, using convolutional neural network CNN model, the input is the current state Corresponding image and actions Corresponding image , the output is the prediction score and actual score , these two networks are exactly the same, and the specific structure of the neural network is as follows:

[0134] First, the action diagram As a convolution kernel, the state diagram Perform a convolution to get the first hidden layer ;

[0135] Using 64 Convolution kernel, Convolutional layer pair with activation function After completing the convolution twice, use , the maximum pooling layer with a stride of 2 completes the pooling to obtain the second hidden layer ;

[0136] Using 128 Convolution kernel, Convolutional layer pair with activation function After completing the convolution twice, use , the maximum pooling layer with a stride of 2 completes the pooling to obtain the third hidden layer ;

[0137] Using 256 Convolution kernel, Convolutional layer pair with activation function After completing the convolution twice, use , the maximum pooling layer with a stride of 2 completes the pooling to obtain the fourth hidden layer ;

[0138] Using 512 Convolution kernel, Convolutional layer pair with activation function After completing the convolution twice, use , the maximum pooling layer with a stride of 2 completes the pooling to obtain the fifth hidden layer ;

[0139] Using 512 Convolution kernel, Convolutional layer pair with activation function After completing the convolution twice, use , the maximum pooling layer with a stride of 2 completes the pooling to obtain the fifth hidden layer ;

[0140] Using 4096 neurons The fully connected layer of the activation function gets the sixth hidden layer , using 1024 neurons The fully connected layer of the activation function gets the seventh hidden layer , and finally use 1 neuron The fully connected layer of the activation function gets the final output. The neural network structure is as follows Figure 8 shown.

[0141] In the neural network design, the first layer uses the action graph to perform a convolution on the state graph, fully considering the relationship between the current layout and polygons. The subsequent convolutional layer fully extracts the image features and ultimately improves the total prediction score of the fully connected layer output.

[0142] Furthermore, the three-dimensional thin-walled parts layout algorithm of the raw material plate with unilateral infinite extension is characterized in that the specific process of the "layout update" in step S2 is as follows:

[0143] Step 4.1, decoding action , get the current placement polygon , use Minkowski and calculate the current polygon to be placed With raw polygons The critical inscribed polygon of , and calculate the current polygon to be placed The circumscribed critical polygon of all placed polygons in the current layout .

[0144] Step 4.2, use the inscribed critical polygon For all external critical Perform a subtraction operation to obtain the collision-free area of ​​the current polygon under the current layout , the formula is as follows.

[0145]

[0146] Step 4.3, use the heuristic placement strategy to complete the placement of the current polygon.

[0147] The heuristic placement strategy adopts the lowest horizontal line placement strategy in the x-direction. Using this strategy can further limit the expansion of parts in the x-direction, making the parts more compact in the x-direction.

[0148] S3. Place the raw material plate at the coordinate origin and transform each 3D part in the projection direction to obtain the transformed part. , the specific calculation formula is as follows:

[0149]

[0150] Then according to the polygon placement angle calculated in S2 and location , for the transformation parts Perform coordinate transformation to obtain the final three-dimensional parts placed on the raw material plate. The calculation method is as follows:

[0151]

[0152] Finally, all parts are placed in a raw material board coordinate system, and the final layout of the three-dimensional assembly is output.

[0153] Finally, it should be noted that the above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art will be able to modify the technical solutions described in the aforementioned embodiments or substitute equivalents for some of the technical features. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention shall be included within the scope of protection of the present invention.

Claims

1. A three-dimensional thin-walled parts layout algorithm for a single-sided infinitely extended raw material plate, characterized by: The method comprises the following steps: S1. Project the 3D part onto a 2D plane using a 3D projection algorithm. Calculate the minimum enveloping polygon of the projected point set using a minimum convex hull algorithm. Calculate the expanded polygon of the minimum enveloping polygon using a polygon expansion algorithm. S2. Encode reinforcement learning actions and states. Encode the expanded polygons into generalized images. The reinforcement learning action is to select a generalized image from these generalized images. Encode the current layout of the raw material board into a generalized image as the reinforcement learning state. Design a state transition function based on the actions and states. Design a reinforcement learning reward function based on the compactness of the polygons after placement and the area size of the individual polygons. , build a deep reinforcement learning model: design two identical neural networks based on the reinforcement learning state: value network and target network , and adopt - Decay strategy exploration allows the agent to interact with the environment and The agent chooses actions according to the strategy After that, update the layout to get the new state , and according to the action and status Calculate the reward of this action , the state ,action , return and the new state Put it into the experience pool, and calculate the prediction score when the data in the experience pool reaches batch1 and actual score : ， And train and update the value network Parameter epoch1, when the total number of rounds reaches epoch2 Parameter assignment Update the target network parameters, output the action sequence when the number of rounds reaches the total number of epochs, and finally generate the placement angle, placement order and placement position of each polygon based on the action sequence; S3. Place the raw material plate at the coordinate origin, transform each 3D part in the projection direction to obtain transformed part 1. Then, based on the polygon placement angle and position calculated in S2, transform part 1 to obtain the final 3D part position on the raw material plate. Finally, place all parts in the raw material plate coordinate system, and output the final nested 3D assembly. The encoding method of "encoding reinforcement learning actions and states" in step S2 is as follows: Reinforcement learning action encoding, extracting the expanded polygon All elements in the array are spaced by rotation angle , traverse the rotation Generates a set of inflated polygon sub-polygons Each action selects the expanded polygon sub-polygon set One element in , the action and dilation subpolygon are represented as the following mapping: Where i and n are integers. The set of polygon sub-polygons for the inflated polygon The rotation angle of the i-th polygon The sub-polygon generated later; The polygon corresponding to the action is regarded as a pixel Image , pixel value Whether there is a polygon vertex within the corresponding pixel range, if Indicates that the pixel is a polygon vertex and the feature matrix is ​​designed and set As the final encoded action, the first feature indicates whether the pixel value is a vertex, and the second and third features are the exact values ​​of the vertex, namely: For polygon vertices, the final action is encoded as: ; Reinforcement learning state encoding, taking the current layout of the raw material board as the reinforcement learning state, and treating the layout as pixels to further extract features Image , pixel value Indicates whether there is a polygon vertex within the pixel range and sets the feature matrix Further description of the status characteristics: Feature Matrix Use the number of polygon vertices in the current pixel , the exact x-coordinate and the exact y-coordinate of the vertex describe the state of the polygon vertex. The specific calculation method is as follows: For the polygon vertices in the raw material board layout, the state encoding feature matrix is ​​finally .

2. The three-dimensional thin-walled parts layout algorithm of a single-sided infinitely extended raw material plate according to claim 1 is characterized in that The three-dimensional part formats in the projection algorithm described in step S1 include point cloud format and polygon vertex set format. For the polygon vertex set format part, step S1 adopts the PCA dimensionality reduction projection method, which specifically includes the following steps: Step 1.1: Read the polygon vertex set from the 3D part ; Step 1.2, calculate the center point coordinates ; Step 1.3, Centralize the Vertex Set Dataset ,get ; Step 1.4, calculate the covariance matrix ; Step 1.5, calculate the eigenvalues ​​of the covariance matrix and eigenvectors , and sort the eigenvalues, select the eigenvectors corresponding to the two largest eigenvalues ​​as the projection basis, and finally calculate the two-dimensional point set after dimensionality reduction ; The method for processing parts in point cloud format is the same as that in polygon vertex collection format, except that the amount of calculation is relatively larger.

3. The three-dimensional thin-walled parts layout algorithm of a single-sided infinitely extended raw material plate according to claim 1 is characterized in that The minimum convex hull algorithm calculation described in step S1 adopts the Graham-Andrew scanning algorithm, and the specific implementation steps are as follows: Step 2.1, traverse the input polygon vertex set , find the point with the smallest vertical coordinate As the starting point of the convex hull, that is ; Step 2.2, All other points in Sort by polar angle for benchmark; Step 2.3, from Start scanning all points and use vector cross product to determine the points Is it inside the convex hull? If so, it is a convex hull point and is added to the convex hull point set. Otherwise, traverse the next point, that is: 。 4. The three-dimensional thin-walled parts layout algorithm of a single-sided infinitely extended raw material plate according to claim 1 is characterized in that The polygon expansion algorithm described in step S1 uses the method of finding the Minkowski sum of the polygon and the fixed radius circle. , the calculation adopts the convolution-based Minkowski vector and calculation, the specific method is as follows: Step 3.1, sort all vertices of polygons M and N to get a counterclockwise vertex sequence 、 ; Step 3.2, calculate the simplified convolution K of M and N, that is and A set of line segments, where and is a convex vertex; Step 3.3: Extract non-overlapping, orientable loops L from K (if the normals of all line segments are consistently facing inward or outward, the loop L is considered orientable), and analyze their nesting relationships for further filtering; Step 3.4, check the intersections between the input polygons placed in the loop layout, filter out the boundaries, and the final result is the Minkowski vector sum; Convert the circle to a polygon and calculate its convex hull with all polygons respectively. The Minkowski sum of the dilated polygonal set is obtained .

5. The three-dimensional thin-walled parts layout algorithm of a single-side infinitely extended raw material plate according to claim 1 is characterized in that The reward function in step S2 ,include and These three parts include: The area of ​​the current polygon; Used to evaluate the compactness of placement. This score is composed of the Minkowski vector sum of the raw material board and the expanded polygon of the current polygon, and the Minkowski vector sum of the current layout area and the expanded polygon of the current polygon. The specific calculation formula is as follows: It is a reward function designed for tasks and planning goals. The specific design method is: the inverse of the product of the length increased in the x direction after the current polygon is placed and the width d of the board is used as ; Final reward function The calculation formula is as follows: are the weights of the three scores.

6. The three-dimensional thin-walled parts layout algorithm of a single-side infinitely extended raw material plate according to claim 1 is characterized in that Step S2 constructs a deep reinforcement learning model, using the DQN deep reinforcement learning model, where the value network and target network Neural network, exactly the same, using convolutional neural network CNN model, the input is the current state and actions , the output is the prediction score and actual score , these two networks are exactly the same, first use the action graph State diagram Perform a convolution and pooling operation, and then Layer convolution layer plus pooling layer and The final network is composed of layers of fully connected layers.

7. The three-dimensional thin-walled parts layout algorithm of a single-side infinitely extended raw material plate according to claim 1 is characterized in that The specific process of "layout update" in step S2 is as follows: Step 4.1, decoding action , get the current placement polygon , use Minkowski and calculate the current polygon to be placed With raw polygons The critical inscribed polygon of , and calculate the current polygon to be placed The circumscribed critical polygon of all placed polygons in the current layout ; Step 4.2, use the inscribed critical polygon For all external critical Perform a subtraction operation to obtain the collision-free area of ​​the current polygon under the current layout , the formula is as follows: Step 4.3, use the heuristic placement strategy to complete the placement of the current polygon.

8. The three-dimensional thin-walled parts layout algorithm of a single-side infinitely extended raw material plate according to claim 7 is characterized in that The heuristic placement strategy described in step 4.3 of "Layout Update" uses the lowest horizontal line placement strategy.

Images