Automatic stone nesting method and apparatus, and electronic device and readable storage medium

Abstract

An automatic stone nesting method and apparatus, and an electronic device and a readable storage medium, which belong to the technical field of stone processing. The method comprises: (1) building a stone nesting simulation environment, and designing an action space and a reward function on the basis of the rules that parts must be positioned within raw materials, the parts should not overlap each other and the utilization rate of the raw materials should be maximized; (2) constructing a PPO-Clip algorithm model comprising an Actor-Critic network, and using an Actor-Critic algorithm to separately train an Actor neural network and a Critic neural network; and (3) on the basis of a stone-part production order, using images obtained by preprocessing parts images and raw-material images as inputs of the networks, and outputting action instructions for arranging the parts, so as to complete an automatic nesting task under a new part order. The present invention can avoid the situation where critical polygon calculation methods are inapplicable to concave or perforated parts, and can also avoid inaccurate algorithm results or low calculation efficiency.

Description

Automatic stone material arrangement method, device, electronic device and readable storage medium

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to Chinese patent application number 202410223626.2, filed with the Patent Office of China on February 29, 2024, entitled “A method, device, electronic device and readable storage medium for automatic stone arrangement”, the entire contents of which are incorporated herein by reference. Technical Field

[0003] The present invention belongs to the technical field of stone processing, and in particular to an automatic stone discharging method, device, electronic equipment and readable storage medium. Background Art

[0004] Stone processing is a complex process requiring specialized skills and equipment. Optimizing stone nesting has always been a major challenge in stone cutting: Within a given spatial layout of raw materials, it's crucial to find an optimal nesting solution to create a number of parts that meet the required shapes and sizes, without overlapping. Traditional nesting methods often rely on manual experience, resulting in low efficiency and accuracy. In recent years, with the development and application of computer science, modern manufacturing and processing industries have become increasingly automated and intelligent, and computer-automated nesting has become a key solution to this problem.

[0005] Currently, the nesting problem can be decomposed into two sub-problems: positioning and sequencing of parts. The positioning problem involves geometric operations, and solving the critical polygon is the key. The sequencing problem often uses heuristic algorithms to optimize the arrangement order of items by adjusting parameters to achieve better quality and sheet material utilization. However, the above methods still have some defects. First, the calculation of critical polygons for irregular parts is very complicated, and most calculation methods cannot calculate concave parts or parts with holes inside; second, the heuristic algorithm is an approximate solution and may fall into local optimal solutions and cannot find the global optimal solution. In addition, the algorithm needs to determine many parameters and variables, such as mutation rate, population size, etc. If these parameters are not set correctly, it may lead to inaccurate algorithm results or low computational efficiency.

[0006] Therefore, although the existing stone arrangement method based on critical polygons and heuristic algorithms can achieve certain results, it still has some inconveniences. In practical applications, we need to improve and optimize the problems in order to obtain a better stone arrangement solution. Summary of the Invention

[0007] The purpose of this invention is to propose an automatic stone nesting method based on image and deep reinforcement learning. By constructing an ActorCritic convolutional network model and using the PPO-Clip algorithm, it does not need to calculate the critical polygons of parts to judge overlap. At the same time, it can continuously update and optimize its own action strategy through trial and error reinforcement learning.

[0008] The present invention also proposes an automatic stone arranging device, which is used to execute the above-mentioned automatic stone arranging method based on image and deep reinforcement learning.

[0009] The present invention also proposes an electronic device for implementing the above-mentioned automatic stone arrangement method based on image and deep reinforcement learning.

[0010] The present invention also proposes a computer-readable storage medium, the program of which, when executed by a processor, implements the above-mentioned automatic stone arrangement method based on image and deep reinforcement learning.

[0011] To achieve this object, the present invention adopts the following technical solutions:

[0012] An automatic stone nesting method based on image and deep reinforcement learning includes the following steps:

[0013] (1) Build a stone nesting simulation environment and design the corresponding action space and reward function based on the rules that the parts must be located inside the raw material, parts cannot overlap, and the utilization rate of the raw material is maximized;

[0014] (2) Construct a PPO-Clip algorithm model including an Actor-Critic network. Based on the stone material arrangement simulation environment, action space and reward function, the Actor-Critic algorithm is used to train the Actor and Critic neural networks simultaneously. The Actor network inputs the current state s, including: raw material information and part information to be arranged, and outputs a as the part number, displacement and rotation angle. Through the strategy function π θ (a|s) represents the probability of outputting a given the input state s. The Critic network evaluates the value of the strategy based on the actions of the Actor network and feeds it back to the Actor network. The network output is a prediction of the reward function in step (1).

[0015] The Actor network and the Critic network adopt the same convolutional network structure: the input of the first convolution layer is the image obtained by preprocessing the part image and the raw material image in the current state, and the number of output channels, convolution kernel size and step size are set; maximum pooling is performed; the input channel number, output channel number, convolution kernel size and step size of the multi-layer convolution layer are set respectively; maximum pooling is performed; it is expanded into one dimension and an SPP layer is used; finally, the output is passed through a fully connected layer; the output of the Actor network is the action taken in the current state, and the output of the Critic network is the reward size after taking the action;

[0016] The Actor-Critic network loss function defined by the PPO-Clip algorithm is as follows:

[0017] in is the ratio of new and old strategies;

[0018] is the advantage function, that is, the difference between the action value function and the policy value function; clip is the clipping function, and ε is a hyperparameter:

[0019] θ k is the kth network parameter; θ is the updated network parameter; π θ (a|s) is the new strategy;

[0020] is the old policy; Q is the action value function, V is the state value function; x is the function independent variable;

[0021] (3) Based on the model obtained by training, according to the new stone parts production order, the part image corresponding to the new stone parts production order and the image obtained by preprocessing the raw material image are used as the input of the network, and the action instructions for arranging the parts are output to complete the automatic material arrangement task under the new stone parts production order.

[0022] Optimally, the step (1) includes:

[0023] (1-1) According to the size of the stone material and the pixel accuracy requirements, set the length and width of the raw material image;

[0024] (1-2) Based on the nesting principle that parts must be located within the raw material, parts must not overlap, and raw material utilization must be maximized, the design action is divided into three sub-actions: part selection, translation, and rotation. The rotation angle is discretized into 2kπ / N, where k = 0, 1, …, N-1; N is a positive integer. The design reward function is:

[0025] Among them, S 已排零件 is the total area of ​​parts that have been discharged onto the raw material, L已排零件 is the total length that has been discharged onto the raw material, h is the width of the raw material, and Fa is a positive number.

[0026] It can be optimized that in step (2), the first convolution layer input is the image obtained by preprocessing the part image and the raw material image in the current state, the number of output channels is 32, the convolution kernel size is 8×8, and the step size is 2; maximum pooling;

[0027] The second convolutional layer has 32 input channels, 64 output channels, a 4×4 kernel size, and a stride of 2; maximum pooling is used.

[0028] The third convolutional layer has 64 input channels, 64 output channels, a convolution kernel size of 3×3, and a stride of 1; maximum pooling is used.

[0029] Optimally, in step (1), the pixel value of each point in the raw image is set to 1.

[0030] An automatic stone material discharging device includes: an environment simulation module, a model building module and a model training module;

[0031] The environment simulation module is configured to build a stone material nesting simulation environment and design the corresponding action space and reward function based on the rules that the parts must be located inside the raw material, parts cannot overlap, and the utilization rate of the raw material is maximized;

[0032] The model building module is configured to build a PPO-Clip algorithm model including an Actor-Critic network, and use the Actor-Critic algorithm to simultaneously train the Actor and Critic neural networks based on the stone nesting simulation environment, action space and reward function; the Actor network inputs the current state s, including: raw material information and information about the parts to be nested, and outputs a as the number, displacement and rotation angle of the part; through the strategy function π θ (a|s) represents the probability of outputting a given the input state s. The Critic network evaluates the value of the strategy based on the actions of the Actor network and feeds it back to the Actor network. The network output is a prediction of the reward function in step (1).

[0033] The same convolutional network structure is used for both the Actor and the Critic: the first convolutional layer input is the preprocessed image of the part image and the raw material image in the current state, and the number of output channels, convolution kernel size, and step size are set; maximum pooling is performed; the number of input channels, output channels, convolution kernel size, and step size of the multi-layer convolutional layer are set respectively; maximum pooling is performed; it is expanded into one dimension and the SPP layer is used; finally, the output is passed through a fully connected layer; the output of the Actor network is the action taken in the current state, and the output of the Critic network is the reward size after taking the action;

[0034] The Actor-Critic network loss function defined by the PPO-Clip algorithm is as follows:

[0035] in is the ratio of new and old strategies;

[0036] is the advantage function, that is, the difference between the action value function and the policy value function; clip is the clipping function, and ε is a hyperparameter:

[0037] θ k is the kth network parameter; θ is the updated network parameter; π θ (a|s) is the new strategy;

[0038] is the old policy; Q is the action value function, V is the state value function; x is the function independent variable;

[0039] The model training module is configured to use the model obtained by training, according to the new stone parts production order, the part image corresponding to the new stone parts production order and the image obtained by preprocessing the raw material image as the input of the network, output the action instructions for arranging the parts, and complete the automatic material arrangement task under the new stone parts production order.

[0040] Optimally, the environment simulation module includes: an image generation module and a function design module;

[0041] The image generation module is configured to set the length and width of the raw material image according to the size of the stone raw material and the pixel accuracy requirement;

[0042] The function design module is configured to follow the nesting principle that parts must be located within the raw material, parts cannot overlap, and the utilization rate of raw materials is maximized. The design action is divided into three sub-actions: selecting parts, translation, and rotation. The rotation angle is discretized into 2kπ / N, where k = 0, 1, ..., N-1; N is a positive integer. The design reward function is:

[0043] Among them, S 已排零件 is the total area of ​​parts that have been discharged onto the raw material, L 已排零件 is the total length that has been discharged onto the raw material, h is the width of the raw material, and Fa is a positive number.

[0044] An electronic device, comprising:

[0045] one or more processors;

[0046] a storage device for storing one or more programs;

[0047] One or more of the programs are executed by one or more of the processors, so that one or more of the processors implement the above-mentioned automatic stone arrangement method based on image and deep reinforcement learning.

[0048] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the above-mentioned automatic stone nesting method based on image and deep reinforcement learning.

[0049] The technical solution provided by the present invention can have the following beneficial effects:

[0050] This solution provides an automatic stone nesting method based on image and deep reinforcement learning. It constructs an ActorCritic convolutional network model and uses the PPO-Clip algorithm. It does not need to calculate the critical polygons of parts to determine overlap. Since the image pixel values ​​will be updated after the parts are placed on the raw materials, it is only necessary to verify whether the pixel values ​​within the part range under the current action are abnormal, thereby avoiding the situation where the critical polygon calculation method is not applicable to concave or hole-containing parts; secondly, the present invention greatly reduces the setting of hyperparameters, and the parameters are obtained through training, avoiding inaccurate algorithm results or low calculation efficiency due to incorrect hyperparameter settings; further, compared with heuristic algorithms, the present invention can continuously update and optimize its own action strategy through trial and error reinforcement learning, so it is adaptive; and when facing complex and unknown situations, reinforcement learning can also achieve better results. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] FIG1 is a flow chart of the automatic stone discharging method proposed by the present invention;

[0052] FIG2 is a schematic diagram of a state space of one embodiment of the present invention;

[0053] FIG3 is a schematic diagram of the network structure of Actor and Critic in the PPO algorithm of the present invention. DETAILED DESCRIPTION

[0054] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0055] The technical solution of this solution will be further explained below through specific implementation methods in conjunction with the accompanying drawings.

[0056] As shown in Figure 1-3, an automatic stone nesting method based on image and deep reinforcement learning includes the following steps:

[0057] (1) Build a stone nesting simulation environment and design the corresponding action space and reward function based on the rules that the parts must be located inside the raw material, parts cannot overlap, and the utilization rate of the raw material is maximized;

[0058] (2) Construct a PPO-Clip algorithm model including an Actor-Critic network, and train both the Actor neural network and the Critic neural network in the Actor-Critic algorithm;

[0059] (3) Based on the model obtained by training, according to the new stone parts production order, the part image corresponding to the new stone parts production order and the image obtained by preprocessing the raw material image are used as the input of the network, and the action instructions for arranging the parts are output to complete the automatic material arrangement task under the new stone parts production order.

[0060] This solution provides an automatic stone nesting method based on image and deep reinforcement learning. It constructs an ActorCritic convolutional network model and uses the PPO-Clip algorithm. It does not need to calculate the critical polygons of parts to determine overlap. Since the image pixel values ​​will be updated after the parts are placed on the raw materials, it is only necessary to verify whether the pixel values ​​within the part range under the current action are abnormal, thereby avoiding the situation where the critical polygon calculation method is not applicable to concave or hole-containing parts; secondly, the present invention greatly reduces the setting of hyperparameters, and the parameters are obtained through training, avoiding inaccurate algorithm results or low calculation efficiency due to incorrect hyperparameter settings; further, compared with heuristic algorithms, the present invention can continuously update and optimize its own action strategy through trial and error reinforcement learning, so it is adaptive; and when facing complex and unknown situations, reinforcement learning can also achieve better results.

[0061] Optimally, the step (1) includes:

[0062] (1-1) According to the size of the stone material and the pixel accuracy requirements, set the length and width of the raw material image;

[0063] The size of stone raw materials can be set according to actual needs;

[0064] (1-2) Based on the nesting principle that parts must be located within the raw material, parts must not overlap, and raw material utilization must be maximized, the design action is divided into three sub-actions: part selection, translation, and rotation. The rotation angle is discretized into 2kπ / N, where k = 0, 1, …, N-1; N is a positive integer. The design reward function is:

[0065] Among them, S 已排零件 is the total area of ​​parts that have been discharged onto the raw material, L 已排零件 is the total length that has been discharged onto the raw material, h is the width of the raw material, and Fa is a positive number.

[0066] Design reward function, for example:

[0067] The reward value is the current area utilization rate when parts are placed without overlapping. If there is overlap or the parts exceed the raw material boundary, the reward value is -100. Regarding the principle that parts must be placed inside the raw material, the present invention can solve it by using an envelope rectangle. The envelope rectangle of the part is limited to the inside of the raw material when it is translated or rotated. Regarding the principle that parts cannot overlap, when judging overlap, since the pixel value of the position after the part is placed on the raw material is -1, if there is no overlap, the pixel values ​​at this position are all 0. If there is overlap, the pixel value is less than 0, which leads to an abnormality. Therefore, it is only necessary to verify whether the pixel value within the range of the part under the current action is abnormal, thereby simplifying the judgment of part overlap. To maximize the utilization of raw materials, S 已排零件 / (h×L 已排零件 ) represents the utilization rate. The higher the utilization rate, the higher the reward. The model will arrange materials based on the standard of the higher the utilization rate, the better.

[0068] In the step (2), a PPO-Clip algorithm model including an Actor-Critic network is constructed. Based on the stone material arrangement simulation environment, action space and reward function, the Actor-Critic algorithm is used to simultaneously train the Actor and Critic neural networks. The Actor network inputs the current state s, including: raw material information and part information to be arranged, and outputs a as the part number, displacement and rotation angle. Through the strategy function π θ (a|s) represents the probability of outputting a given the input state s. The Critic network evaluates the value of the strategy based on the actions of the Actor network and feeds it back to the Actor network. The network output is a prediction of the reward function in step (1).

[0069] The Actor network learns the probability distribution of the action set based on the input information, generates actions based on the probabilities, and adjusts the strategy based on the evaluation of the Critic network. The Critic network evaluates the value of the strategy based on the actions of the Actor network and feeds back to the Actor network. The network output is a prediction of the reward function.

[0070] The Actor network and the Critic network adopt the same convolutional network structure: the input of the first convolution layer is the image obtained by preprocessing the part image and the raw material image in the current state, and the number of output channels, convolution kernel size and stride are set; maximum pooling is performed, that is, downsampling the image and taking the maximum value among adjacent pixels; the number of input channels, output channels, convolution kernel size and stride of the multi-layer convolution layer are set respectively; maximum pooling is performed; it is expanded into one dimension and the SPP layer is used; finally, the output is passed through the fully connected layer; the output of the Actor network is the action taken in the current state, and the output of the Critic network is the reward size after taking the action;

[0071] The Actor-Critic network loss function defined by the PPO-Clip algorithm is as follows:

[0072] in is the ratio of new and old strategies;

[0073] is the advantage function, that is, the difference between the action value function and the policy value function; clip is the clipping function, and ε is a hyperparameter:

[0074] θ k is the kth network parameter; θ is the updated network parameter; π θ (a|s) is the new strategy;

[0075] is the old policy; Q is the action value function, V is the state value function; x is the function independent variable.

[0076] It can be optimized that in step (2), the first convolution layer input is the image obtained by preprocessing the part image and the raw material image in the current state, the number of output channels is 32, the convolution kernel size is 8×8, and the step size is 2; maximum pooling;

[0077] The second convolutional layer has 32 input channels, 64 output channels, a 4×4 kernel size, and a stride of 2; maximum pooling is used.

[0078] The third convolutional layer has 64 input channels, 64 output channels, a convolution kernel size of 3×3, and a stride of 1; maximum pooling is used.

[0079] Optimally, in step (1), the pixel value of each point in the raw image is set to 1.

[0080] In step (1), only the arrangement of the stone can be considered without considering the color, texture and other information of the raw materials. Therefore, the pixel value of each point only needs to be set to 1.

[0081] An automatic stone material discharging device includes: an environment simulation module, a model building module and a model training module;

[0082] The environment simulation module is configured to build a stone material nesting simulation environment and design the corresponding action space and reward function based on the rules that the parts must be located inside the raw material, parts cannot overlap, and the utilization rate of the raw material is maximized;

[0083] The model building module is configured to build a PPO-Clip algorithm model including an Actor-Critic network and simultaneously train the Actor neural network and the Critic neural network in the Actor-Critic algorithm;

[0084] The model training module is configured to use the model obtained by training, according to the new stone parts production order, the part image corresponding to the new stone parts production order and the image obtained by preprocessing the raw material image as the input of the network, output the action instructions for arranging the parts, and complete the automatic material arrangement task under the new stone parts production order.

[0085] The model building module is configured to construct a PPO-Clip algorithm model including an Actor-Critic network. Based on the stone nesting simulation environment, action space and reward function, the Actor-Critic algorithm is used to simultaneously train the Actor and Critic neural networks. The Actor network inputs the current state s, including: raw material information and part information to be nested, and outputs a as the part number, displacement and rotation angle. Through the strategy function π θ (a|s) represents the probability of outputting a given the input state s. The Critic network evaluates the value of the strategy based on the actions of the Actor network and feeds it back to the Actor network. The network output is a prediction of the reward function in step (1).

[0086] The same convolutional network structure is used for both the Actor and the Critic: the first convolutional layer input is the preprocessed image of the part image and the raw material image in the current state, and the number of output channels, convolution kernel size, and step size are set; maximum pooling is performed; the number of input channels, output channels, convolution kernel size, and step size of the multi-layer convolutional layer are set respectively; maximum pooling is performed; it is expanded into one dimension and the SPP layer is used; finally, the output is passed through a fully connected layer; the output of the Actor network is the action taken in the current state, and the output of the Critic network is the reward size after taking the action;

[0087] The Actor-Critic network loss function defined by the PPO-Clip algorithm is as follows:

[0088] in is the ratio of new and old strategies;

[0089] is the advantage function, that is, the difference between the action value function and the policy value function; clip is the clipping function, and ε is a hyperparameter:

[0090] θ k is the kth network parameter; θ is the updated network parameter; π θ (a|s) is the new strategy;

[0091] is the old policy; Q is the action value function, V is the state value function; x is the function independent variable.

[0092] Optimally, the environment simulation module includes: an image generation module and a function design module;

[0093] The image generation module is configured to set the length and width of the raw material image according to the size of the stone raw material and the pixel accuracy requirement;

[0094] The function design module is configured to follow the nesting principle that parts must be located within the raw material, parts cannot overlap, and the utilization rate of raw materials is maximized. The design action is divided into three sub-actions: selecting parts, translation, and rotation. The rotation angle is discretized into 2kπ / N, where k = 0, 1, ..., N-1; N is a positive integer. The design reward function is:

[0095] Among them, S 已排零件 is the total area of ​​parts that have been discharged onto the raw material, L 已排零件 is the total length that has been discharged onto the raw material, h is the width of the raw material, and Fa is a positive number.

[0096] An electronic device, comprising:

[0097] one or more processors;

[0098] a storage device for storing one or more programs;

[0099] One or more of the programs are executed by one or more of the processors, so that one or more of the processors implement an automatic stone arrangement method based on image and deep reinforcement learning as described in any of the above embodiments.

[0100] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements an automatic stone nesting method based on image and deep reinforcement learning as in any of the above embodiments.

[0101] Embodiment 1:

[0102] Step 1: Build a stone nesting simulation environment and design the corresponding action space and reward function based on the rules that parts must be positioned within the raw material, parts cannot overlap, and raw material utilization is maximized. Based on the stone raw material size and pixel accuracy requirements, the raw material image size is set to 300*100, with a length of w=300 and a width of h=100. The pixel value of each point is set to 1. Nesting is based on the principle that parts must be positioned within the raw material, parts cannot overlap, and raw material utilization is maximized.

[0103] The design action is divided into three sub-actions: selecting parts, translation, and rotation. The rotation angle is discretized into 2kπ / N (k=0,1,...,N-1). The design reward function is:

[0104] If the parts are not overlapped, the reward value is the current area utilization rate. If there is overlap or the parts exceed the raw material boundary, the reward value is -100.

[0105] For overlap determination, the pixel value at that position is -1 after the part is placed on the raw material. If there is no overlap, all pixel values ​​at that position are 0. If there is overlap, the pixel value is less than 0, which causes an abnormality. Therefore, it is only necessary to verify whether the pixel values ​​within the part range under the current action are abnormal. The principle that parts must be placed inside the raw material can be solved by using an envelope rectangle. That is, the envelope rectangle of the part is confined to the raw material when it is translated or rotated. It is only necessary to determine whether the vertices of the envelope rectangle are located inside the raw material.

[0106] Step 2: Build a PPO-Clip algorithm model including the Actor-Critic network. The Actor-Critic algorithm requires training both the Actor and Critic neural networks, which are responsible for learning the strategy and the value function respectively.

[0107] The Actor network learns the probability distribution of the action set based on the input information, generates actions based on the probability, and adjusts the strategy based on the evaluation of the Critic network. The input of the Actor network is the current state s (raw material information and part information to be sorted), and the output a is the part number, displacement and rotation angle. The strategy function π θ (a|s) represents the probability of output a given the input state s.

[0108] The Critic network evaluates the value of the strategy based on the actions of the Actor network and feeds it back to the Actor network. The network output is a prediction of the reward function.

[0109] As shown in Figure 3, the Actor and Critic adopt the same convolutional network structure: the input of the first convolution layer is the image obtained by preprocessing the part image and the raw material image in the current state, with an output channel number of 32, a convolution kernel size of 8×8, and a stride of 2; maximum pooling; the second convolution layer has an input channel number of 32, an output channel number of 64, a convolution kernel size of 4×4, and a stride of 2; maximum pooling; the third convolution layer has an input channel number of 64, an output channel number of 64, a convolution kernel size of 3×3, and a stride of 1; maximum pooling; it is expanded into one dimension, using the SPP layer; and finally output through the fully connected layer.

[0110] The output of the Actor network is the action taken in the current state; the output of the Critic network is the reward obtained after taking the action. The input image must contain information about both sorted parts and parts to be sorted. This embodiment fuses these two pieces of information into a single image, as shown in the state space in Figure 2.

[0111] The Actor-Critic network loss function defined by the PPO-Clip algorithm is as follows:

[0112] in is the ratio of new and old strategies;

[0113] is the advantage function, that is, the difference between the action value function and the policy value function; clip is the clipping function, and ε is a hyperparameter:

[0114] θ k is the kth network parameter; θ is the updated network parameter; π θ (a|s) is the new strategy;

[0115] is the old policy; Q is the action value function, V is the state value function; x is the function independent variable.

[0116] The technical principles of this solution have been described above in conjunction with specific embodiments. These descriptions are intended solely to explain the principles of this solution and should not be construed in any way as limiting the scope of protection of this solution. Based on the explanations herein, those skilled in the art will be able to devise other specific implementations of this solution without inventive effort, and these implementations will fall within the scope of protection of this solution.

Claims

1. An automatic stone nesting method based on image and deep reinforcement learning, characterized in that: The following steps are involved: (1) Build a stone nesting simulation environment and design the corresponding action space and reward function based on the rules that the parts must be located inside the raw material, parts cannot overlap, and the utilization rate of the raw material is maximized; (2) Construct a PPO-Clip algorithm model including an Actor-Critic network. Based on the stone material layout simulation environment, action space and reward function, the Actor-Critic algorithm is used to simultaneously train the Actor and Critic neural networks. The Actor network inputs the current state s, including: raw material information and part information to be sorted, and outputs a as the part number, displacement and rotation angle; through the strategy function π θ (a|s) represents the probability of outputting a given the input state s. The Critic network evaluates the value of the strategy based on the actions of the Actor network and provides feedback to the Actor network. The Actor network and the Critic network adopt the same convolutional network structure: the input of the first convolution layer is the image obtained by preprocessing the part image and the raw material image in the current state, and the number of output channels, convolution kernel size and step size are set; maximum pooling is performed; the input channel number, output channel number, convolution kernel size and step size of the multi-layer convolution layer are set respectively; maximum pooling is performed; it is expanded into one dimension and an SPP layer is used; finally, the output is passed through a fully connected layer; the output of the Actor network is the action taken in the current state, and the output of the Critic network is the reward size after taking the action; The Actor-Critic network loss function defined by the PPO-Clip algorithm is as follows: in is the ratio of new and old strategies; is the advantage function, that is, the difference between the action value function and the policy value function; clip is the clipping function, and ε is a hyperparameter: θ k is the kth network parameter; θ is the updated network parameter; π θ (a|s) is the new strategy; is the old policy; Q is the action value function, V is the state value function; x is the function independent variable; (3) Based on the model obtained by training, according to the new stone parts production order, the part image corresponding to the new stone parts production order and the image obtained by preprocessing the raw material image are used as the input of the network, and the action instructions for arranging the parts are output to complete the automatic nesting task under the new stone parts production order; The step (1) comprises: (1-1) According to the size of the stone material and the pixel accuracy requirements, set the length and width of the raw material image; (1-2) Based on the nesting principle that parts must be located within the raw material, parts must not overlap, and raw material utilization must be maximized, the design action is divided into three sub-actions: part selection, translation, and rotation. The rotation angle is discretized into 2kπ / N, where k = 0, 1, …, N-1; N is a positive integer. The design reward function is: Among them, S 已排零件 is the total area of ​​parts that have been discharged onto the raw material, L 已排零件 is the total length that has been discharged onto the raw material, h is the width of the raw material, and Fa is a positive number; In the step (2), the first convolution layer input is the image obtained by preprocessing the part image and the raw material image in the current state, the number of output channels is 32, the convolution kernel size is 8×8, and the step size is 2; maximum pooling; The second convolutional layer has 32 input channels, 64 output channels, a 4×4 kernel size, and a stride of 2; maximum pooling is used. The third convolutional layer has 64 input channels, 64 output channels, a convolution kernel size of 3×3, and a stride of 1; maximum pooling is used. In the step (1), the pixel value of each point in the raw image is set to 1.

2. An automatic stone discharging device, characterized in that: include: Environment simulation module, model building module and model training module; The environment simulation module is configured to build a stone nesting simulation environment and design a corresponding action space and reward function based on the rules that the parts must be located within the raw material, parts cannot overlap, and the utilization rate of the raw material is maximized; the pixel value of each point in the raw material image is set to 1; The model building module is configured to build a PPO-Clip algorithm model including an Actor-Critic network, and use the Actor-Critic algorithm to simultaneously train the Actor and Critic neural networks based on the stone nesting simulation environment, action space and reward function; The Actor network inputs the current state s, including: raw material information and part information to be sorted, and outputs a as the part number, displacement and rotation angle; through the strategy function π θ (a|s) represents the probability of outputting a given the input state s. The Critic network evaluates the value of the strategy based on the actions of the Actor network and provides feedback to the Actor network. The same convolutional network structure is used for both the Actor and the Critic: the first convolutional layer input is the preprocessed image of the part image and the raw material image in the current state, and the number of output channels, convolution kernel size, and step size are set; maximum pooling is performed; the number of input channels, output channels, convolution kernel size, and step size of the multi-layer convolutional layer are set respectively; maximum pooling is performed; it is expanded into one dimension and the SPP layer is used; finally, the output is passed through a fully connected layer; the output of the Actor network is the action taken in the current state, and the output of the Critic network is the reward size after taking the action; The Actor-Critic network loss function defined by the PPO-Clip algorithm is as follows: in is the ratio of new and old strategies; is the advantage function, that is, the difference between the action value function and the policy value function; clip is the clipping function, and ε is a hyperparameter: θ k is the kth network parameter; θ is the updated network parameter; π θ (a|s) is the new strategy; is the old policy; Q is the action value function, V is the state value function; x is the function independent variable; The model training module is configured to use the model obtained by training and the image obtained by pre-processing the part image corresponding to the new stone part production order and the raw material image as the input of the network, output the action instruction of arranging the parts, and complete the automatic nesting task under the new stone part production order; The environment simulation module includes: an image generation module and a function design module; The image generation module is configured to set the length and width of the raw material image according to the size of the stone raw material and the pixel accuracy requirement; The function design module is configured to follow the nesting principle that parts must be located within the raw material, parts cannot overlap, and the utilization rate of raw materials is maximized. The design action is divided into three sub-actions: selecting parts, translation, and rotation. The rotation angle is discretized into 2kπ / N, where k = 0, 1, ..., N-1; N is a positive integer. The design reward function is: Among them, S 已排零件 is the total area of ​​parts that have been discharged onto the raw material, L 已排零件 is the total length that has been discharged onto the raw material, h is the width of the raw material, and Fa is a positive number; The input of the first convolution layer is the image obtained by preprocessing the part image and the raw material image in the current state. The number of output channels is 32, the convolution kernel size is 8×8, and the step size is 2; maximum pooling is used; the number of input channels of the second convolution layer is 32, the number of output channels is 64, the convolution kernel size is 4×4, and the step size is 2; maximum pooling is used; the number of input channels of the third convolution layer is 64, the number of output channels is 64, the convolution kernel size is 3×3, and the step size is 1; maximum pooling is used.

3. An electronic device, characterized in that: include: one or more processors; a storage device for storing one or more programs; One or more of the programs are executed by one or more of the processors, so that one or more of the processors implement the automatic stone nesting method based on image and deep reinforcement learning as described in claim 1.

4. A computer-readable storage medium storing a computer program, characterized in that: When the program is executed by a processor, it implements the automatic stone arrangement method based on image and deep reinforcement learning as described in claim 1.

Images