Workpiece sorting method, device, computer equipment and readable storage medium
By using a pre-trained decoder to determine the target material frame, rotation direction and sorting position in the steel plate cutting and sorting production line, the problems of low automation and long calculation time are solved, and efficient workpiece sorting and production efficiency are achieved promote.
Patent Information
- Application Number
- CN202411096995.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-12
- Publication Date
- 2024-11-05
- Estimated Expiration
- 2044-08-12
AI Technical Summary
The low degree of automation of steel plate cutting sorting production line leads to low production efficiency and high cost. The traditional heuristic algorithm has a long calculation time and cannot meet the real-time sorting needs.
The workpiece sorting method based on pre-trained marquee selection, direction selection and position selection decoder is adopted. By obtaining the workpiece and material frame status matrix, the target material frame, rotation direction and sorting position are determined to achieve efficient sorting of workpieces.
It improves workpiece sorting efficiency, reduces the operation space and sorting difficulty, meets the real-time sorting needs, and reduces labor and costs.
Smart Images

Figure CN118616343B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of artificial intelligence technology, and in particular to a workpiece sorting method, device, computer equipment and readable storage medium. Background Art
[0002] Steel plate blanking and forming are widely used in various industries, but the current steel plate blanking and sorting production lines generally have problems such as low automation, high labor intensity, and low production efficiency, which seriously affect the improvement of workshop production capacity and consume a lot of manpower and material costs. Under the development trend of intelligent manufacturing, the application of steel plate blanking intelligent sorting production lines is becoming more and more extensive. More and more companies choose to use robots to replace the original manual operations to achieve efficient, flexible, and clean automated production. The workpieces cut by the cutting machine are transported to the sorting area along the conveyor belt for sorting. Workpiece sorting is an intermediate link in production. How to sort the workpieces into the material frame not only affects the utilization rate of the material frame, but also directly affects the progress and quality of the entire production process. At the same time, due to the wide variety and quantity of workpieces, how to achieve efficient sorting and real-time packing of workpieces while meeting production constraints has become an important constraint to improve production efficiency and reduce enterprise production costs.
[0003] Traditional workpiece sorting mainly uses heuristic algorithms to traverse each material frame to determine the sorting strategy, such as the Next-fit (nearby adaptation algorithm) algorithm, the First-fit (first adaptation algorithm) algorithm, the Best-fit (best adaptation algorithm) algorithm, etc. Although these algorithms can improve the utilization rate of the material frame and reduce production costs to a certain extent, these algorithms require a lot of computing time and cannot meet the needs of real-time sorting, resulting in low workpiece sorting efficiency. Summary of the invention
[0004] Based on this, it is necessary to provide a workpiece sorting method, device, computer equipment, computer-readable storage medium and computer program product that can improve the workpiece sorting efficiency in response to the above-mentioned technical problems.
[0005] In a first aspect, the present application provides a workpiece sorting method, comprising:
[0006] Obtaining a workpiece state matrix of a workpiece to be processed and a material frame state matrix corresponding to each of a plurality of material frames to be processed, encoding the workpiece state matrix to obtain a workpiece state vector, and encoding a plurality of material frame state matrices to obtain a material frame state vector;
[0007] Based on the workpiece state vector, the material frame state vector and the pre-trained frame selection decoder, a target material frame is determined from multiple material frames to be processed, and a target state vector corresponding to the target material frame is determined;
[0008] Determine the target rotation direction based on the workpiece state vector, the target state vector and a pre-trained direction-selective decoder, and determine the rotated workpiece state vector based on the target rotation direction;
[0009] Based on the rotated workpiece state vector, the target state vector and the pre-trained position selection decoder, a target sorting position is determined among multiple grid positions of the target material frame;
[0010] Sort the workpieces to be processed based on the target material frame and target sorting position.
[0011] In one embodiment, based on the workpiece state vector, the frame state vector and a pre-trained frame decoder, determining a target frame from a plurality of frames to be processed includes:
[0012] A pre-trained selection frame decoder is used to decode the workpiece state vector and the material frame state vector to obtain a first decoding result; each element in the first decoding result is used to represent the probability of sorting the workpiece to be processed into multiple material frames to be processed;
[0013] According to the first decoding result and the frame mask, a target frame is determined from a plurality of frames to be processed.
[0014] In one embodiment, determining a target rotation direction based on a workpiece state vector, a target state vector, and a pre-trained direction-selective decoder includes:
[0015] The workpiece state vector and the target state vector are decoded by using a pre-trained direction-selective decoder to obtain a second decoding result; each element in the second decoding result is used to represent the probability that the workpiece to be processed can be sorted into the target material frame after rotating in a preset rotation direction;
[0016] The target rotation direction is determined according to the second decoding result and the direction mask.
[0017] In one embodiment, based on the rotated workpiece state vector, the target state vector and a pre-trained position selection decoder, a target sorting position is determined in a plurality of grid positions of a target material frame, including:
[0018] The rotated workpiece state vector and the target state vector are decoded using a pre-trained position selection decoder to obtain a third decoding result; each element in the third decoding result is used to characterize the probability of sorting the workpiece to be processed to each grid position in the target material frame;
[0019] According to the third decoding result and the position mask, a target sorting position is determined among a plurality of grid positions of the target material frame.
[0020] In one embodiment, after sorting the workpieces to be processed based on the target material frame and the target sorting position, the method further includes:
[0021] Acquire real-time material frame information and real-time workpiece information, and update the material frame mask, direction mask, and position mask according to the real-time material frame information and real-time workpiece information;
[0022] Workpiece sorting is performed based on the updated material frame mask, the updated direction mask, and the updated position mask.
[0023] In one embodiment, the workpiece state matrix and the frame state matrix are processed by a trained actor network, the actor network comprising a pre-trained frame selection decoder, a pre-trained direction selection decoder and a pre-trained position selection decoder; the training steps of the actor network include:
[0024] Obtaining a workpiece state matrix sample of a workpiece sample to be processed and a material frame state matrix sample corresponding to each of a plurality of material frame samples to be processed, encoding the workpiece state matrix sample to obtain a workpiece state vector sample, and encoding a plurality of material frame state matrix samples to obtain a material frame state vector sample;
[0025] Based on the workpiece state vector, the material frame state vector and the to-be-trained selection frame decoder in the initial actor network, a target material frame sample is determined from a plurality of to-be-processed material frame samples, and a target state vector sample corresponding to the target material frame sample is determined;
[0026] Determine a target rotation direction sample based on the workpiece state vector sample, the target state vector sample and the direction-selective decoder to be trained in the initial actor network, and determine a rotated workpiece state vector sample based on the target rotation direction sample;
[0027] Based on the rotated workpiece state vector sample, the target state vector sample and the to-be-trained position selection decoder in the initial actor network, a target sorting position sample is determined from a plurality of grid position samples of the target material frame sample;
[0028] Based on the workpiece state vector, the material frame state vector and the initial critic network, a review value output by the initial critic network is obtained;
[0029] Sorting the workpiece samples to be processed based on the target material frame samples and the target sorting position samples, and determining the reward value according to the sorted material frame information and the preset reward function;
[0030] The model loss is calculated based on the difference between the reward value and the comment value, and the model parameters of the initial actor network and the initial critic network are adjusted based on the model loss until the preset stopping condition is met to obtain the trained actor network.
[0031] In a second aspect, the present application also provides a workpiece sorting device, comprising:
[0032] An acquisition module, used to acquire a workpiece state matrix of a workpiece to be processed and a material frame state matrix corresponding to each of a plurality of material frames to be processed, and encode the workpiece state matrix to obtain a workpiece state vector, and encode a plurality of material frame state matrices to obtain a material frame state vector;
[0033] A first determination module is used to determine a target material frame from a plurality of material frames to be processed based on a workpiece state vector, a material frame state vector and a pre-trained selection frame decoder, and determine a target state vector corresponding to the target material frame;
[0034] A second determination module is used to determine a target rotation direction based on a workpiece state vector, a target state vector and a pre-trained direction-selective decoder, and to determine a rotated workpiece state vector based on the target rotation direction;
[0035] A third determination module is used to determine a target sorting position among multiple grid positions of a target material frame based on the rotated workpiece state vector, the target state vector and a pre-trained position selection decoder;
[0036] The sorting module is used to sort the workpieces to be processed based on the target material frame and the target sorting position.
[0037] In a third aspect, the present application further provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:
[0038] Obtaining a workpiece state matrix of a workpiece to be processed and a material frame state matrix corresponding to each of a plurality of material frames to be processed, encoding the workpiece state matrix to obtain a workpiece state vector, and encoding a plurality of material frame state matrices to obtain a material frame state vector;
[0039] Based on the workpiece state vector, the material frame state vector and the pre-trained frame selection decoder, a target material frame is determined from multiple material frames to be processed, and a target state vector corresponding to the target material frame is determined;
[0040] Determine the target rotation direction based on the workpiece state vector, the target state vector and a pre-trained direction-selective decoder, and determine the rotated workpiece state vector based on the target rotation direction;
[0041] Based on the rotated workpiece state vector, the target state vector and the pre-trained position selection decoder, a target sorting position is determined among multiple grid positions of the target material frame;
[0042] Sort the workpieces to be processed based on the target material frame and target sorting position.
[0043] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the following steps are implemented:
[0044] Obtaining a workpiece state matrix of a workpiece to be processed and a material frame state matrix corresponding to each of a plurality of material frames to be processed, encoding the workpiece state matrix to obtain a workpiece state vector, and encoding a plurality of material frame state matrices to obtain a material frame state vector;
[0045] Based on the workpiece state vector, the material frame state vector and the pre-trained frame selection decoder, a target material frame is determined from multiple material frames to be processed, and a target state vector corresponding to the target material frame is determined;
[0046] Determine the target rotation direction based on the workpiece state vector, the target state vector and a pre-trained direction-selective decoder, and determine the rotated workpiece state vector based on the target rotation direction;
[0047] Based on the rotated workpiece state vector, the target state vector and the pre-trained position selection decoder, a target sorting position is determined among multiple grid positions of the target material frame;
[0048] Sort the workpieces to be processed based on the target material frame and target sorting position.
[0049] In a fifth aspect, the present application further provides a computer program product, including a computer program, which implements the following steps when executed by a processor:
[0050] Obtaining a workpiece state matrix of a workpiece to be processed and a material frame state matrix corresponding to each of a plurality of material frames to be processed, encoding the workpiece state matrix to obtain a workpiece state vector, and encoding a plurality of material frame state matrices to obtain a material frame state vector;
[0051] Based on the workpiece state vector, the material frame state vector and the pre-trained frame selection decoder, a target material frame is determined from multiple material frames to be processed, and a target state vector corresponding to the target material frame is determined;
[0052] Determine the target rotation direction based on the workpiece state vector, the target state vector and a pre-trained direction-selective decoder, and determine the rotated workpiece state vector based on the target rotation direction;
[0053] Based on the rotated workpiece state vector, the target state vector and the pre-trained position selection decoder, a target sorting position is determined among multiple grid positions of the target material frame;
[0054] Sort the workpieces to be processed based on the target material frame and target sorting position.
[0055] The workpiece sorting method, device, computer equipment, computer-readable storage medium and computer program product obtain the workpiece state matrix of the workpiece to be processed and the frame state matrix corresponding to each of the multiple frames to be processed, encode the workpiece state matrix to obtain the workpiece state vector, and encode the multiple frame state matrices to obtain the frame state vector, which fully considers the state of the workpiece to be processed and each frame to be processed, which is conducive to ensuring that the workpiece is sorted to a suitable frame position, thereby improving the workpiece sorting efficiency; sequentially using the pre-trained frame selection decoder to select boxes, thereby determining the target frame among the multiple frames, and then using the pre-trained direction selection decoder to select the direction, thereby determining the target rotation direction, and then using the pre-trained position selection decoder to select the position, thereby determining the target sorting position among the multiple grid positions of the target frame, and finally sorting the workpiece to be processed based on the target frame and the target sorting position. This method of decomposing a workpiece sorting action into three sub-actions through different decoders reduces the size of the action space and the sorting difficulty, which is conducive to further improving the workpiece sorting efficiency. BRIEF DESCRIPTION OF THE DRAWINGS
[0056] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments of the present application or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.
[0057] FIG1 is a diagram showing an application environment of a workpiece sorting method according to an embodiment;
[0058] FIG2 is a schematic diagram of a workpiece sorting method in one embodiment;
[0059] FIG3 is a structural diagram of an actor network and a critic network in one embodiment;
[0060] FIG4 is a schematic diagram of the overall process of a workpiece sorting method in one embodiment;
[0061] FIG5 is a schematic diagram of a packaging state in one embodiment;
[0062] FIG6 is a structural block diagram of a workpiece sorting device in one embodiment;
[0063] FIG. 7 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION
[0064] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.
[0065] The workpiece sorting method provided in the embodiment of the present application can be applied to the application environment as shown in FIG1 . Among them, the terminal 102 communicates with the robot arm 104 through the network. After being processed by the cutting machine, the steel plate is cut into a variety of different workpieces to be processed in the cutting area. These workpieces to be processed are then transported by a conveyor belt, and after sanding and other process treatments, they finally arrive at the workpiece sorting area. In the workpiece sorting area, there are multiple identical material frames to be processed. The robot arm 104 is used to remove the workpieces to be processed from the conveyor belt based on the instructions of the terminal 102 and accurately sort them to the target sorting position in the target material frame.
[0066] Terminal 102 obtains the workpiece status matrix of the workpiece to be processed and the material frame state matrix corresponding to each of the multiple material frames to be processed , and encode the workpiece state matrix to obtain the workpiece state vector , encode multiple material frame state matrices to obtain the material frame state vector ; Based on the workpiece state vector, the material frame state vector and the pre-trained selection frame decoder, the target material frame is determined from multiple material frames to be processed, and the target state vector corresponding to the target material frame is determined ; Based on the workpiece state vector, the target state vector and the pre-trained direction-selective decoder, the target rotation direction is determined, and the rotated workpiece state vector is determined based on the target rotation direction ; Based on the rotated workpiece state vector, the target state vector and the pre-trained position selection decoder, the target sorting position is determined in multiple grid positions of the target material frame ; Sorting the workpieces to be processed based on the target material frame and the target sorting position. Among them, the terminal 102 can be, but is not limited to, various personal computers, laptops, smart phones, tablet computers, Internet of Things devices and portable wearable devices. The Internet of Things devices can be smart speakers, smart TVs, smart air conditioners, smart car devices, projection devices, etc. Portable wearable devices can be smart watches, smart bracelets, head-mounted devices, etc. The head-mounted devices can be virtual reality (VR) devices, augmented reality (AR) devices, smart glasses, etc.
[0067] In an exemplary embodiment, as shown in FIG2 , a workpiece sorting method is provided, which is described by taking the method applied to the terminal 102 in FIG1 as an example, and includes the following steps 201 to 205. Among them:
[0068] Step 201 , obtaining a workpiece state matrix of a workpiece to be processed and a material frame state matrix corresponding to each of a plurality of material frames to be processed, encoding the workpiece state matrix to obtain a workpiece state vector, and encoding a plurality of material frame state matrices to obtain a material frame state vector.
[0069] The workpieces to be processed are continuously transported to the workpiece sorting area through the conveyor belt. The workpieces to be processed refer to the workpieces that need to be sorted at present. For example, it can be a steel plate workpiece after cutting a whole steel plate, or it can be a workpiece in other fields. The size of each workpiece to be processed can be the same or different. The material frame to be processed refers to the material frame used to hold the workpieces to be processed. The material frame to be processed can be an empty material frame, or it can be a material frame loaded with workpieces.
[0070] The workpiece state matrix is a three-dimensional matrix determined based on the size information and type information of the workpiece to be processed. For example, the size information of the workpiece to be processed includes length, width and height. Since the thickness of the steel plate is almost the same during the unit operation of steel plate blanking, it can be considered that the heights of different workpieces to be processed are the same. The terminal can determine the workpiece state matrix based on the length, width and type information of the workpiece to be processed. In some embodiments, the size information of the workpiece to be processed can be the size information of the minimum circumscribed rectangle of the workpiece to be processed.
[0071] The material frame state matrix is a two-dimensional matrix determined based on the state information of the material frame to be processed. In some embodiments, the material frame to be processed can be divided into a plurality of grid positions, and each workpiece can occupy at least one grid position after being loaded into the material frame to be processed. The terminal can determine the height of the loaded workpiece in each grid position and the workpiece type of the loaded workpiece based on the information of the loaded workpiece in each grid position. The terminal determines the material frame state matrix based on the height of the loaded workpiece and the workpiece type of the loaded workpiece corresponding to each grid position in the material frame to be processed.
[0072] The terminal uses a pre-trained encoder to encode the workpiece state matrix and multiple material frame state matrices. For example, the terminal inputs the workpiece state matrix into the encoder to obtain a workpiece state vector, and inputs multiple material frame state matrices into the encoder to obtain a material frame state vector. In some embodiments, the encoder can use the encoder structure in the Transformer model.
[0073] Step 202 , based on the workpiece state vector, the material frame state vector and the pre-trained selection frame decoder, determine a target material frame from a plurality of material frames to be processed, and determine a target state vector corresponding to the target material frame.
[0074] The marquee decoder refers to a pre-trained decoder, for example, a decoder structure in a Transformer model may be used. In some embodiments, an actor network is pre-trained in the terminal, and the actor network may be obtained by training using a Transformer model. The actor network includes an encoder and multiple decoders, and the network parameters of the multiple decoders are different, wherein the multiple decoders include a marquee decoder.
[0075] The terminal inputs the workpiece state matrix into the encoder in the pre-trained actor network to obtain the workpiece state vector, inputs multiple material frame state matrices into the encoder in the pre-trained actor network to obtain the material frame state vector, and then inputs the workpiece state vector and the material frame state vector into the selection frame decoder to obtain the decoding result output by the selection frame decoder. The selection frame decoder is used to decode the workpiece state vector and the material frame state vector, so that the terminal can use the decoding results to perform selection frame operations on multiple material frames to be processed.
[0076] The target material frame is determined by using a selection frame decoder in combination with the status of the workpiece to be processed and each material frame to be processed. This selection frame operation based on a pre-trained selection frame decoder is conducive to quickly determining the target material frame, thereby improving the workpiece sorting efficiency.
[0077] In some embodiments, the target state vector may be determined by: determining a target material frame state matrix corresponding to the target material frame from a plurality of material frame state matrices, and inputting the target material frame state matrix into a first encoder to obtain a target state vector corresponding to the target material frame. The first encoder may be different from an encoder that encodes the plurality of material frame state matrices.
[0078] Step 203, determining a target rotation direction based on the workpiece state vector, the target state vector and a pre-trained direction-selective decoder, and determining a rotated workpiece state vector based on the target rotation direction.
[0079] The direction-selective decoder refers to a pre-trained decoder, for example, the decoder structure in the Transformer model can be used. The multiple decoders in the actor network also include a direction-selective decoder. The terminal inputs the workpiece state vector and the target state vector into the direction-selective decoder in the actor network to obtain the decoding result output by the direction-selective decoder. The direction-selective decoder is used to decode the workpiece state vector and the target state vector, so that the terminal can use the decoding result to select the rotation direction and obtain the target rotation direction.
[0080] The target rotation direction refers to the rotation direction when the workpiece to be processed is loaded into the material frame to be processed. According to the requirements, the target rotation direction can include multiple, for example, it can be a six-axis rotation direction.
[0081] The method for determining the rotated workpiece state vector may be: determining the rotated workpiece state matrix according to the target rotation direction, inputting the rotated workpiece state matrix into a second encoder to obtain the rotated workpiece state vector. The second encoder may be different from the encoder that encodes the workpiece state matrix.
[0082] Step 204 , based on the rotated workpiece state vector, the target state vector and the pre-trained position selection decoder, determine the target sorting position among the multiple grid positions of the target material frame.
[0083] Among them, the position selection decoder refers to a pre-trained decoder, for example, the decoder structure in the Transformer model can be used. The multiple decoders in the actor network also include a position selection decoder. The terminal inputs the rotated workpiece state vector and the target state vector into the position selection decoder in the actor network to obtain the decoding result output by the position selection decoder. The position selection decoder is used to decode the rotated workpiece state vector and the target state vector, so that the terminal can use the decoding result to select the position in the target material frame to obtain the target sorting position.
[0084] The target sorting position is composed of at least one grid position. The terminal adopts a position selection decoder and can determine the target sorting position in combination with the status of the workpiece after rotation and the status of the target material frame. This position selection operation based on the pre-trained position selection decoder is conducive to quickly determining the target sorting position, thereby improving the workpiece sorting efficiency.
[0085] Step 205 , sorting the workpieces to be processed based on the target material frame and the target sorting position.
[0086] The terminal generates a control command based on the target material frame and the target sorting position, and sends the control command to the robot controller. The control command instructs the robot controller to control the robot to load the workpiece to be processed into the target sorting position in the target material frame according to the control command, thereby realizing real-time sorting of the workpiece to be processed. The workpieces to be processed that are continuously transmitted from the conveyor belt can be sorted according to this workpiece sorting method. The embodiment of the present application does not specifically limit the type of workpiece to be processed. Therefore, the method can be extended to boxes of larger sizes and a larger number of boxes.
[0087] In the above workpiece sorting method, by obtaining the workpiece state matrix of the workpiece to be processed and the material frame state matrix corresponding to each of the multiple material frames to be processed, and encoding the workpiece state matrix to obtain the workpiece state vector, and encoding the multiple material frame state matrices to obtain the material frame state vector, the state of the workpiece to be processed and each material frame to be processed is fully considered, which is conducive to ensuring that the workpiece is sorted to a suitable material frame position, thereby improving the workpiece sorting efficiency; the pre-trained frame selection decoder is used in turn to select the box, so as to determine the target material frame among the multiple material frames, and then the pre-trained direction selection decoder is used to select the direction, so as to determine the target rotation direction, and then the pre-trained position selection decoder is used to select the position, so as to determine the target sorting position among the multiple grid positions of the target material frame, and finally the workpiece to be processed is sorted based on the target material frame and the target sorting position. This method of decomposing a workpiece sorting action into three sub-actions through different decoders reduces the size of the action space and the sorting difficulty, which is conducive to further improving the workpiece sorting efficiency.
[0088] In an exemplary embodiment, based on a workpiece state vector, a material frame state vector and a pre-trained selection frame decoder, a target material frame is determined from multiple material frames to be processed, including: using a pre-trained selection frame decoder to decode the workpiece state vector and the material frame state vector to obtain a first decoding result; each element in the first decoding result is used to characterize the probability of sorting the workpiece to be processed into multiple material frames to be processed; and according to the first decoding result and the material frame mask, a target material frame is determined from multiple material frames to be processed.
[0089] Among them, the input of the marquee encoder usually includes a query vector Q, a key vector K and a numerical vector V. The terminal takes the workpiece state vector as Q and the material frame state vector as K and V, and then inputs Q, K, and V into the marquee decoder in the pre-trained actor network to obtain the first decoding result output by the marquee decoder.
[0090] Each element in the first decoding result corresponds to the probability of sorting the workpiece to be processed into multiple material frames to be processed. Each element in the material frame mask is used to characterize the loading status of each material frame to be processed for the workpiece to be processed. For example, if the material frame to be processed is in a loading status, the element at the corresponding position in the material frame mask is 1, indicating that the workpiece to be processed can be loaded into the material frame to be processed; if the material frame to be processed is in a non-loading status, the element at the corresponding position in the material frame mask is 0, indicating that the workpiece to be processed cannot be loaded into the material frame to be processed.
[0091] In some embodiments, the material frame mask is determined based on the status information of the workpiece to be processed and the status information of each material frame to be processed during the current sorting, and the status information of the workpiece to be processed may include the size information and type information of the workpiece to be processed. The material frame mask determination step includes: the terminal determines in sequence whether each material frame to be processed can be loaded with the workpiece to be processed based on the status information of the workpiece to be processed and the status information of each material frame to be processed, and if so, sets the element corresponding to the material frame to be processed in the material frame mask to 1, and if not, sets the element corresponding to the material frame to be processed in the material frame mask to 0.
[0092] In some embodiments, the terminal may multiply the first decoding result and the frame mask to screen out a target frame, where the target frame is a frame to be processed that is in a loadable state.
[0093] In this embodiment, the workpiece state vector and the material frame state vector are decoded by a pre-trained selection frame decoder, so as to determine the loading state of each material frame to be processed, and then determine the target material frame. Since the target material frame is determined based on the current state of the workpiece and the current state of the material frame, it is conducive to real-time adjustment of the sorting strategy. When the workpiece to be processed arrives at the sorting position, the packing strategy can be quickly determined to ensure that no workpiece is lost. In addition, the method of using a pre-trained selection frame decoder to determine the target material frame is conducive to further improving the workpiece sorting efficiency.
[0094] In an exemplary embodiment, a target rotation direction is determined based on a workpiece state vector, a target state vector and a pre-trained direction-selective decoder, including: using a pre-trained direction-selective decoder to decode the workpiece state vector and the target state vector to obtain a second decoding result; each element in the second decoding result is used to characterize the probability that the workpiece to be processed can be sorted into the target material frame after being rotated in a preset rotation direction; and determining the target rotation direction based on the second decoding result and the direction mask.
[0095] Among them, the input of the selective encoder usually includes a query vector Q, a key vector K and a numerical vector V. The terminal takes the workpiece state vector as Q and the target state vector as K and V, thereby inputting Q, K, and V into the selective decoder in the pre-trained actor network to obtain a second decoding result output by the selective decoder.
[0096] The preset rotation direction is the rotation direction of the workpiece, for example, the preset rotation direction can be a six-axis rotation direction. Each element in the second decoding result corresponds to the probability of each preset rotation direction. In some embodiments, the workpiece to be processed cannot be directly loaded into the target material frame, but can be loaded into the target material frame after rotating in a certain direction. Therefore, the workpiece to be processed can be rotated to improve the utilization rate of the material frame to be processed.
[0097] Each element in the direction mask is used to represent the selectable state of each preset rotation direction. For example, if the preset rotation direction is selectable, the element at the corresponding position in the direction mask is 1, indicating that the workpiece to be processed can be loaded into the target material frame by rotating it according to the preset rotation direction; if the preset rotation direction is not selectable, the element at the corresponding position in the direction mask is 0, indicating that the workpiece to be processed cannot be loaded into the target material frame by rotating it according to the preset rotation direction.
[0098] In some embodiments, the direction mask is determined based on the state information of the workpiece to be processed and the state information of the target material frame during the current sorting. The step of determining the direction mask includes: for each preset rotation direction, the terminal rotates the workpiece to be processed according to the preset rotation direction to obtain the state information of the workpiece after rotation, and determines whether the target material frame can be loaded with the rotated workpiece to be processed according to the state information of the workpiece after rotation and the state information of the target material frame. If yes, the element corresponding to the preset rotation direction in the direction mask is set to 1, and if not, the element corresponding to the preset rotation direction in the direction mask is set to 0.
[0099] In some embodiments, the terminal can multiply the second decoding result and the direction mask to filter out the target rotation direction, which is a preset rotation direction in a rotatable state. Rotating the workpiece to be processed according to the target rotation direction can ensure that the workpiece to be processed is loaded into the target material frame.
[0100] In this embodiment, the workpiece state vector and the target state vector are decoded by a pre-trained direction-selective decoder, thereby determining the probability of each preset rotation direction, and then determining the target rotation direction. Since the target material frame is determined first, and then the target rotation direction is determined based on the target material frame, it is beneficial to improve the efficiency of determining the target rotation direction and ensure the real-time sorting of the workpiece. In addition, the method of using a pre-trained direction-selective decoder to determine the target rotation direction is beneficial to further improve the efficiency of workpiece sorting.
[0101] In an exemplary embodiment, based on the rotated workpiece state vector, the target state vector and a pre-trained position selection decoder, a target sorting position is determined among multiple grid positions of a target material frame, including: using a pre-trained position selection decoder to decode the rotated workpiece state vector and the target state vector to obtain a third decoding result; each element in the third decoding result is used to characterize the probability of sorting the workpiece to be processed to each grid position in the target material frame; and according to the third decoding result and the position mask, the target sorting position is determined among multiple grid positions of the target material frame.
[0102] Among them, the input of the position selection encoder usually includes a query vector Q, a key vector K and a numerical vector V. The terminal uses the rotated workpiece state vector as Q and the target state vector as K and V, thereby inputting Q, K, and V into the position selection decoder in the pre-trained actor network to obtain the third decoding result output by the position selection decoder.
[0103] Each element in the third decoding result corresponds to the probability of each grid position in the target material frame. Each element in the position mask is used to characterize the loadable state of each grid position in the target material frame for the workpiece to be processed. For example, if the grid position is in a loadable state, the element at the corresponding position in the position mask is 1, indicating that the grid position meets the preset constraint conditions; if the grid position is in an unloadable state, the element at the corresponding position in the position mask is 0, indicating that the grid position does not meet the preset constraint conditions. Exemplarily, the preset constraints may include: the same workpieces are placed in the same stack, and the current grid position can be used to load the workpiece to be processed. The current grid position can be used to load the workpiece to be processed means that if the current grid position is loaded with the workpiece to be processed, the area occupied by the workpiece to be processed will not overlap with other loaded workpiece areas.
[0104] In some embodiments, the position mask is determined based on the status information of the workpiece to be processed and the status information of each material frame to be processed during the current sorting. The step of determining the position mask includes: the terminal determines in sequence whether each grid position in the target material frame can meet the preset constraint condition based on the status information of the workpiece to be processed and the status information of the target material frame, and if so, sets the element corresponding to the grid position in the position mask to 1, and if not, sets the element corresponding to the grid position in the position mask to 0.
[0105] In some embodiments, the terminal may multiply the third decoding result and the position mask to screen out the target sorting position, where the target sorting position is a grid position in the target material frame that is in a loadable state.
[0106] In this embodiment, the rotated workpiece state vector and the target state vector are decoded by a pre-trained position selection decoder, so as to determine the loadable state of each grid position in the target material frame, and then determine the target sorting position. Since the target material frame and the target rotation direction of the workpiece are determined in advance, it is beneficial to improve the efficiency of determining the target sorting position; at the same time, the sorting strategy can be adjusted in real time, and the packing strategy can be quickly determined when the workpiece to be processed arrives at the sorting position to ensure that there is no loss of workpieces. The method of using a pre-trained position selection decoder to determine the target sorting position is beneficial to further improve the efficiency of workpiece sorting. In addition, a Transformer model based on a conditional query mechanism is used, and the output of the previous sub-action is used as a query vector and input into the next sub-action network. The conditional query mechanism can make up for the information missing problem between different sub-actions, so that the entire sorting process is a completely observable Markov problem, that is, the action state at the next moment depends only on the current material frame state information and the input workpiece state information, and has nothing to do with the state at the past moment. If a workpiece sorting process is decomposed into three sub-actions, the input state of each sub-action can be determined. For example, the input of the box selection action is the status information of each material frame to be processed and the status information of the workpiece to be processed; the input of the position selection sub-action is the status information of the selected target material frame and the status information of the workpiece to be processed; the input of the position selection sub-action is the status information of the selected target material frame and the status information of the rotated workpiece. This action decomposition mechanism reduces the amount of calculation, which is beneficial to improving the efficiency of workpiece sorting, and allows the algorithm to be extended to large-scale box packing problems.
[0107] In an exemplary embodiment, after sorting the workpieces to be processed based on the target material frame and the target sorting position, it also includes: acquiring real-time material frame information and real-time workpiece information, and updating the material frame mask, direction mask, and position mask according to the real-time material frame information and the real-time workpiece information; sorting the workpieces based on the updated material frame mask, the updated direction mask, and the updated position mask.
[0108] The real-time material frame information refers to the information of each material frame to be processed at the current acquisition moment, and the real-time material frame information corresponding to each material frame to be processed includes the height of the loaded workpieces in each grid position of the material frame to be processed and the workpiece type of the loaded workpieces.
[0109] Real-time workpiece information refers to the size information of the workpiece to be sorted at the current collection time, including the length, width, height and other size information of the workpiece.
[0110] As the workpiece sorting proceeds, each workpiece to be processed is continuously sorted into each material frame to be processed, and the real-time material frame information corresponding to each material frame to be processed is constantly changing. The terminal can continuously update the material frame mask, direction mask, and position mask according to the real-time material frame information and the real-time workpiece information, and obtain the updated material frame mask, updated direction mask, and updated position mask respectively. That is, the material frame mask, direction mask, and position mask are dynamically changing in real time.
[0111] In the subsequent workpiece sorting process, the workpiece sorting can be performed based on the updated material frame mask, the updated direction mask and the updated position mask.
[0112] In this embodiment, the material frame mask, direction mask, and position mask are updated according to the real-time material frame information and the real-time workpiece information, thereby ensuring that the sorting of each workpiece to be processed is based on the latest accurate information, which is conducive to determining the accurate sorting position and improving the sorting efficiency.
[0113] In an exemplary embodiment, a workpiece state matrix and a material frame state matrix are processed by a trained actor network, wherein the actor network includes a pre-trained frame decoder, a pre-trained direction decoder, and a pre-trained position decoder; the training steps of the actor network include: obtaining workpiece state matrix samples of a workpiece sample to be processed and material frame state matrix samples corresponding to a plurality of material frame samples to be processed, encoding the workpiece state matrix samples to obtain workpiece state vector samples, encoding a plurality of material frame state matrix samples to obtain material frame state vector samples; determining a target material frame sample from a plurality of material frame samples to be processed based on the workpiece state vector, the material frame state vector, and the to-be-trained frame decoder in the initial actor network, and determining a target state vector sample corresponding to the target material frame sample; and obtaining a target state vector sample corresponding to the target material frame sample based on the workpiece state vector sample, the target state vector sample, and the initial The direction selection decoder to be trained in the actor network determines the target rotation direction sample, and determines the rotated workpiece state vector sample based on the target rotation direction sample; determines the target sorting position sample from multiple grid position samples of the target material frame sample based on the rotated workpiece state vector sample, the target state vector sample and the position selection decoder to be trained in the initial actor network; obtains the comment value output by the initial critic network based on the workpiece state vector, the material frame state vector and the initial critic network; sorts the workpiece samples to be processed based on the target material frame sample and the target sorting position sample, and determines the reward value according to the sorted material frame information and the preset reward function; calculates the model loss based on the difference between the reward value and the comment value, and adjusts the model parameters of the initial actor network and the initial critic network based on the model loss until the preset stop condition is met, thereby obtaining a trained actor network.
[0114] Among them, taking the unloading scenario of the production line as an example, the terminal controls the robotic arm to sort the workpieces cut from the whole steel plate into the box in real time. The deep reinforcement learning algorithm based on actor-critic can be used, and the overall network structure of encoder-decoder is adopted to optimize the sorting strategy of the workpieces. Among them, the actor network and the critic network can be constructed using the Transformer model. The trained actor network includes a pre-trained encoder, a pre-trained box decoder, a pre-trained direction decoder, and a pre-trained position decoder. The trained critic network includes a pre-trained encoder and a pre-trained comment decoder. As shown in Figure 3, it is a structural diagram of the actor network and the critic network in one embodiment.
[0115] The terminal inputs the workpiece state matrix samples into the decoder in the initial actor network for encoding to obtain workpiece state vector samples, and inputs multiple material frame state matrix samples into the decoder in the initial actor network for encoding to obtain material frame state vector samples.
[0116] The terminal then uses the workpiece state vector sample as Q and the material frame state vector as K and V, and inputs them into the to-be-trained selection frame decoder in the initial actor network to obtain the decoding result output by the to-be-trained selection frame decoder, and based on the decoding result, determines the target material frame sample from multiple to-be-processed material frame samples, and determines the target state vector sample corresponding to the target material frame sample.
[0117] The terminal then uses the workpiece state vector sample as Q and the target state vector sample as K and V, and inputs them into the direction-selective decoder to be trained in the initial actor network to determine the target rotation direction sample, and determines the rotated workpiece state vector sample based on the target rotation direction sample.
[0118] The terminal then uses the rotated workpiece state vector sample as Q and the target state vector sample as K and V, and inputs them into the position decoder to be trained in the initial actor network, thereby determining the target sorting position sample from multiple grid position samples of the target material frame sample.
[0119] The terminal inputs the workpiece state vector and the material frame state vector into the initial critic network to obtain the comment value output by the initial critic network. The sorted material frame information is the material frame information obtained by sorting the workpiece samples to be processed according to the target material frame samples and the target sorting position samples.
[0120] Unlike supervised learning, reinforcement learning requires learning from experience and trying to find strategies to obtain more rewards. Therefore, how to design the reward function is a key factor in the learning process. In the workpiece sorting problem, a common goal is to minimize the total surface area or maximize the space occupancy rate. In order to adapt to the two-dimensional online multi-box packing problem, the preset reward function proposed in the embodiment of the present application can be:
[0121]
[0122]
[0123]
[0124] in, It represents the change rate of the effective use area of the material frame after the workpiece to be processed is placed in the material frame to be processed at time t; It is an integer variable of 0 or 1. When the same type of workpieces to be processed are loaded on the same stack, the value is 1, otherwise it is 0. Aims to maximize the space utilization of the material frame; Indicates the distance between the target material frame and the starting position of the robot arm. The aim is to use material frames that are as close as possible to reduce the ineffective movement distance of the robot arm. , , They are , , The proportion coefficient of .
[0125] The terminal calculates the model loss based on the difference between the reward value and the comment value and the preset loss function, and adjusts the model parameters of the initial actor network and the initial critic network based on the model loss until the preset stop condition is met to obtain a trained actor network.
[0126] In this embodiment, by adopting the actor-critic based deep reinforcement learning algorithm, the overall network structure of encoder-decoder is used for model training, and the actor network obtained by training can realize the optimization of the sorting strategy for workpieces. The neural network model is constructed based on the Transformer model combined with the conditional query mechanism, and the output of the previous sub-action is used as the query vector and input into the next sub-action network. The conditional query mechanism can make up for the information missing problem between different sub-actions, so that the entire sorting process is a completely observable Markov problem. The action decomposition mechanism reduces the amount of calculation, so that the algorithm can be extended to large-scale packing problems. The trained network model can adjust the sorting strategy in real time according to the current workpiece status information and the status information of the material frame. When the workpiece arrives at the sorting position, the packing strategy can be quickly determined to ensure that there is no loss of workpieces. Compared with the traditional workpiece sorting algorithm, the embodiment of the present application can meet the real-time packing constraints and optimize the utilization rate of the material frame while ensuring the stacking order and stacking height of the workpieces, so that the space utilization rate of the material frame is higher.
[0127] To explain the workpiece sorting method and effect in this solution in detail, a most detailed embodiment is described below:
[0128] In the production line unloading scenario, the terminal controls the robotic arm to sort the workpieces into the box in real time after cutting the whole steel plate. The solution of the embodiment of the present application focuses on solving the problem of optimizing the real-time packing strategy of the workpiece after the whole plate is cut. The size information of the workpiece to be processed is calculated by the relevant algorithm after the workpiece passes through the darkroom to obtain the minimum circumscribed rectangle information. In addition, since the thickness of the steel plate is almost the same during the unit operation, the workpiece sorting process is abstracted into a two-dimensional online multi-box packing problem. The packing process of a workpiece can be decomposed into three sub-actions: box selection, direction selection, and position selection. These three sub-actions are executed in sequence, so a neural network model is constructed based on the Transformer model combined with a conditional query mechanism. The output of the previous sub-action is used as a query vector and input into the next sub-action network. Subsequently, the network structure is optimized in combination with the Actor-Critic reinforcement learning framework to achieve the optimization solution of the two-dimensional online multi-box packing process. Since the conditional query mechanism can make up for the information missing problem between different sub-actions, the entire sorting process is a completely observable Markov problem. The action decomposition mechanism reduces the amount of calculation, so that the algorithm can be extended to large-scale packing problems. Experimental results show that the scheme is highly feasible and effective in practical applications.
[0129] As shown in FIG4 , a schematic diagram of the overall process of a workpiece sorting method in an embodiment is shown. A neural network model is established based on a Transformer model with a conditional query mechanism, and the network is trained using the actor-critic algorithm to obtain a trained actor network and a critic network. After being polished and other processes, the workpiece is transported to the workpiece sorting area, which includes a plurality of material frames to be processed in which workpieces are stored. According to the size information of the first workpiece to be processed on the conveyor belt and the state information of all material frames, a pre-trained frame selection decoder is used to perform a frame selection operation to determine the target material frame; according to the information of the target material frame and the information of the workpiece to be processed, a pre-trained direction selection decoder is used to perform a direction selection operation to determine the target rotation direction; according to the rotated workpiece state vector and the information of the target material frame, a pre-trained position selection decoder is used to perform a position selection operation to determine the target sorting position. The robot arm is controlled to remove the workpiece to be processed from the conveyor belt and load it into the target sorting position of the target material frame, and then the next workpiece to be processed is sorted until the material frame cannot meet the constraint conditions to load the current workpiece to be processed, and the workpiece sorting is terminated.
[0130] (1) State space: The key point to be solved in the two-dimensional online multi-box packing problem is the decision of each workpiece sorting step to be processed. A workpiece sorting step is decomposed into three sub-actions: box selection: select the target material box for loading the current workpiece to be processed; direction selection: select the rotation direction of the workpiece to be processed; position selection: select the position of the workpiece to be processed in the target material box. These three sub-actions are ordered and interrelated, and the output of the previous sub-action serves the next sub-action. In the common method, a network is used to output three sub-actions at the same time, and then the Cartesian product of the sub-actions is taken as the final output. The strategy of the embodiment of the present application not only reduces the size of the action space, but also makes the entire packing step a completely observable Markov process. Although a workpiece sorting process can be regarded as a complete Markov process, it is not a completely observable Markov process for each sub-action, so a conditional query mechanism based on the Transformer model is used to make up for the information difference between different actions. As shown in Figure 5, it is a schematic diagram of the packing state in an embodiment. The embodiment of the present application defines 5 packing states: Indicates The length, width and type of workpiece to be processed; Indicates the artifact to be processed To be installed in A frame of materials to be processed; Indicates the current workpiece to be processed The state after being rotated; Indicates the artifact to be processed Load into the material frame to be processed In Location; Indicates The status of the material frame to be processed, Indicates Height map of the material frame to be processed, Indicates A matrix diagram of the types of workpieces loaded in the waiting material frame.
[0131] (2) Action space: The embodiment of the present application decomposes a workpiece sorting process into three interrelated sub-actions, each of which has its own action space. Among them, the action space of the box selection sub-action is the number N of the material boxes to be processed; for example, the workpiece to be processed is only allowed to go around The axis rotates, so the action space of the direction selection sub-action is the two rotation directions of the workpiece to be processed; since the continuous action space has high complexity in solving, the embodiment of the present application discretizes the continuous space and discretizes the bottom area of the frame to be processed into n×n grid positions, so the size of the position selection sub-action space is n×n.
[0132] (3) Reward function: To adapt to the two-dimensional online multi-box packing problem, the preset reward function proposed in the embodiment of the present application can be:
[0133]
[0134]
[0135]
[0136] in, It represents the change rate of the effective use area of the material frame after the workpiece to be processed is placed in the material frame to be processed at time t; It is an integer variable of 0 or 1. When the same type of workpieces to be processed are loaded on the same stack, the value is 1, otherwise it is 0. Aims to maximize the space utilization of the material frame; Indicates the distance between the target material frame and the starting position of the robot arm. The aim is to use material frames that are as close as possible to reduce the ineffective movement distance of the robot arm. , , They are , , The proportion coefficient of .
[0137] (4) Actor network: The actor network is divided into two parts: encoder and decoder. The main body of the network is constructed based on the Transformer model and the CNN network. Since the model input of the embodiment of the present application is the state matrix diagram of the workpiece and the material frame, the encoder constructed using the CNN network is selected to extract features from the input matrix; the decoder constructed based on the conditional query mechanism and the Transformer model is applicable to the strategy decision of each sub-action and is suitable for connecting the entire workpiece sorting sub-action in series.
[0138] The actor network mainly includes an encoder and three query encoders. The encoder is mainly used to integrate the information of the material frame and the workpiece in the two-dimensional online multi-box packing problem environment; the decoder is a query decoder, which mainly selects the action strategy according to the input query object based on the query mechanism. Since it is an online packing problem, position information is not required, so the position encoder in the original Transformer model is discarded. For the two-dimensional online multi-box packing problem, the model needs to perform three sub-actions in sequence, namely, the selection of the material frame, the selection of the rotation direction, and the selection of the position. For each sub-action, an independent decoder is used to select the strategy. Different decoders embed different query condition feature information according to their different sub-action types, and then derive the current action strategy from the integrated environment information output by the encoder. The box selection decoder selects the target material frame according to the size information of the workpiece to be processed; the direction selection decoder selects the target rotation direction of the workpiece to be processed according to the size information of the workpiece to be processed and the state of the target material frame; the position selection decoder selects the target sorting position according to the size information of the workpiece to be processed after rotation and the information of the target material frame. Probabilistic sampling is performed on the output of each decoder to obtain the corresponding sub-action, and the mask generated by the mask module is used to mask impossible actions.
[0139] (5) Critic network: The critic network is also composed of two parts: an encoder and a decoder. In order to reduce the size of network parameters, the critic network and the actor network can share the same encoder. The network structure of the decoder is similar to that of the actor network. Both are built based on the Transformer model, and the position encoder is discarded.
[0140] The above-mentioned workpiece sorting method obtains the workpiece state matrix of the workpiece to be processed and the material frame state matrix corresponding to each of the multiple material frames to be processed, and encodes the workpiece state matrix to obtain the workpiece state vector, and encodes the multiple material frame state matrices to obtain the material frame state vector, which fully considers the state of the workpiece to be processed and each material frame to be processed, which is conducive to ensuring that the workpiece is sorted to a suitable material frame position, thereby improving the workpiece sorting efficiency; the pre-trained frame selection decoder is used in turn to select the box, so as to determine the target material frame among the multiple material frames, and then the pre-trained direction selection decoder is used to select the direction, so as to determine the target rotation direction, and then the pre-trained position selection decoder is used to select the position, so as to determine the target sorting position among the multiple grid positions of the target material frame, and finally the workpiece to be processed is sorted based on the target material frame and the target sorting position. This method of decomposing a workpiece sorting action into three sub-actions through different decoders reduces the size of the action space and the sorting difficulty, which is conducive to further improving the workpiece sorting efficiency. At the same time, the deep reinforcement learning algorithm is used to solve the two-dimensional online multi-box packing problem. Compared with the meta-heuristic algorithm, which starts from scratch and performs a large number of iterations to find the solution each time, the model trained by this method is universal for different workpiece input sequences, and the model has high universality; compared with the heuristic algorithm widely used in the box selection step in the current multi-box packing problem, the embodiment of the present application uses a pre-trained network to select a material frame suitable for loading the current workpiece in the box selection step. Using the network for box selection operations can not only reduce the running time of the algorithm, but also reduce the coupling degree. The conditional query mechanism based on the Transformer model solves the problem of information loss of each sub-action, so that each sub-action in the process of workpiece sorting is completely Markov observable; using different decoders to obtain each sub-action reduces the size of the action space and the difficulty of packing, so this method can be extended to larger boxes and more boxes.
[0141] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.
[0142] Based on the same inventive concept, the embodiment of the present application also provides a workpiece sorting device for implementing the workpiece sorting method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more workpiece sorting device embodiments provided below can refer to the limitations on the workpiece sorting method above, and will not be repeated here.
[0143] In an exemplary embodiment, as shown in FIG6 , a workpiece sorting device 600 is provided, comprising: an acquisition module 610, a first determination module 620, a second determination module 630, a third determination module 640 and a sorting module 650, wherein:
[0144] An acquisition module 610 is used to acquire a workpiece state matrix of a workpiece to be processed and a material frame state matrix corresponding to each of a plurality of material frames to be processed, and encode the workpiece state matrix to obtain a workpiece state vector, and encode a plurality of material frame state matrices to obtain a material frame state vector;
[0145] A first determination module 620 is used to determine a target material frame from a plurality of material frames to be processed based on a workpiece state vector, a material frame state vector and a pre-trained selection frame decoder, and determine a target state vector corresponding to the target material frame;
[0146] A second determination module 630 is used to determine a target rotation direction based on the workpiece state vector, the target state vector and a pre-trained direction-selective decoder, and determine a rotated workpiece state vector based on the target rotation direction;
[0147] A third determination module 640 is used to determine a target sorting position among a plurality of grid positions of a target material frame based on the rotated workpiece state vector, the target state vector and a pre-trained position selection decoder;
[0148] The sorting module 650 is used to sort the workpieces to be processed based on the target material frame and the target sorting position.
[0149] The above-mentioned workpiece sorting device obtains the workpiece state matrix of the workpiece to be processed and the material frame state matrix corresponding to each of the multiple material frames to be processed, and encodes the workpiece state matrix to obtain the workpiece state vector, and encodes the multiple material frame state matrices to obtain the material frame state vector, which fully considers the state of the workpiece to be processed and each material frame to be processed, which is beneficial to ensure that the workpiece is sorted to a suitable material frame position, thereby improving the workpiece sorting efficiency; the pre-trained frame selection decoder is used in turn to select the box, so as to determine the target material frame among the multiple material frames, and then the pre-trained direction selection decoder is used to select the direction, so as to determine the target rotation direction, and then the pre-trained position selection decoder is used to select the position, so as to determine the target sorting position among the multiple grid positions of the target material frame, and finally the workpiece to be processed is sorted based on the target material frame and the target sorting position. This method of decomposing a workpiece sorting action into three sub-actions through different decoders reduces the size of the action space and the sorting difficulty, which is beneficial to further improve the workpiece sorting efficiency.
[0150] In one embodiment, based on the workpiece state vector, the material frame state vector and a pre-trained selection frame decoder, a target material frame is determined from multiple material frames to be processed. The first determination module 620 is also used to: use a pre-trained selection frame decoder to decode the workpiece state vector and the material frame state vector to obtain a first decoding result; each element in the first decoding result is used to characterize the probability of sorting the workpiece to be processed into multiple material frames to be processed; and determine the target material frame from multiple material frames to be processed based on the first decoding result and the material frame mask.
[0151] In one embodiment, the target rotation direction is determined based on the workpiece state vector, the target state vector and a pre-trained direction selection decoder, and the second determination module 630 is also used to: use the pre-trained direction selection decoder to decode the workpiece state vector and the target state vector to obtain a second decoding result; each element in the second decoding result is used to characterize the probability that the workpiece to be processed can be sorted into the target material frame after being rotated in a preset rotation direction; determine the target rotation direction according to the second decoding result and the direction mask.
[0152] In one embodiment, based on the rotated workpiece state vector, the target state vector and a pre-trained position selection decoder, a target sorting position is determined among multiple grid positions of a target material frame. The third determination module 640 is also used to: decode the rotated workpiece state vector and the target state vector using a pre-trained position selection decoder to obtain a third decoding result; each element in the third decoding result is used to characterize the probability of sorting the workpiece to be processed to each grid position in the target material frame; and determine the target sorting position among multiple grid positions of the target material frame according to the third decoding result and the position mask.
[0153] In one embodiment, after sorting the workpieces to be processed based on the target material frame and the target sorting position, the sorting module 650 is also used to: obtain real-time material frame information and real-time workpiece information, and update the material frame mask, direction mask, and position mask according to the real-time material frame information and the real-time workpiece information; and sort the workpieces based on the updated material frame mask, the updated direction mask, and the updated position mask.
[0154] In one embodiment, the workpiece state matrix and the material frame state matrix are processed by a trained actor network, and the actor network includes a pre-trained frame decoder, a pre-trained direction decoder, and a pre-trained position decoder; in terms of the training of the actor network, the workpiece sorting device 600 also includes a training module, and the training module is used to: obtain workpiece state matrix samples of the workpiece sample to be processed and material frame state matrix samples corresponding to each of the multiple material frame samples to be processed, and encode the workpiece state matrix samples to obtain workpiece state vector samples, and encode the multiple material frame state matrix samples to obtain material frame state vector samples; based on the workpiece state vector, the material frame state vector and the to-be-trained frame decoder in the initial actor network, determine the target material frame sample from the multiple material frame samples to be processed, and determine the target state vector sample corresponding to the target material frame sample; based on the workpiece state vector sample, the target The state vector samples and the direction selection decoder to be trained in the initial actor network determine the target rotation direction samples, and determine the rotated workpiece state vector samples based on the target rotation direction samples; based on the rotated workpiece state vector samples, the target state vector samples and the position selection decoder to be trained in the initial actor network, determine the target sorting position samples from multiple grid position samples of the target material frame samples; based on the workpiece state vector, the material frame state vector and the initial critic network, obtain the comment value output by the initial critic network; based on the target material frame samples and the target sorting position samples, sort the workpiece samples to be processed, and determine the reward value according to the sorted material frame information and the preset reward function; calculate the model loss based on the difference between the reward value and the comment value, and adjust the model parameters of the initial actor network and the initial critic network based on the model loss until the preset stop condition is met to obtain the trained actor network.
[0155] Each module in the above workpiece sorting device can be implemented in whole or in part by software, hardware or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each module.
[0156] In an exemplary embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be shown in FIG7. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit, and an input device. The processor, the memory, and the input / output interface are connected via a system bus, and the communication interface, the display unit, and the input device are connected to the system bus via the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner may be implemented through WIFI, a mobile cellular network, near field communication (NFC) or other technologies. When the computer program is executed by the processor, a workpiece sorting method is implemented. The display unit of the computer device is used to form a visually visible picture, which may be a display screen, a projection device, or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the computer device shell, or an external keyboard, touchpad or mouse.
[0157] Those skilled in the art will understand that the structure shown in FIG. 7 is merely a block diagram of a partial structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.
[0158] In an exemplary embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.
[0159] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.
[0160] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.
[0161] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.
[0162] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.
[0163] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.
[0164] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.
Claims
1. A workpiece sorting method, characterized in that: The method comprises: Acquire a workpiece state matrix of a workpiece to be processed and a material frame state matrix corresponding to each of a plurality of material frames to be processed, encode the workpiece state matrix to obtain a workpiece state vector, and encode a plurality of material frame state matrices to obtain a material frame state vector; Based on the workpiece state vector, the material frame state vector and a pre-trained frame selection decoder, a target material frame is determined from a plurality of material frames to be processed, and a target state vector corresponding to the target material frame is determined; Determine a target rotation direction based on the workpiece state vector, the target state vector and a pre-trained direction-selective decoder, and determine a rotated workpiece state vector based on the target rotation direction; Determine a target sorting position among a plurality of grid positions of the target material frame based on the rotated workpiece state vector, the target state vector and a pre-trained position selection decoder; Sorting the workpieces to be processed based on the target material frame and the target sorting position; The step of determining a target material frame from a plurality of material frames to be processed based on the workpiece state vector, the material frame state vector and a pre-trained selection frame decoder comprises: The workpiece state vector and the material frame state vector are decoded by using a pre-trained selection frame decoder to obtain a first decoding result; each element in the first decoding result is used to represent the probability of sorting the workpiece to be processed into multiple material frames to be processed; Determine a target material frame from a plurality of material frames to be processed according to the first decoding result and the material frame mask; The step of determining a target sorting position among a plurality of grid positions of the target material frame based on the rotated workpiece state vector, the target state vector and a pre-trained position selection decoder comprises: The rotated workpiece state vector and the target state vector are decoded by using a pre-trained position selection decoder to obtain a third decoding result; each element in the third decoding result is used to characterize the probability of the workpiece to be processed being sorted to each grid position in the target material frame; According to the third decoding result and the position mask, a target sorting position is determined among a plurality of grid positions of the target material frame.
2. The method according to claim 1, characterized in that The determining of the target rotation direction based on the workpiece state vector, the target state vector and a pre-trained direction-selective decoder comprises: The workpiece state vector and the target state vector are decoded by using a pre-trained direction-selective decoder to obtain a second decoding result; each element in the second decoding result is used to represent the probability that the workpiece to be processed can be sorted into the target material frame after rotating in a preset rotation direction; The target rotation direction is determined according to the second decoding result and the direction mask.
3. The method according to claim 1, characterized in that After sorting the workpieces to be processed based on the target material frame and the target sorting position, the method further includes: Acquire real-time material frame information and real-time workpiece information, and update a material frame mask, a direction mask, and a position mask according to the real-time material frame information and the real-time workpiece information; Workpiece sorting is performed based on the updated material frame mask, the updated direction mask, and the updated position mask.
4. The method according to any one of claims 1 to 3, characterized in that Processing the workpiece state matrix and the frame state matrix through a trained actor network, wherein the actor network includes the pre-trained frame selection decoder, the pre-trained direction selection decoder and the pre-trained position selection decoder; The training steps of the actor network include: Obtaining a workpiece state matrix sample of a workpiece sample to be processed and a material frame state matrix sample corresponding to each of a plurality of material frame samples to be processed, encoding the workpiece state matrix sample to obtain a workpiece state vector sample, and encoding a plurality of material frame state matrix samples to obtain a material frame state vector sample; Based on the workpiece state vector, the material frame state vector and the to-be-trained selection frame decoder in the initial actor network, a target material frame sample is determined from a plurality of to-be-processed material frame samples, and a target state vector sample corresponding to the target material frame sample is determined; Determine a target rotation direction sample based on the workpiece state vector sample, the target state vector sample and the direction-selective decoder to be trained in the initial actor network, and determine a rotated workpiece state vector sample based on the target rotation direction sample; Determine a target sorting position sample from a plurality of grid position samples of the target material frame sample based on the rotated workpiece state vector sample, the target state vector sample and a position selection decoder to be trained in the initial actor network; Based on the workpiece state vector, the material frame state vector and the initial critic network, obtaining a review value output by the initial critic network; Sorting the to-be-processed workpiece samples based on the target material frame samples and the target sorting position samples, and determining a reward value according to the sorted material frame information and a preset reward function; A model loss is calculated based on the difference between the reward value and the comment value, and model parameters of the initial actor network and the initial critic network are adjusted based on the model loss until a preset stop condition is met, thereby obtaining a trained actor network.
5. A workpiece sorting device, characterized in that: The device comprises: An acquisition module, used to acquire a workpiece state matrix of a workpiece to be processed and a material frame state matrix corresponding to each of a plurality of material frames to be processed, and encode the workpiece state matrix to obtain a workpiece state vector, and encode a plurality of material frame state matrices to obtain a material frame state vector; A first determination module is used to determine a target material frame from a plurality of material frames to be processed based on the workpiece state vector, the material frame state vector and a pre-trained selection frame decoder, and determine a target state vector corresponding to the target material frame; a second determination module, configured to determine a target rotation direction based on the workpiece state vector, the target state vector and a pre-trained direction-selective decoder, and determine a rotated workpiece state vector based on the target rotation direction; A third determination module, configured to determine a target sorting position among a plurality of grid positions of the target material frame based on the rotated workpiece state vector, the target state vector and a pre-trained position selection decoder; A sorting module, used for sorting the workpieces to be processed based on the target material frame and the target sorting position; The first determination module is further used to decode the workpiece state vector and the material frame state vector using a pre-trained selection frame decoder to obtain a first decoding result; each element in the first decoding result is used to characterize the probability of sorting the workpiece to be processed into multiple material frames to be processed; and determine a target material frame from multiple material frames to be processed according to the first decoding result and the material frame mask; The third determination module is further used to decode the rotated workpiece state vector and the target state vector using a pre-trained position selection decoder to obtain a third decoding result; each element in the third decoding result is used to characterize the probability of sorting the workpiece to be processed to each grid position in the target material frame; According to the third decoding result and the position mask, a target sorting position is determined among a plurality of grid positions of the target material frame.
6. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 4 are implemented.
7. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 4 are implemented.
8. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 4 are implemented.
