Three-dimensional automatic boxing system and method based on deep learning

Abstract

The invention relates to a three-dimensional automatic boxing system and method based on deep learning, and belongs to the technical field of information. The problems that in the current wharf wood loading and unloading process, efficiency is low, the space utilization rate is insufficient, and potential safety hazards exist are solved. According to the system, a container space is discretized into a three-dimensional grid structure, and an optimal placement position is dynamically selected by utilizing a greedy algorithm and a DQN algorithm in deep reinforcement learning according to the size and shape of wood and the state of the container, so that automatic boxing of the wood is realized. The system continuously optimizes a placement strategy by evaluating the size of the residual space and a stability index in combination with a reward mechanism, and maximizes the space utilization rate of the container. The wood loading efficiency and safety are effectively improved, the manual operation intensity is reduced, the placement strategy can be flexibly adjusted according to the actual situation, and the device is suitable for various scenes such as wharf wood loading and unloading.

Description

Technical Field

[0001] The present invention belongs to the field of information technology and relates to a three-dimensional automatic packing system and method based on deep learning. Background Art

[0002] Currently, the loading and unloading of timber at docks mainly relies on manual operations, which are carried out by adding cables, similar to the loading and unloading of container cargo. This process has the following problems:

[0003] (1) Lack of unified planning and control can easily lead to chaotic timber stacking, affecting transportation efficiency and safety.

[0004] (2) Reliance on manual experience and lack of precise analysis and optimization of factors such as wood size, shape, and container space result in low space utilization and low loading efficiency.

[0005] (3) Manual operation is prone to errors, resulting in unstable wood stacking and posing a safety hazard.

[0006] The existing automatic packing models based on deep learning are complex in design, difficult to debug, and have a long training cycle, which limits their promotion in practical applications. Some systems rely on pointer networks, which are sensitive to the input order and difficult to adapt to the actual situation where the shapes and specifications of dock timber vary greatly and the order is not fixed. Existing technologies and methods cannot effectively solve the problems of low efficiency, low space utilization, and insufficient safety in dock timber loading and unloading, nor can they adapt to the actual situation where the shapes and specifications of dock timber vary greatly and the order is not fixed.

[0007] In response to the above problems, the present invention provides a three-dimensional automatic packing system and method based on deep learning. By establishing a three-dimensional grid model and combining DQN and greedy algorithms, automatic packing of wood can be achieved, space utilization can be improved, loading efficiency can be improved, manual operation intensity can be reduced, and loading safety can be ensured. Summary of the invention

[0008] In view of this, the purpose of the present invention is to provide a three-dimensional automatic packing system and method based on deep learning. For dock timber, based on the existing three-dimensional timber loading method, DQN is used to transform the timber placement problem into a task of optimizing space utilization in a dynamic environment. We designed a three-dimensional grid model and used a heuristic search algorithm to generate a feasibility coding vector in the current state to guide the agent to choose an effective placement position when making decisions. At the same time, an improved maximum remaining space strategy is adopted to ensure that the space constraints and stability requirements of the container are met while improving the loading efficiency. Through continuous iteration and learning, our system effectively solves the inefficiency problem existing in traditional loading methods, realizes the optimal placement strategy for timber, and thus maximizes the comprehensive loading benefits of containers.

[0009] In order to achieve the above object, the present invention provides the following technical solutions:

[0010] A three-dimensional automatic boxing system based on deep learning, the system comprising:

[0011] The grid initialization module is used to discretize the container space into a three-dimensional grid structure;

[0012] A state evaluation module, used to evaluate the current grid state and generate a feasibility coding vector;

[0013] The DQN module receives the current state and feasibility encoding vector and selects the optimal placement position based on the greedy algorithm;

[0014] The state update module is used to place the wood in the optimal position and update the grid state;

[0015] The reward calculation module is used to calculate the reward value based on the placement results.

[0016] Further, the DQN module includes:

[0017] Encoder, used to extract important features in the current state;

[0018] The decoder is used to output the Q value corresponding to the actionable action based on the feature vector generated by the encoder.

[0019] Furthermore, the decoder adopts a graph attention mechanism.

[0020] Furthermore, the state evaluation module evaluates the quality of each candidate position according to the remaining space size and the stability index.

[0021] Furthermore, the stability index includes the inclination of the wood after placement and the contact area with surrounding wood.

[0022] Furthermore, the reward calculation module calculates a total reward value according to the utilization reward, the space optimization reward, the height constraint reward and the successful placement reward.

[0023] Furthermore, the space optimization rewards and punishes the placement strategy that results in large blocks of free space.

[0024] Furthermore, an experience replay pool is included to store the experience gained during the training process.

[0025] Furthermore, the DQN module learns based on experiences randomly sampled from the experience replay pool.

[0026] A three-dimensional automatic boxing method based on deep learning, comprising:

[0027] Discretize the container space into a three-dimensional grid structure;

[0028] Evaluate the current grid state and generate a feasibility encoding vector;

[0029] Select the optimal placement location based on the greedy algorithm;

[0030] Place the wood in the optimal position and update the grid state;

[0031] The reward value is calculated based on the placement results.

[0032] The beneficial effects of the present invention are:

[0033] (1) By combining the greedy algorithm and DQN, the present invention can effectively select the optimal placement strategy according to the distribution of the remaining space in the container. Compared with the manual operation method, it can significantly improve the utilization of space and reduce idle space and loading waste. In the case of large differences in wood specifications, the system can still dynamically adjust the placement position according to the size and shape of the wood to optimize the overall layout.

[0034] (2) The system's automated placement decision reduces manual intervention, reduces work intensity, and saves human resources. In actual operation, it is particularly suitable for scenarios with large and frequent demand for timber transportation.

[0035] (3) The wood placement system can adjust its strategy in real time, dynamically optimizing the placement method based on the size of each batch of wood, the space in the container, and the current loading status. Even in the face of ever-changing conditions during transportation, the system can respond quickly and make reasonable decisions to ensure that the loading effect is always maintained in the optimal state.

[0036] (4) The system's automated decision-making mechanism is continuously optimized through continuous training and feedback mechanisms, which can effectively reduce errors and inefficient placement in human operations, ensure the accuracy of each placement decision, and thus improve the stability of the overall operation. For large-scale timber transportation tasks, it can ensure that the loading process of each batch of timber is efficient and stable, reducing secondary damage caused by unreasonable stacking during transportation.

[0037] Other advantages, objectives and features of the present invention will be described in the following description to some extent, and to some extent, will be obvious to those skilled in the art based on the following examination and study, or can be taught from the practice of the present invention. The objectives and other advantages of the present invention can be realized and obtained through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention will be described in detail below in conjunction with the accompanying drawings, wherein:

[0039] Figure 1 A schematic diagram of grid initialization provided by an embodiment of the present invention;

[0040] Figure 2 A wood placement flow chart of a dock wood automatic packing system based on a greedy algorithm and DQN provided in an embodiment of the present invention;

[0041] Figure 3 A DQN enhanced flow chart of the automatic packing system for dock timber based on the greedy algorithm and DQN provided in an embodiment of the present invention;

[0042] Figure 4 The overall architecture diagram of the automatic dock timber packing system based on the greedy algorithm and DQN provided in the embodiment of the present invention;

[0043] Figure 5 The packing simulation result provided by the embodiment of the present invention. DETAILED DESCRIPTION

[0044] The following describes the embodiments of the present invention by specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments only illustrate the basic concept of the present invention in a schematic manner, and the following embodiments and features in the embodiments can be combined with each other without conflict.

[0045] Among them, the drawings are only used for illustrative explanations, and they only represent schematic diagrams rather than actual pictures, and should not be understood as limitations on the present invention. In order to better illustrate the embodiments of the present invention, some parts of the drawings may be omitted, enlarged or reduced, and do not represent the size of actual products. For those skilled in the art, it is understandable that some well-known structures and their descriptions in the drawings may be omitted.

[0046] The same or similar numbers in the drawings of the embodiments of the present invention correspond to the same or similar parts; in the description of the present invention, it should be understood that if the terms "upper", "lower", "left", "right", "front", "rear", etc. indicate the orientation or position relationship, they are based on the orientation or position relationship shown in the drawings, which is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operate in a specific orientation. Therefore, the terms describing the position relationship in the drawings are only used for illustrative purposes and cannot be understood as limiting the present invention. For ordinary technicians in this field, the specific meanings of the above terms can be understood according to specific circumstances.

[0047] See also Figure 1 to Figure 5 , an automatic packing system for dock timber based on DQN and greedy algorithm, the specific steps are:

[0048] Step S1: discretize the space of the entire container into a three-dimensional grid structure, and initialize all grids, that is, assign a value of "0" to each element in the array.

[0049] Step S2: Using a greedy algorithm, the system places wood pieces layer by layer starting from the bottom upward until all grids in all layers cannot accommodate any wood pieces. At this point, a complete packing process stops.

[0050] Step S3: Through DQN, during the placement process, it will continuously learn how to select the placement location through a feedback mechanism to optimize the entire placement process.

[0051] Step S4: Check the packing simulation result diagram of the trained network to verify the performance and superiority of the model in the wood automatic packing task.

[0052] The step S1 comprises:

[0053] Step A1: Discretize the space of the entire container into a three-dimensional grid structure, where each grid unit represents a certain spatial resolution. In this way, each data in the array can represent a state mark in the grid.

[0054] During the object placement process, the state of the grid is dynamically updated. When an object is successfully placed, the relevant grid cell is marked as occupied (usually represented by a value of "1"), while unoccupied cells remain at "0".

[0055] The step S2 comprises:

[0056] Step B1: Start from the bottom layer and analyze the grid status of the current layer. If the available space of the current layer is not enough to place the current wood, skip the current layer and go to the next layer for inspection.

[0057] Step B2: Traverse all available spaces in the current layer and calculate the pros and cons of each candidate position based on the evaluation function. The evaluation function comprehensively considers the remaining space size and stability, and selects the position with the highest evaluation function value as the placement position of the wood.

[0058] Step B3: Place the wood in the optimal position and update the grid status, marking the relevant grid cell as occupied (value 1). If the wood is large or heavy, the stability of the bottom layer is given priority; smaller or lighter wood is allocated to higher layers to optimize space utilization.

[0059] The judgment of the available space in the above step B1 needs to consider whether the size of the wood is completely adapted to the free area of ​​the current layer. The available space is obtained by analyzing the grid state of the container, where the value of each grid unit represents the corresponding occupancy.

[0060] In the above step B2, considering the size of the remaining space mainly refers to giving priority to the location with less space after placement to avoid waste. Considering stability mainly refers to placing it at the bottom or close to other wood to ensure that the wood will not tilt or move after placement. The greedy strategy will traverse the grid of the entire level to find the best location to place the wood. This process involves judging whether the remaining grids of the current layer meet the conditions for placing wood, that is, whether the remaining grids of the current layer can fully accommodate the wood to be placed. If not, the placement attempt will be stopped immediately. The relevant objectives and constraints are as follows:

[0061]

[0062] Among them, S in formula (1) i These are all the options available in stage i. grid_x and grid_y represent the grid status of the three-dimensional container on the x and y axes respectively. l and d are wood specification parameters. x and y are the coordinates occupied by the wood.

[0063] During the entire search process, the system will evaluate the effectiveness of each candidate location and select the optimal solution. If a location meets the conditions for placing wood and the surrounding space of the location is relatively large, then the location will be given priority. After a placement operation is successfully completed, the system will update the status of the container, recalculate the remaining available space, and prepare for the placement of the next layer. This mechanism ensures maximum utilization of space, and even when there is insufficient space on the bottom layer, it is still possible to find an area for placement upwards. Since the texture of wood is uniform, for wood of different lengths and diameters, the system will take these factors into account during the placement process to ensure the stability of placement. The system can flexibly select different optimal levels according to different wood specifications.

[0064] The step S3 comprises:

[0065] Step C1: Initialize the model and empty the experience replay buffer to store the experience during training. Get the initial state from the environment, including the grid state of the current container, the specifications of the wood to be placed, and the container utilization rate. The initial state comes from the complete state after deep learning processing, that is, the initial state is a feature representation of the complete environment state.

[0066] Step C2: According to the current state, use the ε-greedy strategy to select an action: select the action with the largest Q value in the current network with probability 1-ε and execute it.

[0067] Step C3: The agent updates the state according to the placement result in combination with the environment, and calculates the corresponding rewards, including utilization rewards, space optimization rewards, height constraint rewards, and successful placement rewards. Then the current experience is stored in the replay pool.

[0068] Step C4: Randomly sample a batch of experiences from the replay pool, and use the target network and the current network to calculate the target Q value and predict the Q value after executing the current action, and then update the current network parameters to minimize the mean square error.

[0069] In the above step C1, the system expands the DQN model to obtain better training efficiency and comprehensive placement effectiveness. This system introduces an encoder-decoder architecture. The encoder part is completed by a multi-layer fully connected network (MLP), which can extract important features in the state and map the environmental state (including wood size, container state, etc.) to a low-dimensional latent space representation. The decoder part uses the graph attention mechanism (GAT) as the core part of the decoder. According to the feature vector generated by the encoder, it outputs the Q value or strategy corresponding to the actionable action. This enables the agent to effectively select the best action in a complex environment, enhances the model's attention to the importance of different actions, and thus achieves efficient placement of wood.

[0070] In the above step C2, the input of DQN is the current state, including the occupancy of the container, the size of the wood, and the current utilization rate. Finally, the Q value of each possible action (such as the location of each piece of wood) is output, and the agent selects the action with the largest Q value to execute. During the training process, the reinforcement learning part of DQN uses the experience replay mechanism to randomly sample past experience (state, action, reward, next state) for learning, which effectively breaks the correlation between data and improves learning efficiency. Its main loss function and parameter update are as follows:

[0071] L(θ)=E[(yQ(s,a;θ)) 2 ] (6)

[0072]

[0073] Where y in formula (1) represents the target value Q, Q(s, a; θ) represents the Q value prediction of the current network for action a in state s, θ represents the weight of the current model, and α in formula (2) is the learning rate.

[0074] In step C3 above, the utilization reward is designed to be a positive number, indicating the effectiveness of the current placement strategy, so that the agent tends to choose placement strategies that can improve container utilization. The space optimization reward sets a penalty term to punish placement strategies that cause large blocks of free space. This penalty can be achieved by calculating the size of the gap between adjacent wood. If the gap is too large, the reward is reduced. The height constraint reward punishes the choice of placing beyond the container boundary, usually a specified constant. The completion placement reward indicates the success of the placement action. This reward can help the agent form positive feedback during the learning process, usually the number of successful placements at one time. The relevant reward function is designed as follows:

[0075]

[0076] R g =-λ·N g (9)

[0077] R=R u +R g +R h +R s (10)

[0078] Among them, R u , R g , R h , R s They represent utilization reward, space optimization reward, height constraint reward and successful placement reward respectively.

[0079] During the training process of step C4 above, the parameters of the current network are copied to the target network at fixed steps until a complete packing is completed.

[0080] Table 1 is a wood placement algorithm provided by an embodiment of the present invention.

[0081] Table 1

[0082]

[0083] Table 2 shows the comparison of container utilization

[0084] method Simulate manual operation Deep Reinforcement Learning 30 random pieces of wood 0.29 0.29 50 random pieces of wood 0.45 0.49 70 random wood 0.54 0.62 100 random pieces of wood 0.59 0.69

[0085] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the purpose and scope of the technical solution, which should be included in the scope of the claims of the present invention.

Claims

1. A three-dimensional automatic packaging system based on deep learning, characterized by: The system includes: The grid initialization module is used to discretize the container space into a three-dimensional grid structure; A state evaluation module, used to evaluate the current grid state and generate a feasibility coding vector; The DQN module receives the current state and feasibility encoding vector and selects the optimal placement position based on the greedy algorithm; The state update module is used to place the wood in the optimal position and update the grid state; The reward calculation module is used to calculate the reward value based on the placement results.

2. The three-dimensional automatic packaging system based on deep learning according to claim 1, characterized in that: The DQN module includes: Encoder, used to extract important features in the current state; The decoder is used to output the Q value corresponding to the actionable action based on the feature vector generated by the encoder.

3. The three-dimensional automatic packaging system based on deep learning according to claim 2 is characterized in that: The decoder adopts a graph attention mechanism.

4. The three-dimensional automatic packaging system based on deep learning according to claim 1, characterized in that: The state evaluation module evaluates the quality of each candidate position according to the remaining space size and stability index.

5. The three-dimensional automatic packaging system based on deep learning according to claim 4 is characterized in that: The stability index includes the inclination of the wood after placement and the contact area with the surrounding wood.

6. The three-dimensional automatic packaging system based on deep learning according to claim 1, characterized in that: The reward calculation module calculates a total reward value according to the utilization reward, the space optimization reward, the height constraint reward and the successful placement reward.

7. The three-dimensional automatic packaging system based on deep learning according to claim 6 is characterized in that: The space optimization rewards and penalizes placement strategies that result in large blocks of free space.

8. The three-dimensional automatic packaging system based on deep learning according to claim 1, characterized in that: The three-dimensional automatic box packing system also includes an experience replay pool for storing experience in the training process.

9. The three-dimensional automatic packaging system based on deep learning according to claim 8, characterized in that: The DQN module learns based on experiences randomly sampled from the experience replay pool.

10. A three-dimensional automatic packaging method based on deep learning, characterized in that: include: Discretize the container space into a three-dimensional grid structure; Evaluate the current grid state and generate a feasibility encoding vector; Select the optimal placement location based on the greedy algorithm; Place the wood in the optimal position and update the grid state; The reward value is calculated based on the placement results.

Images