An online three-dimensional packing method and apparatus with physical stability verification

Abstract

This invention discloses an online 3D bin packing method and apparatus with physical stability verification. The method includes, before selecting a placement location for an item within a bin, performing physical stability verification based on the height map of the already packed items in each grid within the bin and the dimensions of the item to be packed to generate a feasibility mask for candidate placement locations. This includes enumerating all possible candidate placement locations based on the height map of the already packed items, and for each candidate placement location, iterating and sequentially performing static centroid stability detection and mass distribution transfer centroid stability detection to generate a feasibility mask. The value of each element in the feasibility mask is 1 or 0, indicating whether the corresponding grid is a candidate placement location for the item. This invention aims to improve the stability of online 3D bin packing by utilizing physical stability verification, preventing item collapse due to stability issues, and improving the efficiency of online 3D bin packing.

Description

Technical Field

[0001] This invention relates to three-dimensional packing technology in the fields of logistics and industrial automation, specifically to an online three-dimensional packing method and apparatus with physical stability verification. Background Technology

[0002] The Bin Packing Problem (BPP) is a classic combinatorial optimization problem. Its main objective is to pack a set of items of different sizes into containers with limited capacity, minimizing the number of containers used. The Bin Packing Problem is an NP-hard problem, meaning that there is no known algorithm that can determine the absolutely optimal solution in polynomial time.

[0003] Based on the dimensions of the items and containers, the bin packing problem can be divided into one-dimensional bin packing problems (1D-BPP), two-dimensional bin packing problems (2D-BPP), and three-dimensional bin packing problems (3D-BPP). In the one-dimensional bin packing problem, items and containers are simplified to a one-dimensional length or weight problem, with the goal of using as few containers as possible to pack all items, such as loading goods of different weights into a transport vehicle with a limited load capacity. In the two-dimensional bin packing problem, items and containers are treated as rectangles; the key is how to most efficiently utilize the space of the rectangular containers without overlap, such as placing rectangular items of different sizes on rectangular shelves in warehouse management. The three-dimensional bin packing problem involves packing cubic items of different sizes into cubic containers, with the goal of minimizing the number of boxes used while satisfying various constraints. It is commonly used in logistics and warehousing management to pack furniture and other items of different shapes into containers, introducing more solution complexity. Given a project... The length is ,width and height The three-dimensional "weights" (sizes) of 3D-BPP coordinate the allocation of the project across three dimensions, each with its own capacity, including length. ,width and height Assuming 3D-BPP also aims to pack items into boxes using the fewest possible containers. 3D-BPP has wide practical applications in modern packing, logistics, and manufacturing, particularly in the development of core technologies for palletizing robots for intelligent logistics. Palletizing robots are designed to pack boxes into standard-sized rectangular containers. Maximizing the use of pallet containers can improve productivity in inventory, packing, transportation, and warehousing. Due to its computational complexity, 3D-BPP has been explored relatively less compared to 1D-BPP. Especially when the problem size increases, exact algorithms (whether using integer linear programming or branch and bound) cannot provide a solution in a finite time. Solving medium-sized 3D-BPP problems still requires resorting to heuristic algorithms.

[0004] In many application scenarios, there is a significant demand for a more challenging problem—Online 3D Box Packing (BPP). In BPP, the agent / robot is not provided with information about the complete sequence of items (similar to Tetris). For example, when a robot is continuously packing packages, the limited field of view of the camera means that the RGB-D sensors placed around the robot can only provide a partial view of the item sequence. Since the conveyor moves sequentially during the packing process, the robot can only unload and place packages within a very short time. This constraint makes BPP not a purely combinatorial optimization problem, as the optimal solution cannot be obtained through enumeration. How to achieve fast, efficient, and optimal BPP has become a critical technical problem that urgently needs to be solved. Summary of the Invention

[0005] The technical problem to be solved by the present invention is to provide an online three-dimensional packing method and apparatus with physical stability verification, which addresses the above-mentioned problems of the prior art. The present invention aims to improve the stability of online three-dimensional packing by using physical stability verification, prevent the collapse of the packing objects due to stability issues, and improve the efficiency of online three-dimensional packing.

[0006] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is as follows:

[0007] An online 3D packing method with physical stability verification includes selecting the items to be packed. Before placing the item in the container, obtain the height map of the loaded items in each grid within the container. Items to be packed The dimensions, and based on the height of the contents in each grid within the container. Items to be packed The dimensions were physically stable to generate a representation of the object to be loaded. Feasibility mask for candidate placement locations :

[0008] S101, based on dimensions Diagram showing the height of items loaded in each grid within the container. Enumerate all possible candidate placement positions, with an initial generation size of . Feasibility mask ,in , and These are the length, width, and height of the container, and the feasibility mask. Elements outside the candidate placement position have a value of 0;

[0009] S102, iterate through all candidate placement positions and select a current candidate placement position. ;

[0010] S103, for the contents Place candidate placement locations Perform a static centroid stability test. If the centroid stability test passes, proceed to step S104; otherwise, determine the feasibility mask. Current candidate placement position The element has a value of 0; jump to step S105;

[0011] S104, for the contents Place candidate placement locations Perform a centroid stability test on the mass distribution transfer. If the centroid stability test on the mass distribution transfer passes, then determine the feasibility mask. Current candidate placement position The element is set to 1; otherwise, a feasibility mask is determined. Current candidate placement position The element takes the value 0;

[0012] S105, determine if all candidate placement positions have been traversed. If not, jump to step S102 to continue traversing; otherwise, determine the feasibility mask. Once generated, the feasibility mask... The value of each element in the grid is 1 or 0, indicating whether the corresponding grid can be used as a loading object. Candidate placement positions.

[0013] Optionally, in step S101, the dimensions are... Height diagram of each grid in the box When enumerating all possible candidate placement locations, the constraints that all possible candidate placement locations must satisfy are:

[0014] ;

[0015] ;

[0016] in, For items to be packed Placement starting coordinates On the axis The value on, For items to be packed On the axis The dimensions and length on top For the box body on the axis The dimensions and length on top Indicates the item to be loaded and already loaded items Along axis The sequential relationship variable, with a value of 1 indicating the items to be loaded. The items have already been loaded. If it is in front of, to the left of, or below, then it is 0; otherwise, it is 0. A collection of already loaded items; axis for One of the three axes.

[0017] Optionally, in step S103, the object to be packed... Place candidate placement locations Static centroid stability testing includes: calculating the load. Place in the current candidate placement Calculate the convex hull region of all bottom support points by projecting the center of mass onto the XY plane on the bottom surface of the box and the bottom support points. Determine whether the projection of the center of mass onto the XY plane on the bottom surface of the box falls within the convex hull region. If the projection of the center of mass onto the XY plane on the bottom surface of the box falls within the convex hull region, the center of mass stability test is considered to have passed; otherwise, the center of mass stability test is considered to have failed.

[0018] Optionally, in step S104, the object to be packed... Place candidate placement locations The centroid stability test for mass distribution transfer includes:

[0019] S201, Loading the item Place in the current candidate placement The support relationships are modeled as directed graphs. Directed graph The nodes in the text are objects, and the objects include items to be loaded. The relationship between the object and the loaded object, with the edges representing the supporting relationships of the object;

[0020] S202, from the directed graph Items to be loaded Starting from the node of contact, the recursive construction only includes the object to be loaded, whether directly or indirectly subjected to it. All items already loaded in the mass distribution are considered as items to be loaded. Adaptive stacked tree The adaptive stacked tree For directed graphs Chinese and Israeli items to be loaded This is a subgraph of the root node, where each node directly or indirectly bears the load to be loaded. The mass distribution of the loaded material;

[0021] S203, from adaptive stacked tree Starting from the root node, traverse towards the leaf nodes, for each current node obtained during the traversal... Determine the current node The number of child nodes is determined. If the number of child nodes is 1, proceed to step S204; if the number of child nodes is 2, proceed to step S205; if the number of child nodes is 2 or more, proceed to step S206; if it is an adaptive stacking tree... Once the traversal is complete, proceed to step S207.

[0022] S204, for the current node child nodes Calculate the distance from the root node to the child nodes using the following formula. The projection points of the centroid of the node group onto the XY plane on the bottom surface of the box:

[0023] ;

[0024] in, From root node to child nodes The coordinates of the centroid of the node group projected onto the XY plane on the bottom surface of the box. and Each is the current node child nodes The mass of the object, and Each is the current node child nodes Find the coordinates of the center of mass projected onto the XY plane on the bottom surface of the box; jump to step S203 to continue traversing;

[0025] S205, for the current node Each child node Calculate the distance from the root node to the current node according to the following formula. child nodes Node group quality distribution:

[0026] ;

[0027] ;

[0028] in, child node Supporting force It is the acceleration due to gravity. For the current node The coordinates of the center of mass projected onto the XY plane on the bottom surface of the box. and child nodes and child nodes With the current node The coordinates of the midpoint of the bottom contact surface, child nodes and child nodes The current node is formed when the number of child nodes is 2. The two child nodes, From the root node to the current node child nodes The quality distribution of the node group, child node The mass of the object; calculate the distance from the root node to each child node according to the following formula. The projection points of the centroid of the node group onto the XY plane on the bottom surface of the box:

[0029] ;

[0030] in, From root node to child nodes The coordinates of the centroid of the node group projected onto the XY plane on the bottom surface of the box. child node Find the coordinates of the center of mass projected onto the XY plane on the bottom surface of the box; jump to step S203 to continue traversing;

[0031] S206, for the current node Each child node Determine child nodes With the current node Bottom contact support point ,according to Calculate the current node The center of mass is projected onto the XY plane on the bottom surface of the box. To each support point distance And calculate the child nodes according to the following formula. Support force :

[0032] ;

[0033] in, For the current node The center of mass is projected onto the XY plane on the bottom surface of the box. To support point distance, child node With the current node The bottom contact support point, For the current node The corresponding total mass of the node group For the current node, gravitational acceleration is used. Each child node Calculate the distance from the root node to the current node according to the following formula. child nodes Node group quality distribution:

[0034] ;

[0035] Calculate the distance from the root node to each child node according to the following formula. The projection points of the centroid of the node group onto the XY plane on the bottom surface of the box:

[0036] ;

[0037] Jump to step S203 and continue traversing;

[0038] S207, calculate the convex hull region of the bottom support point of the loaded object corresponding to each leaf node; determine whether the XY plane projection point of the centroid of the node group corresponding to each leaf node on the bottom surface of the box falls within the convex hull region corresponding to the leaf node. If the XY plane projection point of the centroid of the node group corresponding to each leaf node on the bottom surface of the box falls within the convex hull region corresponding to the leaf node, the centroid stability test of mass distribution transfer is deemed to have passed; otherwise, the centroid stability test of mass distribution transfer is deemed to have failed.

[0039] Optionally, the selection of the item to be loaded... Placement locations within the enclosure include:

[0040] S301, Obtain the item to be loaded. The packing environment status, including the height diagram of the packed items in each grid within the box. Items to be packed Size tensor and items to be packed Feasibility mask for candidate placement locations :

[0041] S302, the packing environment state is used to generate a state feature vector through a state convolutional neural network. ;

[0042] S303, the state feature vector Predict X-direction actions using the x-output head of the action network from a pre-trained reinforcement learning algorithm model. ; the state feature vector and X-direction movement Predicting Y-direction actions using the y-output head in an action network ; the state feature vector X-direction movement and Y-direction movement Predicting orientation actions using the o-output head in an action network ;

[0043] S304, regarding the predicted X-direction motion Y-direction movement and directional movements Combined with feasibility mask Modulation is performed dimensionally in the order of X-direction, Y-direction, and orientation to obtain the object to be loaded. Packing process inside the box To obtain the items to be loaded The placement position within the box, among which , and These represent the actions in the X direction, Y direction, and orientation, respectively, and represent the offset in the X direction, the offset in the Y direction, and the placement angle.

[0044] Optionally, in step S304, a feasibility mask is incorporated. Modulation is performed dimensionally in the order of X-direction, Y-direction, and orientation to obtain the object to be loaded. Packing process inside the box To obtain the items to be loaded The function expression for the placement position within the box is:

[0045] ;

[0046] ;

[0047] ;

[0048] ;

[0049] in, For feasibility masking Modulated X-direction motion The probability distribution, For modulation function, Current packing environment status The X-direction motion is predicted by the x-output head. The probability distribution, Feasibility mask The X-direction component, To perform an action in a given X direction After feasibility mask M n Modulated Y-direction motion The probability distribution, To be in a given current packing environment state Under the condition of movement in the X direction The original Y-direction movement The probability distribution, Feasibility mask Action in the X direction under the condition Movement in the Y direction The feasibility mask value components, To perform an action in a given X direction Movement in the Y direction After feasibility masking Modulated orientation movement The probability distribution, To be in a given current packing environment state Under the condition of movement in the X direction Movement in the Y direction The original orientation of the movement The probability distribution, Feasibility mask Items to be loaded In the direction of the action The next corresponding X-direction action Movement in the Y direction The feasibility mask value is used to indicate whether the action is physically feasible; For the first The unmodulated probability value of each candidate action in the original action distribution. The value is assigned to the feasibility mask corresponding to the candidate action position. This represents the action probability value after masking correction. To Normalized probability output; This is a preset, extremely small positive number used to replace the probability of an inactive action.

[0050] Optionally, the reinforcement learning algorithm model is an action-value network model consisting of an action network and a value network used to assist in the training of the action network, and the reward function used by the value network during training has the following functional expression:

[0051] ;

[0052] ;

[0053] ;

[0054] ;

[0055] in, As a reward, For collision-free rewards, Incentives are given for volume utilization. Given the volume of all safe loading positions, if there is no placed object blocking the straight path from the robotic arm access line fixed on the box to a certain loading position, then the loading position is determined to be a safe loading position. This refers to the total volume of the box. Let n be the volume of the object to be packed. The incremental reward is for the boundary margin. For items to be packed When placed in the corresponding candidate position, the minimum distance from the projection point of its centroid on the bottom surface of the box to the boundary of the support area is used to characterize the support stability margin of the object to be loaded at that position. The preset boundary margin threshold is used to limit the minimum distance required to ensure that the object to be loaded has sufficient support and stability margin.

[0056] The loss function used during training of the action-value network model is:

[0057] ;

[0058] ;

[0059] ;

[0060] ;

[0061] in, For loss function, , and As weight, For the loss of the action network, For the loss of the value network, To mitigate losses due to infeasibility probability, This is an infeasibility probability suppression term. For the first The immediate reward obtained after performing a step action; This is a reward discount factor used to balance current rewards with future benefits. For the value network to the next state Value estimation; For the value network to the current state Value estimation; For Action Networks In the current state Down-output actions The probability, For items to be packed In the direction The corresponding position below The feasibility mask value is used to indicate whether the action is physically feasible.

[0062] The present invention also provides an online three-dimensional packing device with physical stability verification, including a conveyor belt, an RGB-D sensor and a robotic arm. The conveyor belt is located within the field of view of the RGB-D sensor. The control end of the conveyor belt and the output end of the RGB-D sensor are respectively connected to the control unit of the robotic arm. The control unit of the robotic arm includes a processor and a memory connected to each other. The processor is programmed or configured to execute the online three-dimensional packing method with physical stability verification.

[0063] The present invention also provides a computer-readable storage medium storing a computer program or instructions that are programmed or configured to execute the online three-dimensional packing method with physical stability verification by a processor.

[0064] The present invention also provides a computer program product, including a computer program or instructions, which are programmed or configured to execute the online three-dimensional packing method with physical stability verification by a processor.

[0065] Compared with the prior art, the present invention mainly has the following beneficial effects: the method of the present invention includes selecting the object to be loaded. Before placing the items in the container, this includes determining the height of the contents based on the grid layout within the container. Items to be packed The dimensions are used to verify physical stability in order to generate a feasibility mask for candidate placement locations. This includes based on the height map of the loaded items. Enumerate all possible candidate placement locations, and for each candidate placement location, iterate through it and sequentially perform static centroid stability checks and mass distribution propagation centroid stability checks to generate a feasibility mask. Feasibility mask The value of each element in the grid is 1 or 0, indicating whether the corresponding grid can be used as a loading object. The candidate placement position, through static centroid stability detection and mass distribution transfer centroid stability detection, can improve the stability of online 3D packing by utilizing physical stability verification, preventing the collapse of the items to be packed due to stability issues, which would lead to re-unloading and packing, thereby effectively improving the efficiency of online 3D packing. Attached Figure Description

[0066] Figure 1 This is a schematic diagram of the basic process of the method in an embodiment of the present invention.

[0067] Figure 2 This is a schematic diagram of the structure of the object to be packed and the box in an embodiment of the present invention.

[0068] Figure 3 This is a schematic diagram of an adaptive stacking tree in an embodiment of the present invention, wherein... Figure 3 (a) and Figure 3 (c) shows the different placement states of the items to be loaded. Figure 3 (b) is Figure 3 (a) Adaptive stacked tree, Figure 3 (d) is Figure 3 (c) Adaptive stacked tree.

[0069] Figure 4 Selecting the object to be loaded in the embodiments of the present invention A schematic diagram of the network structure where the enclosure is located.

[0070] Figure 5 This is a schematic diagram of the data structure of the packing environment state in an embodiment of the present invention, wherein... Figure 5 (a) is a diagram showing the height of the loaded items. , Figure 5 (b) is a feasibility mask , Figure 5 (c) is the item to be loaded. Size tensor .

[0071] Figure 6 This is a schematic diagram of the safe loading position in an embodiment of the present invention, wherein... Figure 6 (a) is a schematic diagram of a loading location where there is a risk of collision. Figure 6 (b) is a schematic diagram of the safe loading position. Detailed Implementation

[0072] To enable those skilled in the art to better understand the technical solutions of the present invention, the technical solutions of the present invention will be further described in detail below with reference to the accompanying drawings in the embodiments of the present invention.

[0073] like Figure 1 As shown, the online three-dimensional packing method with physical stability verification in this embodiment includes selecting the items to be packed. Before placing the item in the container, obtain the height map of the loaded items in each grid within the container. Items to be packed The dimensions, and based on the height of the contents in each grid within the container. Items to be packed The dimensions were physically stable to generate a representation of the object to be loaded. Feasibility mask for candidate placement locations :

[0074] S101, based on dimensions Diagram showing the height of items loaded in each grid within the container. Enumerate all possible candidate placement positions, with an initial generation size of . Feasibility mask ,in , and These are the length, width, and height of the container, and the feasibility mask. Elements outside the candidate placement position have a value of 0;

[0075] S102, iterate through all candidate placement positions and select a current candidate placement position. ;

[0076] S103, for the contents Place candidate placement locations Perform a static centroid stability test. If the centroid stability test passes, proceed to step S104; otherwise, determine the feasibility mask. Current candidate placement position The element has a value of 0; jump to step S105;

[0077] S104, for the contents Place candidate placement locations Perform a centroid stability test on the mass distribution transfer. If the centroid stability test on the mass distribution transfer passes, then determine the feasibility mask. Current candidate placement position The element is set to 1; otherwise, a feasibility mask is determined. Current candidate placement position The element takes the value 0;

[0078] S105, determine if all candidate placement positions have been traversed. If not, jump to step S102 to continue traversing; otherwise, determine the feasibility mask. Once generated, the feasibility mask... The value of each element in the grid is 1 or 0, indicating whether the corresponding grid can be used as a loading item. Candidate placement positions.

[0079] Figure 2 This is a schematic diagram of the structure of the object to be packed and the box in this embodiment. The dimensions of the box are: Items to be packed The size is ,in , and These are the length, width, and height of the box, respectively. , and items to be packed Length, width, and height; items to be loaded The bottom left corner is used as the item to be loaded. The positioning reference point. In step S101 of this embodiment, the size is... Height diagram of each grid in the box When enumerating all possible candidate placement locations, the constraints that all possible candidate placement locations must satisfy are:

[0080] ;

[0081] ;

[0082] in, For items to be packed Placement starting coordinates On the axis The value on, For items to be packed On the axis The dimensions and length on top For the box body on the axis The dimensions and length on top Indicates the item to be loaded and already loaded items Along axis The sequential relationship variable, with a value of 1 indicating the items to be loaded. The items have already been loaded. If it is in front of, to the left of, or below, then it is 0; otherwise, it is 0. A collection of already loaded items; axis for One of the three axes.

[0083] In order to determine the stability of candidate placement actions in real time and efficiently during the packing process, this embodiment adopts physical stability verification that combines static centroid stability detection and mass distribution transfer centroid stability detection.

[0084] In step S103 of this embodiment, the object to be loaded... Place candidate placement locations Static centroid stability testing includes: calculating the load. Place in the current candidate placement The system calculates the convex hull region of all bottom support points by projecting the centroid onto the XY plane on the bottom surface of the box and the bottom support points. It then determines whether the projected centroid onto the XY plane on the bottom surface of the box falls within the convex hull region. If the projected centroid onto the XY plane on the bottom surface of the box falls within the convex hull region, the centroid stability test is considered passed; otherwise, the centroid stability test is considered failed. The static centroid stability test criterion is simple and fast, making it suitable as the basic layer for stability analysis.

[0085] In scenarios involving multiple stacked objects, relying solely on the centroid criterion may not cover all complex force chains. Therefore, this embodiment proposes an Adaptive Stacking Tree for centroid stability detection based on mass distribution propagation. This tree updates and calculates stability only for the active subset that the current new object directly or indirectly depends on, avoiding redundant traversal of the global structure. Its complexity is significantly reduced compared to the traditional top-down approach. Down to This significantly improves real-time performance. Specifically, in step S104 of this embodiment, the object to be loaded... Place candidate placement locations The centroid stability test for mass distribution transfer includes:

[0086] S201, Loading the item Place in the current candidate placement The support relationships are modeled as directed graphs. Directed graph The nodes in the text are objects, and the objects include items to be loaded. The relationship between the object and the loaded object, with the edges representing the supporting relationships of the object;

[0087] S202, from the directed graph Items to be loaded Starting from the node of contact, the recursive construction only includes the object to be loaded, whether directly or indirectly subjected to it. All items already loaded in the mass distribution are considered as items to be loaded. Adaptive stacked tree The adaptive stacked tree For directed graphs Chinese and Israeli items to be loaded This is a subgraph of the root node, where each node directly or indirectly bears the load to be loaded. The mass distribution of the loaded material; Figure 3 This is a schematic diagram of the adaptive stacking tree in this embodiment, wherein... Figure 3 (a) and Figure 3 (c) shows two ways to place the items to be loaded. Figure 3 (b) is Figure 3 (a) Adaptive stacked tree, Figure 3 (d) is Figure 3(c) An adaptive stacking tree, where A and H represent the loaded items, respectively;

[0088] S203, from adaptive stacked tree Starting from the root node, traverse towards the leaf nodes, for each current node obtained during the traversal... Determine the current node The number of child nodes is determined. If the number of child nodes is 1, proceed to step S204; if the number of child nodes is 2, proceed to step S205; if the number of child nodes is 2 or more, proceed to step S206; if it is an adaptive stacking tree... Once the traversal is complete, proceed to step S207.

[0089] S204, for the current node child nodes Calculate the distance from the root node to the child nodes using the following formula. The projection points of the centroid of the node group onto the XY plane on the bottom surface of the box:

[0090] ;

[0091] in, From root node to child nodes The coordinates of the centroid of the node group projected onto the XY plane on the bottom surface of the box. and Each is the current node child nodes The mass of the object, and Each is the current node child nodes Find the coordinates of the center of mass projected onto the XY plane on the bottom surface of the box; jump to step S203 to continue traversing;

[0092] S205, for the current node Each child node Calculate the distance from the root node to the current node according to the following formula. child nodes Node group quality distribution:

[0093] ;

[0094] ;

[0095] in, child node Supporting force It is the acceleration due to gravity. For the current node The coordinates of the center of mass projected onto the XY plane on the bottom surface of the box. and child nodes and child nodes With the current node The coordinates of the midpoint of the bottom contact surface, child nodes and child nodes The current node is formed when the number of child nodes is 2. The two child nodes, From the root node to the current node child nodes The quality distribution of the node group, child node The mass of the object; calculate the distance from the root node to each child node according to the following formula. The projection points of the centroid of the node group onto the XY plane on the bottom surface of the box:

[0096] ;

[0097] in, From root node to child nodes The coordinates of the centroid of the node group projected onto the XY plane on the bottom surface of the box. child node Find the coordinates of the center of mass projected onto the XY plane on the bottom surface of the box; jump to step S203 to continue traversing;

[0098] S206, for the current node Each child node Determine child nodes With the current node Bottom contact support point ,according to Calculate the current node The center of mass is projected onto the XY plane on the bottom surface of the box. To each support point distance And calculate the child nodes according to the following formula. Support force :

[0099] ;

[0100] in, For the current node The center of mass is projected onto the XY plane on the bottom surface of the box. To support point distance, child node With the current node The bottom contact support point, For the current node The corresponding total mass of the node group For the current node, gravitational acceleration is used. Each child node Calculate the distance from the root node to the current node according to the following formula. child nodes Node group quality distribution:

[0101] ;

[0102] Calculate the distance from the root node to each child node according to the following formula. The projection points of the centroid of the node group onto the XY plane on the bottom surface of the box:

[0103] ;

[0104] Jump to step S203 and continue traversing;

[0105] S207, calculate the convex hull region of the bottom support point of the loaded object corresponding to each leaf node; determine whether the XY plane projection point of the centroid of the node group corresponding to each leaf node on the bottom surface of the box falls within the convex hull region corresponding to the leaf node. If the XY plane projection point of the centroid of the node group corresponding to each leaf node on the bottom surface of the box falls within the convex hull region corresponding to the leaf node, the centroid stability test of mass distribution transfer is deemed to have passed; otherwise, the centroid stability test of mass distribution transfer is deemed to have failed.

[0106] In the above-mentioned centroid stability detection for mass distribution transfer, the method in this embodiment adopts differentiated processing based on the number of support contact points in terms of specific force distribution: in the case of a single support, the mass of the supporting object below is directly inherited; in the case of two supports, the reaction force ratio is solved using lever balance; and in the case of multiple supports, the contact reaction force distribution is approximately solved using the least squares method. This method approximates the high-overhead rigid body simulation, ensuring a balance between judgment accuracy (≈99.9%) and computational efficiency. Through the above process, the system can quickly generate a stability feasibility mask for each candidate placement position and orientation. It can be used to select the items to be loaded. Placement location within the container. Select the item to be packed. The placement method within the container can be chosen according to actual needs. For example, a traditional random selection strategy can be used. However, the traditional random selection strategy is not the optimal strategy. To improve the selection of items to be packed... To ensure the stability of the placement position within the container, this embodiment employs a reinforcement learning algorithm model to select the object to be packed. Placement location within the enclosure. Feasibility mask. The input is directly passed to the policy network to constrain the action distribution and reasoning process, thereby ensuring that the final decision has both high space utilization and meets the requirements of physical stability.

[0107] In online 3D bin packing tasks with extremely high action space dimensions, directly predicting complete 3D discrete actions (x, y, o) not only incurs enormous search and computational complexity but may also cause the policy network to fail to converge. This embodiment proposes a decoupled action prediction reinforcement learning algorithm model that significantly reduces the computational burden while maintaining high-resolution discretization. Figure 4 As shown, this embodiment uses a reinforcement learning algorithm model to select the object to be loaded. Placement locations within the enclosure include:

[0108] S301, Obtain the item to be loaded. The packing environment status, including the height diagram of the packed items in each grid within the box. Items to be packed Size tensor and items to be packed Feasibility mask for candidate placement locations :

[0109] S302, the packing environment state is used to generate a state feature vector through a state convolutional neural network. ;

[0110] S303, according to Sequential modulation, including the state feature vector Predict X-direction actions using the x-output head of the action network from a pre-trained reinforcement learning algorithm model. ; the state feature vector and X-direction movement Predicting Y-direction actions using the y-output head in an action network ; the state feature vector X-direction movement and Y-direction movement Predicting orientation actions using the o-output head in an action network ;

[0111] S304, regarding the predicted X-direction motion Y-direction movement and directional movements Combined with feasibility mask Modulation is performed dimensionally in the order of X-direction, Y-direction, and orientation to obtain the object to be loaded. Packing process inside the box To obtain the items to be loaded The placement position within the box, among which , and These represent the movements in the X direction, Y direction, and orientation, respectively, and represent the offset in the X direction, the offset in the Y direction, and the placement angle.

[0112] Figure 5 This is a schematic diagram of the data structure for the packing environment state in this embodiment. For example... Figure 5 As shown in (a), this embodiment shows the height of the loaded items in each grid within the box. This is an integer matrix, where each element represents the number of layers on the corresponding grid where items have been placed; for example... Figure 5 As shown in (c), the object to be loaded Size tensor This is a three-channel tensor, where the three channels are respectively generated by the object to be loaded. length ,width and height The size of the fill is The matrix, such as Figure 5 (b) shows the item to be loaded. Feasibility mask for candidate placement locations For size The matrix, feasibility mask The value of each element in the grid is 1 or 0, indicating whether the corresponding grid can be used as a loading item. Candidate placement positions.

[0113] In this embodiment, the stateful convolutional neural network includes an input layer, a downsampling layer, a feature aggregation layer, a high-level semantic layer, and an output layer connected in sequence, wherein:

[0114] The input layer is a convolutional kernel. The convolutional module is used to extract local stacking height and feasible region features;

[0115] The downsampling layer is a convolutional module or a max pooling module with a stride of 2, used to obtain spatial relationships with a larger field of view;

[0116] The feature aggregation layer is a convolutional kernel. The convolutional module is used to capture placement relationships across regions;

[0117] The high-level semantic layer consists of convolutional modules and ReLU activation functions, used to extract overall binning morphology features;

[0118] The output layer consists of an output layer and a global average pooling module, used to output a fixed-length state feature vector. This state feature vector is generated by passing the binning environment state through a state convolutional neural network. It is a high-dimensional state embedding, which serves as a shared input to the subsequent action network for predicting placement location and orientation.

[0119] In step S303 of this embodiment, the three-dimensional discrete action is... The process is decomposed into three sequential sub-decision heads: the x-head, the y-head, and the o-head. During prediction, the x-head first outputs the lateral position distribution, then the y-head predicts the lateral distribution using the x-head's output as a condition; finally, the o-head, given the known x-head output, predicts the lateral distribution. Predicting placement orientation under the premise of conditional dependency. Through this conditional dependency mechanism, the entire action generation process forms a chain-like causal structure, avoiding direct modeling of a large joint distribution. Specifically, in this embodiment, the state feature vector... Predict the action (position distribution) in the x-direction using the x-output head of the action network. The function expression is:

[0120] ;

[0121] in, The softmax activation function is used. It is the ReLU activation function. and For weight parameters, and The bias parameter is used to set the state feature vector. and position in the x direction Predict the action (position distribution) in the y-direction using the y-output head of the action network. The function expression is:

[0122] ;

[0123] in, State feature vector and position in the x direction splicing operation, and For weight parameters, and The bias parameter is used to set the state feature vector. Position in the x-direction and position in the y direction Predict orientation actions (position distribution) using the output head of the action network. The function expression is:

[0124] ;

[0125] in, State feature vector Position in the x-direction and position in the y direction splicing operation, and For weight parameters, and This is the bias parameter.

[0126] In step S304 of this embodiment, a feasibility mask is incorporated. Modulation is performed dimensionally in the order of X-direction, Y-direction, and orientation to obtain the object to be loaded. Packing process inside the box To obtain the items to be loaded The function expression for the placement position within the box is:

[0127] ;

[0128] ;

[0129] ;

[0130] ;

[0131] in, For feasibility masking Modulated X-direction motion The probability distribution, For modulation function, Current packing environment status The X-direction motion is predicted by the x-output head. The probability distribution, Feasibility mask The X-direction component, To perform an action in a given X direction After feasibility mask M n Modulated Y-direction motion The probability distribution, To be in a given current packing environment state Under the condition of movement in the X direction The original Y-direction movement The probability distribution, Feasibility mask Action in the X direction under the condition Movement in the Y direction The feasibility mask value components, To perform an action in a given X direction Movement in the Y direction After feasibility masking Modulated orientation movement The probability distribution, To be in a given current packing environment state Under the condition of movement in the X direction Movement in the Y direction The original orientation of the movement The probability distribution, Feasibility mask Items to be loaded In the direction of the action The next corresponding X-direction action Movement in the Y direction The feasibility mask value is used to indicate whether the action is physically feasible; For the first The unmodulated probability value of each candidate action in the original action distribution. The value is assigned to the feasibility mask corresponding to the candidate action position. This represents the action probability value after masking correction. To Normalized probability output; This is a preset, extremely small positive number used to replace the probability of an inactive action.

[0132] In the traditional approach, the motion space is approximately However, when the resolution reaches 100×100, it becomes difficult to apply in real-time systems. But with the decoupled action prediction reinforcement learning algorithm model proposed in this embodiment, the computational complexity is reduced to... This means that it is linearly related to the planar resolution, enabling efficient inference at resolutions of 100×100 or even higher, and supporting millimeter-level placement accuracy and drift tolerance.

[0133] In this embodiment, the reinforcement learning algorithm model is an action-value network model consisting of an action network and a value network used to assist in the training of the action network, and the reward function used by the value network during training has the following expression:

[0134] ;

[0135] ;

[0136] ;

[0137] ;

[0138] in, As a reward, For collision-free rewards, Incentives are given for volume utilization. Given the volume of all safe loading positions, if there is no placed object blocking the straight path from the robotic arm access line fixed on the box to a certain loading position, then the loading position is determined to be a safe loading position. This refers to the total volume of the box. Let n be the volume of the object to be packed. As a boundary margin reward increment, when the support stability margin meets the boundary margin threshold requirement, the packing action is rewarded through the boundary margin reward to guide the reinforcement learning algorithm model to prioritize the loading position with higher stability margin. For items to be packed When placed in the corresponding candidate position, the minimum distance from the projection point of its centroid on the bottom surface of the box to the boundary of the support area is used to characterize the support stability margin of the object to be loaded at that position. The preset boundary margin threshold is used to limit the minimum distance required to ensure that the object to be loaded has sufficient support and stability margin. Figure 6 This is a schematic diagram of the safe loading position in this embodiment, wherein... Figure 6 (a) is a schematic diagram of a loading location where there is a risk of collision. Figure 6 (b) is a schematic diagram of a safe loading position where there are no obstructions on the straight path from the robotic arm access line fixed to the container to a certain loading position. The action network and value network can adopt the required network model as needed, such as multilayer perceptron (MLP).

[0139] To ensure the stability and effectiveness of training, the action-value network model uses a combination of infeasibility probability suppression loss during training. The probability of assigning the strategy to positions deemed infeasible by the mask is further penalized; the loss is based on the value backpropagation of the action network and the value network, including the loss of the action network. With the loss of value network This ensures that the gradient estimation variance is controllable and converges stably. The combination of these three loss functions effectively improves the learning efficiency of the policy in large-scale, high-dimensional action spaces. Specifically, in this embodiment, the loss function used during training of the action-value network model is:

[0140] ;

[0141] ;

[0142] ;

[0143] ;

[0144] in, For loss function, , and As weight, For the loss of the action network, For the loss of the value network, The infeasibility probability suppression loss is used to minimize the probability quality of infeasible locations and reduce ineffective exploration. For the first The immediate reward obtained after performing a step action; This is a reward discount factor used to balance current rewards with future benefits. For value networks For the next state Value estimation; For the value network to the current state Value estimation; For Action Networks In the current state Down-output actions The probability, For items to be packed In the direction The corresponding position below The feasibility mask value is used to indicate whether the action is physically feasible.

[0145] In this embodiment, the action-value network model is trained using the PyTorch framework. Training in a 100×100 high-resolution state space takes approximately 12 hours, with a single decision time of less than 10 milliseconds. This performance ensures the real-time requirements of the method in industrial online packing scenarios. Experiments show that the method in this embodiment can stably achieve high space utilization and packing item quantity in different scenarios, verifying the universality and robustness of the method.

[0146] To verify the effectiveness of the online 3D bin packing method with physical stability verification in this embodiment, comprehensive testing was conducted on three datasets: RS, CUT-1, and CUT-2. 2000 sequences were generated for each dataset, covering randomness, theoretical optimal solvability, and dependency constraints. This embodiment specifically addresses feasibility masks. Infeasibility probability suppresses loss and according to An ablation experiment was designed using conditional variable inputs implemented by sequential modulation. The evaluation indicators included space utilization and the number of items. The experimental results are shown in Table 1.

[0147]

[0148] As shown in Table 1, the feasibility mask in the online 3D packing method with physical stability verification in this embodiment is... Infeasibility probability suppresses loss and according to The conditional variable inputs implemented by sequential modulation all contribute positively to the final performance.

[0149] The method in this embodiment is compared with the existing Bullet simulation engine and heuristic rule-based online 3D bin packing method, and evaluated using four indicators: space utilization, number of items, stability probability, and time. The experimental results are shown in Table 2.

[0150]

[0151] In Table 2, the Bullet simulation engine failed to pack the boxes, so the space utilization, item quantity, and stability probability are all "-". Table 2 shows that compared to the Bullet simulation engine's time-consuming calculations and overly conservative heuristic rules, the method in this embodiment achieves stability determination with near-simulation-level accuracy (≈99.9%), balancing high precision and high efficiency.

[0152] The method of this embodiment was deployed in a 3D online bin packing scenario with one-step look-ahead (BPP-1). One-step look-ahead (BPP-1) means that only the current item n to be packed can be seen, and subsequent items cannot be seen, which is a typical online bin packing scenario. In this experiment, space utilization, item quantity, stability rate, and collision rate were used as evaluation indicators. The experimental results are shown in Table 3. The "no-collision penalty" refers to the removal of the no-collision reward. The ablation version is used to verify the collision-free reward proposed in the method of this embodiment. The necessity of training packing strategies.

[0153]

[0154] As shown in Table 3, when the method of this embodiment was deployed in a real production line in a 1-step look-ahead (BPP-1) 3D online packing scenario, verification revealed that the method of this embodiment achieved 100% stacking stability in 50 actual test sequences. (In the absence of collision rewards...) Guided by the instructions, the robot was able to place items in a "from farthest to closest" order, with only one sequence experiencing a slight collision, and no collision-free reward was used. The number of collisions increases significantly. The results shown in Table 3 fully demonstrate that the method in this embodiment does not have a collision reward. It is effective and can achieve good performance while maintaining stack stability and avoiding collisions. It has excellent stability, collision avoidance and executability in industrial environments.

[0155] Furthermore, this embodiment also provides an online 3D packing device with physical stability verification, including a conveyor belt, an RGB-D sensor, and a robotic arm. The conveyor belt is located within the field of view of the RGB-D sensor. The control end of the conveyor belt and the output end of the RGB-D sensor are respectively connected to the control unit of the robotic arm. The control unit of the robotic arm includes a processor and a memory interconnected. The processor is programmed or configured to execute the online 3D packing method with physical stability verification. This embodiment also provides a computer-readable storage medium storing a computer program or instructions programmed or configured to execute the online 3D packing method with physical stability verification via a processor. This embodiment also provides a computer program product including a computer program or instructions programmed or configured to execute the online 3D packing method with physical stability verification via a processor.

[0156] Those skilled in the art will understand that the technical solutions provided by this invention may take the form of a method, system, or computer program product. Therefore, this invention may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this invention may take the form of a computer program product embodied on one or more computer-readable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, produce an implementation of the flowchart... Figure 1 One or more processes and / or boxes Figure 1 The computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1The functions specified in one or more boxes. These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable apparatus for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0157] The above description is merely a preferred embodiment of the present invention. The scope of protection of the present invention is not limited to the above embodiments. All technical solutions falling within the scope of the present invention's concept are within the scope of protection of the present invention. It should be noted that for those skilled in the art, any improvements and modifications made without departing from the principle of the present invention should also be considered within the scope of protection of the present invention.

Claims

1. An online three-dimensional packing method with physical stability verification, characterized in that, Includes selecting items to be loaded Before placing the item in the container, obtain the height map of the loaded items in each grid within the container. Items to be packed The dimensions, and based on the height of the contents in each grid within the container. Items to be packed The dimensions were physically stable to generate a representation of the object to be loaded. Feasibility mask for candidate placement locations : S101, based on dimensions Diagram showing the height of items loaded in each grid within the container. Enumerate all possible candidate placement positions, with an initial generation size of . Feasibility mask ,in , and These are the length, width, and height of the container, and the feasibility mask. Elements outside the candidate placement position have a value of 0; S102, iterate through all candidate placement positions and select a current candidate placement position. ; S103, for the contents Place candidate placement locations Perform a static centroid stability test. If the centroid stability test passes, proceed to step S104; otherwise, determine the feasibility mask. Current candidate placement position The element has a value of 0; jump to step S105; S104, for the contents Place candidate placement locations Perform a centroid stability test on the mass distribution transfer. If the centroid stability test on the mass distribution transfer passes, then determine the feasibility mask. Current candidate placement position The element is set to 1; otherwise, a feasibility mask is determined. Current candidate placement position The element takes the value 0; S105, determine if all candidate placement positions have been traversed. If not, jump to step S102 to continue traversing; otherwise, determine the feasibility mask. Once generated, the feasibility mask... The value of each element in the grid is 1 or 0, indicating whether the corresponding grid can be used as a loading object. Candidate placement positions; In step S104, the object to be loaded Place candidate placement locations The centroid stability test for mass distribution transfer includes: S201, Loading the item Place in the current candidate placement The support relationships are modeled as directed graphs. Directed graph The nodes in the text are objects, and the objects include items to be loaded. The relationship between the object and the loaded object, with the edges representing the supporting relationships of the object; S202, from the directed graph Items to be loaded Starting from the node of contact, the recursive construction only includes the object to be loaded, whether directly or indirectly subjected to it. All items already loaded in the mass distribution are considered as items to be loaded. Adaptive stacked tree The adaptive stacked tree For directed graphs Chinese and Israeli items to be loaded This is a subgraph of the root node, where each node directly or indirectly bears the load to be loaded. The mass distribution of the loaded material; S203, from adaptive stacked tree Starting from the root node, traverse towards the leaf nodes, for each current node obtained during the traversal... Determine the current node The number of child nodes is determined. If the number of child nodes is 1, proceed to step S204; if the number of child nodes is 2, proceed to step S205; if the number of child nodes is 2 or more, proceed to step S206; if it is an adaptive stacking tree... Once the traversal is complete, proceed to step S207. S204, for the current node child nodes Calculate the distance from the root node to the child nodes using the following formula. The projection points of the centroid of the node group onto the XY plane on the bottom surface of the box: ； in, From root node to child nodes The coordinates of the centroid of the node group projected onto the XY plane on the bottom surface of the box. and Each is the current node child nodes The mass of the object, and Each is the current node child nodes Find the coordinates of the center of mass projected onto the XY plane on the bottom surface of the box; jump to step S203 to continue traversing; S205, for the current node Each child node Calculate the distance from the root node to the current node according to the following formula. child nodes Node group quality distribution: ； ； in, child node Supporting force It is the acceleration due to gravity. For the current node The coordinates of the center of mass projected onto the XY plane on the bottom surface of the box. and child nodes and child nodes With the current node The coordinates of the midpoint of the bottom contact surface, child nodes and child nodes The current node is formed when the number of child nodes is 2. The two child nodes, From the root node to the current node child nodes The quality distribution of the node group, child node The mass of the object; calculate the distance from the root node to each child node according to the following formula. The projection points of the centroid of the node group onto the XY plane on the bottom surface of the box: ； in, From root node to child nodes The coordinates of the centroid of the node group projected onto the XY plane on the bottom surface of the box. child node Find the coordinates of the center of mass projected onto the XY plane on the bottom surface of the box; jump to step S203 to continue traversing; S206, for the current node Each child node Determine child nodes With the current node Bottom contact support point ,according to Calculate the current node The center of mass is projected onto the XY plane on the bottom surface of the box. To each support point distance And calculate the child nodes according to the following formula. Support force : ； in, For the current node The center of mass is projected onto the XY plane on the bottom surface of the box. To support point distance, child node With the current node The bottom contact support point, For the current node The corresponding total mass of the node group For the current node, gravitational acceleration is used. Each child node Calculate the distance from the root node to the current node according to the following formula. child nodes Node group quality distribution: ； Calculate the distance from the root node to each child node according to the following formula. The projection points of the centroid of the node group onto the XY plane on the bottom surface of the box: ； Jump to step S203 and continue traversing; S207, calculate the convex hull region of the bottom support point of the loaded object corresponding to each leaf node; determine whether the XY plane projection point of the centroid of the node group corresponding to each leaf node on the bottom surface of the box falls within the convex hull region corresponding to the leaf node. If the XY plane projection point of the centroid of the node group corresponding to each leaf node on the bottom surface of the box falls within the convex hull region corresponding to the leaf node, the centroid stability test of mass distribution transfer is deemed to have passed; otherwise, the centroid stability test of mass distribution transfer is deemed to have failed.

2. The online three-dimensional packing method with physical stability verification according to claim 1, characterized in that, In step S101, based on the dimensions... Height diagram of each grid in the box When enumerating all possible candidate placement locations, the constraints that all possible candidate placement locations must satisfy are: ； ； in, For items to be packed Placement starting coordinates On the axis The value on, For items to be packed On the axis The dimensions and length on top For the box body on the axis The dimensions and length on top Indicates the item to be loaded and already loaded items Along axis The sequential relationship variable, with a value of 1 indicating the items to be loaded. The items have already been loaded. If it is in front of, to the left of, or below, then it is 0; otherwise, it is 0. A collection of already loaded items; axis for One of the three axes.

3. The online three-dimensional packing method with physical stability verification according to claim 1, characterized in that, In step S103, the object to be loaded Place candidate placement locations Static centroid stability testing includes: calculating the load. Place in the current candidate placement Calculate the convex hull region of all bottom support points by projecting the center of mass onto the XY plane on the bottom surface of the box and the bottom support points. Determine whether the projection of the center of mass onto the XY plane on the bottom surface of the box falls within the convex hull region. If the projection of the center of mass onto the XY plane on the bottom surface of the box falls within the convex hull region, the center of mass stability test is considered to have passed; otherwise, the center of mass stability test is considered to have failed.

4. The online three-dimensional packing method with physical stability verification according to claim 1, characterized in that, The selection of items to be loaded Placement locations within the enclosure include: S301, Obtain the item to be loaded. The packing environment status, including the height diagram of the packed items in each grid within the box. Items to be packed Size tensor and items to be packed Feasibility mask for candidate placement locations : S302, the packing environment state is used to generate a state feature vector through a state convolutional neural network. ; S303, the state feature vector Predict X-direction actions using the x-output head of the action network from a pre-trained reinforcement learning algorithm model. ; the state feature vector and X-direction movement Predicting Y-direction actions using the y-output head in an action network ; the state feature vector X-direction movement and Y-direction movement Predicting orientation actions using the o-output head in an action network ; S304, regarding the predicted X-direction motion Y-direction movement and directional movements Combined with feasibility mask Modulation is performed dimensionally in the order of X-direction, Y-direction, and orientation to obtain the object to be loaded. Packing process inside the box To obtain the items to be loaded The placement position within the box, among which , and These represent the movements in the X direction, Y direction, and orientation, respectively, and represent the offset in the X direction, the offset in the Y direction, and the placement angle.

5. The online three-dimensional packing method with physical stability verification according to claim 4, characterized in that, In step S304, a feasibility mask is incorporated. Modulation is performed dimensionally in the order of X-direction, Y-direction, and orientation to obtain the object to be loaded. Packing process inside the box To obtain the items to be loaded The function expression for the placement position within the box is: ； ； ； ； in, For feasibility masking Modulated X-direction motion The probability distribution, For modulation function, Current packing environment status The X-direction motion is predicted by the x-output head. The probability distribution, Feasibility mask The X-direction component, To perform an action in a given X direction After feasibility mask M n Modulated Y-direction motion The probability distribution, To be in a given current packing environment state Under the condition of movement in the X direction The original Y-direction movement The probability distribution, Feasibility mask Action in the X direction under the condition Movement in the Y direction The feasibility mask value components, To perform an action in a given X direction Movement in the Y direction After feasibility masking Modulated orientation movement The probability distribution, To be in a given current packing environment state Under the condition of movement in the X direction Movement in the Y direction The original orientation of the movement The probability distribution, Feasibility mask Items to be loaded In the direction of the action The next corresponding X-direction action Movement in the Y direction The feasibility mask value is used to indicate whether the action is physically feasible; For the first The unmodulated probability value of each candidate action in the original action distribution. The value is assigned to the feasibility mask corresponding to the candidate action position. This represents the action probability value after masking correction. To Normalized probability output; This is a preset, extremely small positive number used to replace the probability of an inactive action.

6. The online three-dimensional packing method with physical stability verification according to claim 5, characterized in that, The reinforcement learning algorithm model is an action-value network model consisting of an action network and a value network used to assist in the training of the action network, and the reward function used by the value network during training has the following functional expression: ； ； ； ； in, As a reward, For collision-free rewards, Incentives are given for volume utilization. Given the volume of all safe loading positions, if there is no placed object blocking the straight path from the robotic arm access line fixed on the box to a certain loading position, then the loading position is determined to be a safe loading position. This refers to the total volume of the box. Let n be the volume of the object to be packed. The incremental reward is for the boundary margin. For items to be packed When placed in the corresponding candidate position, the minimum distance from the projection point of its centroid on the bottom surface of the box to the boundary of the support area is used to characterize the support stability margin of the object to be loaded at that position. The preset boundary margin threshold is used to limit the minimum distance required to ensure that the object to be loaded has sufficient support and stability margin. The loss function used during training of the action-value network model is: ； ； ； ； in, For loss function, , and As weight, For the loss of the action network, For the loss of the value network, To mitigate losses due to infeasibility probability, For the first The immediate reward obtained after performing a step action; This is a reward discount factor used to balance current rewards with future benefits. For the value network to the next state Value estimation; For the value network to the current state Value estimation; For Action Networks In the current state Down-output actions The probability, For items to be packed In the direction The corresponding position below The feasibility mask value is used to indicate whether the action is physically feasible.

7. An online three-dimensional packing device with physical stability verification, characterized in that, The method includes a conveyor belt, an RGB-D sensor, and a robotic arm. The conveyor belt is located within the field of view of the RGB-D sensor. The control end of the conveyor belt and the output end of the RGB-D sensor are respectively connected to the control unit of the robotic arm. The control unit of the robotic arm includes a processor and a memory connected to each other. The processor is programmed or configured to execute the online three-dimensional packing method with physical stability verification as described in any one of claims 1 to 6.

8. A computer-readable storage medium storing a computer program or instructions, characterized in that, The computer program or instructions are programmed or configured to execute, via a processor, the online three-dimensional packing method with physical stability verification as described in any one of claims 1 to 6.

9. A computer program product, comprising a computer program or instructions, characterized in that, The computer program or instructions are programmed or configured to execute, via a processor, the online three-dimensional packing method with physical stability verification as described in any one of claims 1 to 6.

Images