Three-dimensional packaging processing method, device, electronic equipment and storage medium

Abstract

The three-dimensional packing processing method, device, electronic device and storage medium provided by the present invention can standardize the packing problem instances to be processed, provide a data structure that can accurately describe the packing state, and provide a basis for rapid iteration of subsequent optimization models. Based on the reinforcement learning algorithm, a more efficient optimization model for the three-dimensional packing problem is given. The optimization model can automatically match the corresponding specific problems through learning, and can adapt to the packing problems of box-carriages of different scales.

Description

Technical Field

[0001] The present invention relates to the technical field of three-dimensional regular object packing, and more specifically, to a three-dimensional packing processing method, device, electronic equipment and storage medium. Background Art

[0002] The three-dimensional bin packing problem is a type of bin packing problem (also known as the cutting stock problem). In most actual production needs, solving the three-dimensional bin packing problem means using one or more algorithms to find a relatively optimal answer within a certain period of time.

[0003] At present, a commonly used class of meta-heuristic algorithms has high flexibility and is widely used in NP optimization problems. However, correspondingly, this type of algorithm lacks the use of specific problem properties and information, is broad but less targeted, and often has room for optimization improvement compared to heuristic algorithms in the initial value selection and solution iteration process. Summary of the invention

[0004] In view of this, in order to solve the above problems, the present invention provides a three-dimensional packing processing method, device, electronic device and storage medium, and the technical solution is as follows:

[0005] A three-dimensional boxing processing method, the method comprising:

[0006] Obtaining a first packing problem instance to be processed, wherein the first packing problem instance includes description data of a box and a spatial state of a carriage;

[0007] Standardizing the description data of the box and the spatial state of the carriage in the first packing problem instance to obtain the standard description data of the box and the standard spatial state of the carriage in the first packing problem instance;

[0008] Inputting the standard description data of the box and the standard spatial state of the carriage in the first packing problem instance into a three-dimensional packing model, wherein the three-dimensional packing model is obtained by training a neural network based on a reinforcement learning algorithm in advance;

[0009] Obtain a packing solution with the highest actual loading rate output by the three-dimensional packing model for the first packing problem instance.

[0010] Preferably, the process of pre-training a neural network based on a reinforcement learning algorithm to obtain the three-dimensional packing model includes:

[0011] Obtaining a second packing problem instance for training, wherein the second packing problem instance includes description data of the box and a spatial state of the compartment;

[0012] Standardizing the description data of the box and the spatial state of the carriage in the second packing problem instance to obtain standard description data of the box and the standard spatial state of the carriage in the second packing problem instance;

[0013] Determine a packing action of the second packing problem instance based on a greedy strategy, and process standard description data of the box in the second packing problem instance based on the packing action to update a standard space state of the carriage in the second packing problem instance until a preset packing termination state is reached;

[0014] Obtain the standard spatial state of the carriage under the packing termination state, discretize it into a three-dimensional matrix and calculate the actual loading rate of the second packing problem instance under the packing termination state;

[0015] The three-dimensional matrix and the actual loading rate are input into a neural network so that the neural network adjusts the weight coefficients of each layer with the maximum actual loading rate as the target until the preset training end condition is met, and the neural network is used as a three-dimensional packing model.

[0016] Preferably, the standard description data is an octet array:

[0017] (the length of the box, the width of the box, the height of the box, the coordinates of the box in the length direction of the compartment where it is located, the coordinates of the box in the width direction of the compartment where it is located, the coordinates of the box in the height direction of the compartment where it is located, the loading orientation of the box, and the number of the compartment where the box is located);

[0018] The standard space state is a secondary array:

[0019] [(the height of the starting line segment, (the lateral position of the starting line segment, the length of the starting line segment))], [(the height of the ending line segment, (the lateral position of the ending line segment, the length of the ending line segment))];

[0020] The starting line segment represents the starting loading position of the carriage, and the ending line segment represents the ending loading position of the carriage.

[0021] Preferably, the processing of the standard description data of the box in the second packing problem instance based on the packing action to update the standard space state of the carriage in the second packing problem instance includes:

[0022] Determining a target box to be moved and a loading position of the target box based on the box packing action;

[0023] Determine the target compartment where the target box is located and the current target standard space state of the target compartment according to the loading position of the target box;

[0024] The relative position relationship between the box and the start line segment and the end line segment in the target standard space state is determined by using the length, width and height of the target box, and the start line segment and the end line segment in the target standard space state are updated based on the relative position relationship.

[0025] Preferably, updating the start line segment and the end line segment in the target standard space state based on the relative position relationship includes:

[0026] Determine a target line segment to be updated in the target standard space state based on the relative position relationship, wherein the target line segment includes a start line segment and / or an end line segment;

[0027] Determine the intersection of the target line segment and the target box according to the length, width and height of the box in the standard description data of the target box, and add the non-intersection part to the target line segment to update the target line segment;

[0028] The other line segments that have not been updated in the target standard space state and the updated target line segments are combined and calculated.

[0029] A three-dimensional box processing device, the device comprising:

[0030] An instance acquisition module, used to acquire a first packing problem instance to be processed, wherein the first packing problem instance includes description data of a box body and a spatial state of a carriage;

[0031] A standardization module, used for standardizing the description data of the box and the spatial state of the carriage in the first packing problem instance to obtain the standard description data of the box and the standard spatial state of the carriage in the first packing problem instance;

[0032] A packing processing module is used to input the standard description data of the box and the standard spatial state of the carriage in the first packing problem instance into a three-dimensional packing model, wherein the three-dimensional packing model is obtained by pre-training a neural network based on a reinforcement learning algorithm; and obtain a packing solution with the highest actual loading rate output by the three-dimensional packing model for the first packing problem instance.

[0033] Preferably, the process of the packing processing module pre-training a neural network based on a reinforcement learning algorithm to obtain the three-dimensional packing model includes:

[0034] A second packing problem instance for training is obtained, wherein the second packing problem instance includes description data of a box and a spatial state of a car body; the description data of the box and the spatial state of the car body in the second packing problem instance are standardized to obtain standard description data of the box and standard spatial state of the car body in the second packing problem instance; a packing action of the second packing problem instance is determined based on a greedy strategy, and the standard description data of the box in the second packing problem instance is processed based on the packing action to update the standard spatial state of the car body in the second packing problem instance until a preset packing termination state is entered; the standard spatial state of the car body in the packing termination state is obtained, discretized into a three-dimensional matrix, and the actual loading rate of the second packing problem instance in the packing termination state is calculated; the three-dimensional matrix and the actual loading rate are input into a neural network, so that the neural network adjusts the weight coefficients of each layer with the maximum actual loading rate as the target, until the preset training end condition is met, and the neural network is used as a three-dimensional packing model.

[0035] Preferably, the standard description data is an octet array:

[0036] (the length of the box, the width of the box, the height of the box, the coordinates of the box in the length direction of the compartment where it is located, the coordinates of the box in the width direction of the compartment where it is located, the coordinates of the box in the height direction of the compartment where it is located, the loading orientation of the box, and the number of the compartment where the box is located);

[0037] The standard space state is a secondary array:

[0038] [(the height of the starting line segment, (the lateral position of the starting line segment, the length of the starting line segment))], [(the height of the ending line segment, (the lateral position of the ending line segment, the length of the ending line segment))];

[0039] The starting line segment represents the starting loading position of the carriage, and the ending line segment represents the ending loading position of the carriage.

[0040] An electronic device comprises: at least one memory and at least one processor; the memory stores a program, the processor calls the program stored in the memory, and the program is used to implement any one of the three-dimensional packing processing methods.

[0041] A storage medium stores computer executable instructions, wherein the computer executable instructions are used to execute any one of the three-dimensional box packing processing methods.

[0042] Compared with the prior art, the present invention has the following beneficial effects:

[0043] The three-dimensional packing processing method, device, electronic device and storage medium provided by the present invention can standardize the packing problem instances to be processed, provide a data structure that can accurately describe the packing state, and provide a basis for rapid iteration of subsequent optimization models, thereby providing a more efficient optimization model for the three-dimensional packing problem based on the reinforcement learning algorithm. The optimization model can automatically match the corresponding specific problems through learning and can adapt to the packing problems of boxes-carriages of different scales. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0045] Figure 1 A method flow chart of a three-dimensional box processing method provided by an embodiment of the present invention;

[0046] Figure 2 An example of box loading provided by an embodiment of the present invention;

[0047] Figure 3 An example of a carriage provided in an embodiment of the present invention;

[0048] Figure 4 Another method flow chart of the three-dimensional box processing method provided by an embodiment of the present invention;

[0049] Figure 5 Another box loading example provided by an embodiment of the present invention;

[0050] Figure 6 Another box loading example provided by an embodiment of the present invention;

[0051] Figure 7 A schematic structural diagram of a three-dimensional box processing device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0052] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0053] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0054] The packing problem is a relatively classic and complex problem in discrete combinatorial optimization mathematics, which mainly includes three types: one-dimensional packing, two-dimensional (plane) packing and three-dimensional packing. In the actual production and transportation process, solving the three-dimensional packing problem of regular rectangular bodies (also called cuboids) has a wide range of practical significance, such as material cutting, truck container loading and transportation, warehouse storage, etc. However, in general, packing problems are NP-complete problems. At present, there is no algorithm to obtain an exact solution to NP-complete problems in an effective time. If you try to use an exhaustive method to get the answer, the calculation time increases exponentially with the complexity of the problem, which is often unavailable at the order of magnitude of the actual problem. Therefore, although the packing problem has been widely discussed and studied since the early 1970s, and can even be traced back to the layout problem studied by Gauss in 1831, the next adaptation, first adaptation, descending next adaptation and reconciliation algorithms that have appeared one after another are all approximate algorithms. At present, there is no theoretical result or numerical algorithm to obtain an exact solution in an effective time. Therefore, in most actual production needs, solving the three-dimensional packing problem means using one or more algorithms to seek a relatively better answer within a certain time.

[0055] One type of algorithm that comes to mind is the heuristic algorithm, such as the first-fit algorithm and the best-fit algorithm. This type of algorithm is often designed and solved based on a certain packing problem, so it is relatively fit to the problem, has a high efficiency in solving, and performs well in terms of results. However, this type of algorithm also has disadvantages. They often have large differences in effectiveness in different instances, are not very versatile, and cannot meet the future demand for highly automated production logistics and other links.

[0056] In order to enhance the flexibility of the algorithm, another type of meta-heuristic algorithm has emerged, including Ant Colony Algorithm, Genetic Algorithm, Particle Swarm Algorithm, Simulated Annealing Algorithm, Tabu Search, etc. This type of algorithm can automatically search for feasible solutions in the feasible solution space (Feasible Set), and heuristically select, combine, and regenerate the searched feasible solutions. It can also loop this process, so as to search for a better solution within a limited solution time, without relying on professional analysis and experience for specific problems. Therefore, this type of meta-heuristic algorithm has high flexibility and is widely used in NP optimization problems. However, correspondingly, this type of algorithm lacks the use of specific problem properties and information, and is extensive but less targeted. Compared with heuristic algorithms, there is often room for optimization improvement in the initial value selection and solution iteration process.

[0057] In order to develop a three-dimensional packing algorithm with a certain degree of wide application and problem-specificity, the present invention proposes a data structure that accurately describes the packing state. This data structure can describe the position of any box in the entire carriage after each packing, whether any spatial point is occupied by a box, and other states. At the same time, the update efficiency is high, providing a basis for rapid iteration for subsequent optimization models. Then, the present invention provides a more efficient optimization model for the three-dimensional packing problem based on the reinforcement learning algorithm. The optimization model can automatically match the corresponding specific problems through learning. Since this process is automatically implemented by the optimization model, it can adapt to the packing problems of boxes and carriages of different scales.

[0058] The embodiment of the present invention provides a three-dimensional box processing method, the method flow chart of which is as follows: Figure 1 As shown, the following steps are included:

[0059] S10, obtaining a first packing problem instance to be processed, wherein the first packing problem instance includes description data of the box body and the spatial state of the compartment.

[0060] In an embodiment of the present invention, the packing problem instance to be processed can be set by the user, and the user can indicate the descriptive data of the box in the instance, such as length, width, height, loading orientation, etc., and the spatial state of the carriage, i.e., the length, width and height of the carriage.

[0061] S20, standardizing the description data of the box and the spatial state of the carriage in the first packing problem instance to obtain standard description data of the box and the standard spatial state of the carriage in the first packing problem instance.

[0062] In the embodiment of the present invention, an efficient update data structure can be used for the description data of the box and the spatial state of the carriage:

[0063] The standard description data is an octet array - (the length of the box, the width of the box, the height of the box, the coordinates of the box in the length direction of the carriage, the coordinates of the box in the width direction of the carriage, the coordinates of the box in the height direction of the carriage, the loading orientation of the box, and the number of the carriage where the box is located).

[0064] Specifically, select a suitable unified unit, such as meters or centimeters, and the subsequent data will be recorded in this unit. Each box is represented by an octet array (l i , w i ,h i , x i =0,y i =0, z i =0,d i =0,n i =0) for initialization, where (l i , w i ,h i ) are the length, width and height of box i, respectively, (x i ,y i , z i , d i , n i ) respectively represent the three-dimensional coordinates of the loading position of the box according to the length, width and height of the car where it is located, as well as the loading orientation of the box and the number of the car where the box is located.

[0065] See also Figure 2 In the box loading example shown, the standard description data for box A is (2.15, 0.47, 0.52, 0, 0, 0, 0, 0).

[0066] Regarding the loading orientation of the box, depending on the problem constraints, only one side may be facing upward or multiple sides may be facing upward, etc.

[0067] In addition, the standard spatial state is a secondary array - [(height of the starting line segment, (lateral position of the starting line segment, length of the starting line segment))], [(height of the ending line segment, (lateral position of the ending line segment, length of the ending line segment))];

[0068] The starting line segment represents the starting position of the carriage for packing, and the ending line segment represents the ending position of the carriage for packing.

[0069] Specifically, for each carriage, the secondary array [(0, (0, (0, width)))], [(0, (length, (0, width)))] is used for initial normalization. Figure 3In the example of the carriage shown, the secondary array of the carriage during initial standardization is [(0, (0, (0, 2.40)))], [(0, (5.60, (0, 2.40)))]. The secondary array respectively represents the starting segment (i.e., segment a) and the ending segment (i.e., segment b) of the loading position of the box in the carriage.

[0070] Because when packing, the bottom surface of the box has two sets of parallel edges, and at least one edge in each set will be close to the edge of another box or the wall of the car. In other words, after the box is loaded, it can only move to one side at most in the x-axis direction and the y-axis direction. Otherwise, it means that the box is loaded in the air on all sides, and there is still room to move front and back (or left and right). Therefore, we characterize the possible loading position as follows: an edge that is parallel to the width of the car and can completely overlap with the front edge of the bottom of the box when the box is completely and stably loaded (that is, the box does not overlap with other boxes and the box is not suspended on the ground). Figure 3 The front edge of the bottom surface of the box body refers to the edge of the bottom surface of the box body that is parallel to the vehicle width direction and has a smaller lateral position.

[0071] As the boxes are loaded into the carriage, the information recorded in the above arrays gradually increases, and both arrays will eventually appear in the following format:

[0072] [(h1, (p1, (a1, b1), (a2, b2), (a3, b3),...), (p2, (a1, b1), (a2, b2), (a3, b3),...),...),

[0073] (h2, (p1, (a1, b1), (a2, b2), (a3, b3),...), (p2, (a1, b1), (a2, b2), (a3, b3),...),...), (h3, (...)),...]

[0074] Where h1, h2, h3, ... represent the height of the recorded line segment, p1, p2, p3, ... represent the horizontal position of the recorded line segment, (a1, b1), (a2, b2), ... represent the left and right endpoints of the recorded line segment at the height and horizontal position. Figure 2 For example,

[0075] At this time, the standard space state of the carriage can be expressed as:

[0076] Start Segment

[0077] [(0, (2.15, (0, 0.47)), (1.07, (0.47, 2.21)), (1.42, (2.21, 2.40))),

[0078] (0.52, (0, (0, 0.47))), (0.89, (0, (0.47, 2.40))), (1.23, (1.07, (2.21, 2.40)))]

[0079] End Segment

[0080] [(0, (5.60, (0, 2.40))), (0.52, (2.15, (0, 0.47))), (0.89, (1.07, (0.47, 2.40))),

[0081] (1.23, (1.42, (2.21, 2.40)))]

[0082] Under this data structure, the loading position only needs to start from the starting segment or its combination, and check that it cannot cross the ending segment.

[0083] S30, inputting the standard description data of the box body and the standard spatial state of the carriage in the first packing problem instance into a three-dimensional packing model, where the three-dimensional packing model is obtained by pre-training a neural network based on a reinforcement learning algorithm.

[0084] In the embodiment of the present invention, reinforcement learning is a Markov decision process, which mainly includes three elements: state, action, and reward. Based on the aforementioned data format, the three-dimensional packing optimization problem can be converted into a form that conforms to the general model of reinforcement learning:

[0085] 1) State. The state of the three-dimensional packing at any moment includes the spatial state of the compartment and all the boxes that are not loaded into the car. Using the above data format, the state of each spatial point in the compartment can be clearly described, plus the list of boxes that are not loaded into the car (that is, the list of octets [(l i , w i ,h i , 0, 0, 0, 0, 0), ..., i = {box not loaded into the car}]), this set of computer data can establish a one-to-one correspondence with the state of three-dimensional box loading, and fully and accurately describe the state in the reinforcement learning framework. Here are the concepts of intermediate state, terminal state and sub-state. The intermediate state refers to the state in which at least one box has not been loaded, and the terminal state refers to the state in which all boxes are loaded. For any intermediate state s, state s′ is obtained after a box is put into state s, and we call s′ a sub-state of s.

[0086] It should be noted that in other vehicles with a limited number, where it is necessary to consider whether all boxes can be loaded, any space in any compartment cannot accommodate any unloaded box, which can also be defined as a termination state, and the present invention is still applicable.

[0087] 2) Action. In the above data format, it is assumed that the box and loading position for each step have been selected. In the three-dimensional packing problem, these selections are actions. In the construction of the data format, we assume the fact that when packing, the two sets of parallel edges on the bottom of the box, each set has at least one edge that is close to another box or the wall of the carriage. In other words, when the box is loaded, it can only move to one side at most in the x-axis direction and the y-axis direction. Otherwise, it means that the box is suspended on all sides, and there is still room to move in the front and back (or left and right).

[0088] Since the goal of optimization is to maximize the actual loading rate, this default fact is reasonable. On this basis, a series of actions can be derived, such as "select the longest box / the highest box / the box closest to the starting (green) side length / the box with the largest volume, etc.", and "select the box and place it on the left / place it on the right / stack it high first / stack it on the ground first, etc.", or "select the box closest to the starting (green) side length" and so on. After selecting the edge where the box can be placed, choose "place it on the left / place it on the right / stack it high first / stack it on the ground first, etc.", and choose "the longest box / the highest box / the box closest to the starting (green) side length / the box with the largest volume", etc., which will form a complete action. In the heuristic algorithm, the selection method of the above actions will be limited in advance, and in the reinforcement learning framework, this selection process will be implemented by the algorithm.

[0089] 3) Reward, value and value function. In some problems, there are quantifiable values ​​of intermediate states, and rewards and values ​​can be designed accordingly. However, in the middle of the three-dimensional packing process, it is difficult to measure the quality of a state. With the help of a simple indicator, the actual loading rate will increase and decrease during the packing process, and it is not monotonically increasing, so we design the reward for the intermediate step to be 0. Therefore, for any intermediate state s, there is a value function:

[0090]

[0091] For all terminal states s, set its value function V(s) to the actual loading rate. Generally speaking, the Q-learning method is often used to connect these elements, that is, to construct and use the state-action value function Q(s, a) (state-action value function).

[0092] In the three-dimensional packing problem, taking an action in any state will deterministically transform to a new state, so the process of constructing the state-action value function Q(s, a) is simplified, and only the value function V(s) is needed. Based on the order of magnitude of the actual problem, in most cases, we cannot traverse to obtain the V(s) value of the state, so the present invention uses a three-dimensional convolutional neural network to construct a function with parameters To fit V(s), where the parameter w∈R d is the weight of the neural network, and we can see that d<<|{s}|. Finally, the result is output through several fully connected layers, and the loss function is the mean square error function.

[0093] In the specific implementation process, the process of pre-training the neural network based on the reinforcement learning algorithm to obtain the three-dimensional box packing model includes the following steps. The method flow chart is as follows: Figure 4 As shown:

[0094] S101, obtaining a second packing problem instance for training, wherein the second packing problem instance includes description data of the box and the spatial state of the compartment.

[0095] In the embodiment of the present invention, the samples used for training contain a certain number of bin packing problem instances.

[0096] S102, standardizing the description data of the box and the spatial state of the carriage in the second packing problem instance to obtain standard description data of the box and the standard spatial state of the carriage in the second packing problem instance.

[0097] In the embodiment of the present invention, for the standardization process of the description data of the box and the spatial state of the compartment in the second packing problem instance, reference can be made to the standardization process of the description data of the box and the spatial state of the compartment in the first packing problem instance in the above step S20, which will not be repeated here.

[0098] S103, determining a packing action of the second packing problem instance based on a greedy strategy, and processing standard description data of the box in the second packing problem instance based on the packing action to update the standard space state of the carriage in the second packing problem instance until a preset packing termination state is reached.

[0099] In the embodiment of the present invention, an ∈-greedy strategy is used to determine the packing action to obtain the next standard space state. The relevant parameters of the greedy strategy can be preset and are not limited here.

[0100] That is, for a state s0, let its sub-state set be S0, and we have

[0101]

[0102] Until the termination state.

[0103] Each time a box is loaded, the standard description information of the box and the above secondary array will be updated as follows:

[0104] Update the number and 3D position of the boxes loaded into the carriage.

[0105] Update the start and end segments of the corresponding positions according to the box position.

[0106] In order to clearly illustrate this process, Figure 2 For example, put a new box B, that is, according to the packing action, the target box to be moved is box B, and the target carriage is Figure 2 The specific loading position of the carriage shown is as follows Figure 5 shown.

[0107] Further, the length, width and height of box B are used to determine the Figure 2 The relative position relationship between the start line segment and the end line segment in the target standard space state is calculated, and the start line segment and the end line segment in the target standard space state are updated based on the relative position relationship.

[0108] Specifically, a target line segment to be updated in the target standard space state is determined based on the relative position relationship, the target line segment including a start line segment and / or an end line segment; the intersection of the target line segment and the target box is determined according to the length, width, and height of the box in the standard description data of the target box, and the non-intersection part is supplemented to the target line segment to update the target line segment; other line segments that are not updated in the target standard space state and the updated target line segment are calculated.

[0109] Combination Figure 5 From the above, we can see that the placement of box B will affect the line segment sets at 4 groups of positions, so the line segment sets at the positions indicated by the 4 arrows need to be updated:

[0110] Update the standard description information of the box, including the three-dimensional coordinates of the vertex of the box (closest to the origin), the loading orientation (for simplicity, it is 0, i.e., no rotation), and the number of the carriage where the box is located. That is, (1.41, 1.23, 0.52, 0, 0, 0, 0) --> (1.41, 1.23, 0.52, 1.42, 1.17, 0, 0, N).

[0111] Update the line segment set information at the positions pointed by the four arrows respectively. The principle is to find the affected line segments in this line segment set and modify them, and then calculate the set union with the remaining unaffected line segments.

[0112] Let’s take an edge as an example.

[0113] Position 1: left edge of the bottom of the box, height 0, horizontal position 1.42,

[0114] Step 1: Read the position information from the starting segment array

[0115] [(0, (2.15, (0, 0.47)), (1.07, (0.47, 2.21)), (1.42, (2.21, 2.40))),

[0116] (0.52, (0, (0, 0.47))),

[0117] (0.89, (0, (0.47, 2.40))),

[0118] (1.23, (1.07, (2.21, 2.40)))]

[0119] Read the position information from the end segment array and find that there is no information at this position

[0120] [(0, (5.60, (0, 2.40)))(0.52, (2.15, (0, 0.47))),

[0121] (0.89, (1.07, (0.47, 2.40))),

[0122] (1.23, (1.42, (2.21, 2.40)))]

[0123] Step 2: Add the newly generated line segment (1.17, 2.40) for box A to the information of position 1. Since the upper right side of position 1 is the position of the box, this edge will cover the old starting line segment:

[0124] (0, (1.42, (2.21, 2.40))) + (0, (1.42, empty set)) vs (1.17, 2.40)

[0125] Remove (1.17, 2.40) from (2.21, 2.40) and add the original starting segment (which is an empty set here), and the remaining set is an empty set. So the starting segment array is updated to

[0126]

[0127] (0.52, (0, (0, 0.47))),

[0128] (0.89, (0, (0.47, 2.40))),

[0129] (1.23, (1.07, (2.21, 2.40)))]

[0130] At the same time, a new ending line segment is created:

[0131] (0, (1.42, (2.21, 2.40))) + (0, (1.42, empty set)) vs (1.17, 2.40),

[0132] Remove (2.21, 2.40) from (1.17, 2.40), and add the original end segment (empty set here), leaving (1.17, 2.21), so the end segment array is updated (added)

[0133] [(0, (5.60, (0, 2.40)), (1.42, (1.17, 2.21)),)

[0134] (0.52, (2.15, (0, 0.47))),

[0135] (0.89, (1.07, (0.47, 2.40))),

[0136] (1.23, (1.42, (2.21, 2.40)))]

[0137] Here, the line segments are intersected and complemented.

[0138] Position 2: The right edge of the bottom of the box, height 0, horizontal position 2.83. Since the upper left side of position 2 is the position of the box, this edge will create a new starting line segment, and the set intersection and complement operations are the same.

[0139] Position 3: The left edge of the top surface of the box, height 0.52, horizontal position 1.42. Since the lower right side of position 3 is the position of the box, this edge will cover the old end line segment, creating a new start line segment, and the same applies to the set intersection and complement operations.

[0140] Position 4: The left edge of the bottom of the box, height 0.52, horizontal position 2.83. Since the lower left side of position 4 is the position of the box, this edge will create a new end line segment, and the set intersection and complement operations are the same.

[0141] After the above four rounds of array updates are completed, the new carriage status is as follows:

[0142] Start Segment

[0143] [(0, (2.15, (0, 0.47)), (1.07, (0.47, 2.21)), (1.42 (no information)), (2.83, (1.17, 2.40))),

[0144] (0.52, (0, (0, 0.47)), (1.42, (1.17, 2.40)), (2.83, (no information)),

[0145] (0.89, (0, (0.47, 2.40))),

[0146] (1.23, (1.07, (2.21, 2.40)))]

[0147] End Segment

[0148] [(0, (5.60, (0, 2.40)), (1.42, (1.17, 2.21)), (2.83, (no information))),

[0149] (0.52, (2.15, (0, 0.47)), (1.42, (no information)), (2.83, (1.17, 2.40)),

[0150] (0.89, (1.07, (0.47, 2.40))),

[0151] (1.23, (1.42(2.212.40))).

[0152] In the actual algorithm implementation, after each box is loaded, the positions of the four affected line segment data can be clearly determined according to the position and size of the box, and the existing contents of the four positions in the two arrays can be directly accessed with the help of the height h and the lateral position p. At each position, what is actually performed is the regular intersection and complement calculation of a finite set in one-dimensional space. Therefore, the above data structure can not only accurately describe the size and position of the car and the box, but also efficiently perform operations to update the car state, providing a basis for the subsequent optimization model. At the same time, the above data structure can also serve other three-dimensional packing algorithms.

[0153] S104, obtaining the standard spatial state of the carriage in the packing termination state, discretizing it into a three-dimensional matrix and calculating the actual loading rate of the second packing problem instance in the packing termination state.

[0154] In an embodiment of the present invention, a simple discrete function is constructed to convert each state array data, that is, the standard space state, into a 0-1 matrix in three-dimensional space and input it into a neural network for training, where 0 represents that there is no box at the space point position, and 1 represents that there is a box at the space point position, and the boundary position of the box is regarded as 1.

[0155] S105, inputting the three-dimensional matrix and the actual loading rate into the neural network, so that the neural network adjusts the weight coefficients of each layer with the maximum actual loading rate as the target, until the current preset training end condition is met, and the neural network is used as a three-dimensional packing model.

[0156] S40, obtaining a packing solution with the highest actual loading rate output by the three-dimensional packing model for the first packing problem instance.

[0157] As can be seen from the above, in the embodiment of the present invention, the actual loading rate is used as the target of the three-dimensional packing optimization. For ease of understanding, the actual loading rate is described below:

[0158] Given a rectangular carriage with known length, width and height. Given a set of different rectangular boxes with known length, width, height and quantity. Put the boxes into the carriage one by one, requiring that there should be no overlap between the boxes. How can we load all the boxes with the highest actual loading rate?

[0159] Actual loading rate = total volume of the box / loaded compartment volume.

[0160] It should be noted that if N vehicles are used to load all the boxes, the loading compartment volume refers to the total volume of the first N-1 vehicles + the actual length * width * height of the Nth vehicle. Figure 6 In the box loading example shown, the actual vehicle length used is 2.15 meters.

[0161] Considering how to distinguish different solutions for the same number of vehicles, for example, when all boxes can be loaded in one carriage, distinguishing the pros and cons of different solutions, considering only the number of vehicles is an inaccurate optimization goal, so a common exact indicator is the loading rate.

[0162] In the actual production process, the box body must be loaded into the pallet first and then the pallet is loaded into the carriage, and different car models are common. This invention focuses on the state description data structure of three-dimensional packaging and the design of reinforcement learning algorithm, and does not discuss the above situation. In fact, the present invention can be simply generalized to solve the box-pallet-carriage multi-level packaging problem and the multi-carriage type packaging problem.

[0163] The present invention has the following advantages:

[0164] The data structure that describes the three-dimensional packing status simplifies the complex conditions in the three-dimensional space to the level of a set of line segments. It can not only accurately represent the packing status of any carriage-box scale problem at any time, but also quickly update the packing status.

[0165] The data structure can also be simply modified to be applicable to packing problems in various scenarios, such as multi-layer packing by palletizing first and then packing, precision material cutting, etc. It is also applicable to other problems involving three-dimensional space representation and modification.

[0166] For different actual packing problems, the overall algorithm solution can take into account both the wide range of applications and the targeted adaptation to the problems.

[0167] High computing efficiency, the number of boxes completed within 1 minute is 10 3 The packing scheme of the order of magnitude can achieve an actual loading rate of 72% and above in problem instances with different numbers of box types.

[0168] Based on the three-dimensional box processing method provided in the above embodiment, the embodiment of the present invention further provides a device for executing the three-dimensional box processing method. The structural schematic diagram of the device is shown in FIG. Figure 7 As shown, including:

[0169] An instance acquisition module 10 is used to acquire a first packing problem instance to be processed, wherein the first packing problem instance includes description data of the box and a spatial state of the compartment;

[0170] A standardization module 20, used for standardizing the description data of the box and the spatial state of the carriage in the first packing problem instance, so as to obtain the standard description data of the box and the standard spatial state of the carriage in the first packing problem instance;

[0171] The packing processing module 30 is used to input the standard description data of the box and the standard spatial state of the carriage in the first packing problem instance into the three-dimensional packing model, where the three-dimensional packing model is obtained by pre-training the neural network based on the reinforcement learning algorithm; and obtain the packing solution with the highest actual loading rate output by the three-dimensional packing model for the first packing problem instance.

[0172] Optionally, the process of the packing processing module 30 pre-training the neural network based on the reinforcement learning algorithm to obtain the three-dimensional packing model includes:

[0173] A second packing problem instance for training is obtained, wherein the second packing problem instance includes description data of the box and the spatial state of the carriage; the description data of the box and the spatial state of the carriage in the second packing problem instance are standardized to obtain standard description data of the box and the standard spatial state of the carriage in the second packing problem instance; a packing action of the second packing problem instance is determined based on a greedy strategy, and the standard description data of the box in the second packing problem instance is processed based on the packing action to update the standard spatial state of the carriage in the second packing problem instance until a preset packing termination state is entered; the standard spatial state of the carriage in the packing termination state is obtained, discretized into a three-dimensional matrix, and the actual loading rate of the second packing problem instance in the packing termination state is calculated; the three-dimensional matrix and the actual loading rate are input into a neural network, so that the neural network adjusts the weight coefficient of each layer with the maximum actual loading rate as the target, until the current preset training end condition is met, and the neural network is used as a three-dimensional packing model.

[0174] Optional, standard description data as an octet array:

[0175] (the length of the box, the width of the box, the height of the box, the coordinates of the box in the length direction of the compartment where it is located, the coordinates of the box in the width direction of the compartment where it is located, the coordinates of the box in the height direction of the compartment where it is located, the loading orientation of the box, and the number of the compartment where the box is located);

[0176] The standard space state is a two-level array:

[0177] [(height of the starting line segment, (horizontal position of the starting line segment, length of the starting line segment))], [(height of the ending line segment, (horizontal position of the ending line segment, length of the ending line segment))];

[0178] The starting line segment represents the starting position of the carriage for packing, and the ending line segment represents the ending position of the carriage for packing.

[0179] Optionally, the packing processing module 30 processes the standard description data of the box in the second packing problem instance based on the packing action to update the standard space state of the carriage in the second packing problem instance, including:

[0180] Based on the packing action, the target box to be moved and the loading position of the target box are determined; according to the loading position of the target box, the target carriage where the target box is located and the current target standard space state of the target carriage are determined; the length, width and height of the target box are used to determine the relative position relationship between the box and the starting line segment and the ending line segment in the target standard space state, and the starting line segment and the ending line segment in the target standard space state are updated based on the relative position relationship.

[0181] Optionally, the process of the box packing processing module 30 updating the start line segment and the end line segment in the target standard space state based on the relative position relationship includes:

[0182] Based on the relative position relationship, the target line segment to be updated in the target standard space state is determined, and the target line segment includes a start line segment and / or an end line segment; the intersection part of the target line segment and the target box is determined according to the length, width and height of the box in the standard description data of the target box, and the non-intersection part is supplemented to the target line segment to update the target line segment; other line segments that are not updated in the target standard space state and the updated target line segment are calculated.

[0183] It should be noted that the detailed functions of each module in the embodiment of the present invention can be found in the corresponding disclosed part of the above-mentioned three-dimensional box packing processing method, which will not be repeated here.

[0184] The three-dimensional packing processing device provided in the embodiment of the present invention can standardize the packing problem instances to be processed, provide a data structure that can accurately describe the packing state, and provide a basis for rapid iteration of subsequent optimization models, thereby providing a more efficient optimization model for the three-dimensional packing problem based on the reinforcement learning algorithm. The optimization model can automatically match the corresponding specific problems through learning, and can adapt to the packing problems of boxes-carriages of different scales.

[0185] An embodiment of the present invention further provides an electronic device, comprising: at least one memory and at least one processor; the memory stores a program, the processor calls the program stored in the memory, and the program is used to implement any one of the three-dimensional packing processing methods described.

[0186] An embodiment of the present invention further provides a storage medium, in which computer executable instructions are stored, and the computer executable instructions are used to execute any one of the three-dimensional box packing processing methods described above.

[0187] The three-dimensional packing processing method, device, electronic device and storage medium provided by the present invention are introduced in detail above. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for general technical personnel in this field, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

[0188] It should be noted that each embodiment in this specification is described in a progressive manner, and each embodiment focuses on the differences from other embodiments, and the same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part description.

[0189] It should also be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device that includes a series of elements is inherent to the elements, or also includes elements inherent to these processes, methods, articles or devices. In the absence of further restrictions, the elements defined by the sentence "including a..." do not exclude the presence of other identical elements in the process, method, article or device that includes the elements.

[0190] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A three-dimensional box processing method, characterized in that: The method comprises: Obtaining a first packing problem instance to be processed, wherein the first packing problem instance includes description data of a box and a spatial state of a carriage; Standardizing the description data of the box and the spatial state of the carriage in the first packing problem instance to obtain the standard description data of the box and the standard spatial state of the carriage in the first packing problem instance; Inputting the standard description data of the box and the standard spatial state of the carriage in the first packing problem instance into a three-dimensional packing model, wherein the three-dimensional packing model is obtained by training a neural network based on a reinforcement learning algorithm in advance; Obtaining a packing solution with the highest actual loading rate output by the three-dimensional packing model for the first packing problem instance; The process of pre-training the neural network based on the reinforcement learning algorithm to obtain the three-dimensional box packing model includes: Obtaining a second packing problem instance for training, wherein the second packing problem instance includes description data of the box and a spatial state of the compartment; Standardizing the description data of the box and the spatial state of the carriage in the second packing problem instance to obtain standard description data of the box and the standard spatial state of the carriage in the second packing problem instance; Determine a packing action of the second packing problem instance based on a greedy strategy, and process standard description data of the box in the second packing problem instance based on the packing action to update a standard space state of the carriage in the second packing problem instance until a preset packing termination state is reached; Obtain the standard spatial state of the carriage under the packing termination state, discretize it into a three-dimensional matrix and calculate the actual loading rate of the second packing problem instance under the packing termination state; The three-dimensional matrix and the actual loading rate are input into a neural network so that the neural network adjusts the weight coefficients of each layer with the maximum actual loading rate as the target until the preset training end condition is met, and the neural network is used as a three-dimensional packing model.

2. The method according to claim 1, characterized in that The standard describes data as an eight-element array: (the length of the box, the width of the box, the height of the box, the coordinates of the box in the length direction of the compartment where it is located, the coordinates of the box in the width direction of the compartment where it is located, the coordinates of the box in the height direction of the compartment where it is located, the loading orientation of the box, and the number of the compartment where the box is located); The standard space state is a secondary array: [(the height of the starting line segment, (the horizontal position of the starting line segment, the length of the starting line segment))], [(the height of the ending line segment, (the horizontal position of the ending line segment, the length of the ending line segment))]; The starting line segment represents the starting loading position of the carriage, and the ending line segment represents the ending loading position of the carriage.

3. The method according to claim 2, characterized in that The processing of the standard description data of the box in the second packing problem instance based on the packing action to update the standard space state of the carriage in the second packing problem instance includes: Determining a target box to be moved and a loading position of the target box based on the box packing action; Determine the target compartment where the target box is located and the current target standard space state of the target compartment according to the loading position of the target box; The relative position relationship between the box and the start line segment and the end line segment in the target standard space state is determined by using the length, width and height of the target box, and the start line segment and the end line segment in the target standard space state are updated based on the relative position relationship.

4. The method according to claim 3, characterized in that The updating of the start line segment and the end line segment in the target standard space state based on the relative position relationship includes: Determine a target line segment to be updated in the target standard space state based on the relative position relationship, wherein the target line segment includes a start line segment and / or an end line segment; Determine the intersection of the target line segment and the target box according to the length, width and height of the box in the standard description data of the target box, and add the non-intersection part to the target line segment to update the target line segment; The other line segments that have not been updated in the target standard space state and the updated target line segments are combined and calculated.

5. A three-dimensional packaging processing device, characterized in that: The device comprises: An instance acquisition module, used to acquire a first packing problem instance to be processed, wherein the first packing problem instance includes description data of a box body and a spatial state of a carriage; A standardization module, used for standardizing the description data of the box and the spatial state of the carriage in the first packing problem instance to obtain the standard description data of the box and the standard spatial state of the carriage in the first packing problem instance; a packing processing module, configured to input the standard description data of the box body and the standard spatial state of the carriage in the first packing problem instance into a three-dimensional packing model, wherein the three-dimensional packing model is obtained by pre-training a neural network based on a reinforcement learning algorithm; and obtain a packing solution with the highest actual loading rate output by the three-dimensional packing model for the first packing problem instance; The process of the packing processing module pre-training the neural network based on the reinforcement learning algorithm to obtain the three-dimensional packing model includes: A second packing problem instance for training is obtained, wherein the second packing problem instance includes description data of a box and a spatial state of a car body; the description data of the box and the spatial state of the car body in the second packing problem instance are standardized to obtain standard description data of the box and standard spatial state of the car body in the second packing problem instance; a packing action of the second packing problem instance is determined based on a greedy strategy, and the standard description data of the box in the second packing problem instance is processed based on the packing action to update the standard spatial state of the car body in the second packing problem instance until a preset packing termination state is entered; the standard spatial state of the car body in the packing termination state is obtained, discretized into a three-dimensional matrix, and the actual loading rate of the second packing problem instance in the packing termination state is calculated; the three-dimensional matrix and the actual loading rate are input into a neural network, so that the neural network adjusts the weight coefficients of each layer with the maximum actual loading rate as the target, until the preset training end condition is met, and the neural network is used as a three-dimensional packing model.

6. The device according to claim 5, characterized in that The standard describes data as an eight-element array: (the length of the box, the width of the box, the height of the box, the coordinates of the box in the length direction of the compartment where it is located, the coordinates of the box in the width direction of the compartment where it is located, the coordinates of the box in the height direction of the compartment where it is located, the loading orientation of the box, and the number of the compartment where the box is located); The standard space state is a secondary array: [(the height of the starting line segment, (the horizontal position of the starting line segment, the length of the starting line segment))], [(the height of the ending line segment, (the horizontal position of the ending line segment, the length of the ending line segment))]; The starting line segment represents the starting loading position of the carriage, and the ending line segment represents the ending loading position of the carriage.

7. An electronic device, characterized in that: include: at least one memory and at least one processor; The memory stores a program, and the processor calls the program stored in the memory, wherein the program is used to implement the three-dimensional box packing processing method according to any one of claims 1 to 4.

8. A storage medium, characterized in that: The storage medium stores computer executable instructions, and the computer executable instructions are used to execute the three-dimensional box packing processing method according to any one of claims 1 to 4.

Images