A Loading Optimization Method for Cylindrical Cargo
By employing a hierarchical collaborative optimization framework, combined with circular grid enumeration, multi-objective elite genetic algorithm, and deep reinforcement learning, the loading problem of cylindrical cargo was solved, achieving efficient and stable space utilization and computational efficiency, thus meeting practical loading requirements.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- HANGZHOU DIANZI UNIV
- Filing Date
- 2026-01-12
- Publication Date
- 2026-06-09
AI Technical Summary
Existing loading algorithms cannot effectively address the issues of insufficient space utilization, high stability requirements, and complex orientation of cylindrical cargo, resulting in high computational overhead and significant fluctuations in the quality and stability of the solutions, making it difficult to obtain high-quality loading schemes within an acceptable timeframe.
A hierarchical collaborative optimization framework is adopted to decompose the cylindrical cargo loading problem into optimization within the carrying unit and optimization within the transportation space. Packing optimization algorithms based on circle center grid enumeration, multi-objective elite genetic algorithm and deep reinforcement learning are respectively used to generate efficient and stable loading schemes.
It significantly improves the space utilization of cylindrical cargo, enhances global optimization capabilities, meets the flexibility and computational efficiency required for practical needs, and ensures that computational complexity is within an acceptable range.
Smart Images
Figure CN122175474A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of warehouse optimization technology, specifically relating to a loading optimization method for cylindrical goods. Background Technology
[0002] In the logistics and warehousing sector, cargo loading optimization is a core element in improving operational efficiency and reducing transportation costs. Existing research on loading algorithms and systems largely focuses on regular rectangular or cuboid cargo, with optimization objectives primarily aimed at maximizing space utilization in two-dimensional or three-dimensional space through orthogonal arrangement. These algorithms typically utilize strategies such as residual rectangle segmentation and heuristic search, and have already achieved relatively mature applications.
[0003] However, applying optimization algorithms suitable for rectangular goods directly to cylindrical goods presents numerous limitations. First, the geometry of cylindrical goods prevents them from being stacked as seamlessly as rectangular goods, inevitably creating unusable curved gaps between adjacent items, resulting in inherently insufficient space utilization. Second, the stability requirements (placement and rolling) of cylinders within containers are far higher than those for cuboids, necessitating additional consideration of physical constraints in their arrangement; simple orthogonal models cannot meet the safety requirements of actual loading. Furthermore, cylindrical goods (such as paper rolls and barrelled liquids) often involve different orientations, such as upright or horizontal placement, further increasing the complexity of the layout.
[0004] Currently, there is a relative lack of efficient loading optimization techniques specifically designed for cylindrical cargo. Existing solutions employ a framework of "global optimization with jump-restart + greedy improvement based on subset swaps." While this approach possesses some optimization capability, it often faces two limitations under complex constraints and multi-objective demands: first, the solution process is highly sensitive to the initial solution, resulting in significant fluctuations in solution quality and stability; second, pursuing higher loading rates typically requires frequent subset swap evaluations and multiple iterations, leading to substantial computational overhead and making it difficult to consistently obtain high-quality loading solutions within an acceptable timeframe. Therefore, there is an urgent need in this field for an intelligent optimization method that can simultaneously consider loading rate, stability, and solution efficiency to fill the gap in existing technologies in this specialized area. Summary of the Invention
[0005] To address the shortcomings of existing technologies, this invention provides a loading optimization method for cylindrical cargo. The core of this invention lies in proposing a hierarchical collaborative optimization framework, which decomposes the complex cylindrical cargo loading problem into three levels, and adopts the most suitable optimization algorithm for the characteristics of each level, and finally outputs a global optimization solution through a collaborative mechanism.
[0006] In a first aspect, embodiments of this application provide a loading optimization method for cylindrical cargo, the steps of which include:
[0007] Step 1, Optimization within the carrying unit: For cylindrical goods that need to be loaded into a single carrying unit (such as a pallet), different strategies are used to optimize the layout based on whether the specifications are uniform.
[0008] If the goods are of the same specifications, a precise search algorithm based on the enumeration of the center grid is used to obtain the layout scheme; if the goods are of mixed specifications, a layout scheme based on the multi-objective elite genetic algorithm is used.
[0009] Step 2, Optimization within the transportation space: Treat each loaded carrier unit in the obtained layout scheme as a whole, and use a packing optimization algorithm based on deep reinforcement learning to further optimize its placement scheme in the entire transportation space (such as a container).
[0010] Step 3, Final Solution Generation and Output: Integrate the optimization results of Step 1 and Step 2, complete the parameterized representation of the loading solution, output the final loading position and orientation of each cylindrical cargo, and provide detailed information on the loading solution in the form of structured data.
[0011] Secondly, embodiments of this application provide a loading optimization system for cylindrical cargo, comprising the following modules:
[0012] Optimization module within the carrying unit: For cylindrical goods that need to be loaded into a single carrying unit (such as a pallet), different strategies are used to optimize the layout based on whether the specifications are uniform.
[0013] If the goods are of the same specifications, a precise search algorithm based on the enumeration of the center grid is used to obtain the layout scheme; if the goods are of mixed specifications, a layout scheme based on the multi-objective elite genetic algorithm is used.
[0014] The optimization module within the transportation space treats each loaded carrier unit in the obtained layout scheme as a whole and uses a deep reinforcement learning-based packing optimization algorithm to further optimize its placement scheme within the entire transportation space (such as a container).
[0015] Final solution generation and output module: Integrates the optimization results of the optimization module within the carrying unit and the optimization module within the transportation space, completes the parameterized representation of the loading solution, outputs the final loading position and orientation of each cylindrical cargo, and provides detailed information on the loading solution in the form of structured data.
[0016] The beneficial effects of this invention are as follows:
[0017] 1. High space utilization: Through precise geometric models (such as staggered placement) and advanced intelligent optimization algorithms (such as multi-objective elite genetic algorithms), the ineffective gaps between cylindrical goods are significantly reduced.
[0018] 2. Strong global optimization capability: The hierarchical strategy modularizes complex problems, and the bin packing optimization algorithm based on deep reinforcement learning ensures coordinated optimization from local units to the global space, avoiding local optima.
[0019] 3. Highly practical and flexible: With built-in stability constraints, the algorithm can flexibly adjust the optimization objective through the evaluation function according to actual needs (such as prioritizing filling or prioritizing averaging).
[0020] 4. High computational efficiency: The algorithm selection for different scenarios (exact enumeration, heuristic search) effectively controls the computational complexity while ensuring the quality of the solution, meeting the time requirements of practical applications. Attached Figure Description
[0021] To make the objectives, technical solutions, and advantages of the present invention clearer, some exemplary embodiments of the present invention will be described below in conjunction with the accompanying drawings. It should be understood that these drawings and corresponding descriptions are exemplary and explanatory in nature, and are not intended to limit the scope of the present invention. Based on the drawings and spirit shown herein, those skilled in the art can make reasonable deductions to obtain other feasible implementation methods.
[0022] Figure 1 This is a schematic diagram of the method flow according to an embodiment of the present invention.
[0023] Figure 2 This is a schematic diagram of the precise search algorithm based on the enumeration of the center grid in an embodiment of the present invention.
[0024] Figure 3 This is a flowchart of the multi-objective elite genetic algorithm according to an embodiment of the present invention.
[0025] Figure 4 This is a flowchart of the bin packing optimization algorithm based on deep reinforcement learning, according to an embodiment of the present invention. Detailed Implementation
[0026] The following detailed description of some embodiments of the present invention, in conjunction with the accompanying drawings and algorithm flow, will illustrate these embodiments in detail. Those skilled in the art will understand that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit its scope.
[0027] like Figure 1 As shown in the figure, this application provides a loading optimization method for cylindrical cargo, including optimization within the carrying unit, optimization within the transportation space, and final solution generation and output.
[0028] Optimization within a carrying unit: For cylindrical goods that need to be loaded into a single carrying unit (such as a pallet), different strategies are used for layout optimization depending on whether their specifications are uniform.
[0029] Optimization within the transportation space: Each loaded carrier unit in the obtained layout scheme is regarded as a whole, and its placement scheme in the entire transportation space (such as a container) is further optimized using a packing optimization algorithm based on deep reinforcement learning.
[0030] Final scheme generation and output: The layout scheme obtained by layout optimization is input into the trained deep reinforcement learning sequential generation network. Based on the output bin packing order and the heuristic bin packing execution result, the structured output data of the final loading scheme is generated.
[0031] The structured output data includes the final coordinates of each carrier unit within the transportation space, the specific location of each item on the carrier unit, and the corresponding loading list and statistical indicators. This structured output data can be used for subsequent visualization, loading guidance, or record management.
[0032] In one possible implementation, for optimizing cylindrical goods of the same specifications within a carrying unit, a precise search algorithm based on the enumeration of the center grid is used to obtain the layout scheme.
[0033] The precise search algorithm based on circular grid enumeration is suitable for cylindrical goods of a single upright size (horizontal placement or a mixture of upright and horizontal placement is often not used for stability reasons). Its goal is to find a layout that maximizes the number of cylindrical goods placed on a given load-bearing unit base plate. The specific steps are as follows:
[0034] 1-1 Input parameters: Length L of the bearing unit, width W, radius r of the bottom surface of the cylinder.
[0035] 1-2 Layout Pattern Definition: The core is to calculate the staggered placement pattern. Let N be the number of goods in two rows along the length direction (N is an integer greater than or equal to 1).
[0036] 1-3 Geometric Modeling: Connect the centers of two adjacent cylinders in adjacent rows as the hypotenuse, and combine this with the right-angled sides in the length and width directions of the supporting unit to form a virtual right triangle. The hypotenuse of this triangle is 2r long, and the right-angled side in the length direction is d. The distance between the first and last centers of the N goods (i.e., the two centers closest to the outside in the length direction) is L-2r, which can also be regarded as N-1 right-angled sides, so the right-angled side d=(L-2r) / (N-1). According to the Pythagorean theorem, the side length of this right triangle in the width direction of the supporting unit is h=sqrt((2r)^2-d^2). Calculate the number of goods M in the width direction. In the width W, the first goods need 2r space, and each additional goods need h space. Therefore, the maximum number of goods that can be accommodated in the width direction is M=floor((W-2r) / h)+1.
[0037] 1-4 Enumeration and Comparison: Iterate through all possible rows of goods in the long direction (the number of goods in the two rows of the carrying unit in the wide direction is the number of rows of goods in the long direction) (N+1)r <= L <= 2Nr, and calculate the total loading number P corresponding to each value of N. If N is even, then P = N*M / 2; otherwise, P = (N-1)*M / 2 + floor((M+1) / 2). At the same time, iterate through the possible rows in the wide direction (the number of goods in the two rows of the carrying unit in the long direction is the number of rows of goods in the wide direction) (M+1)r <= W <= 2Mr. If M is even, then P = M*N / 2; otherwise, P = (M-1)*N / 2 + floor((N+1) / 2). Finally, select the layout scheme that maximizes P.
[0038] If only the maximum load capacity is considered, the resulting layout often represents an asymmetric extreme state, posing stability and operational risks. Therefore, this algorithm abandons the strategy of simply pursuing theoretical extrema, instead prioritizing the symmetry and boundary fit of the layout. By using this as a reference optimization objective, it ensures that the generated scheme achieves the optimal balance in terms of space utilization, structural stability, and operational convenience in an engineering-practical sense.
[0039] In one possible implementation, a layout scheme based on a multi-objective elite genetic algorithm is used to optimize the arrangement of cylindrical goods of different specifications within the carrying unit.
[0040] This paper addresses the problem of mixed loading layout of various cylindrical goods of different specifications on the bottom surface of several load-bearing units. Unlike the geometric enumeration of goods of the same specification in the first embodiment, this embodiment needs to simultaneously consider multiple objectives such as space utilization, center of gravity balance and boundary fit, while ensuring geometric non-overlap, center of gravity stability and good operability. Therefore, a multi-objective elite genetic algorithm is used for optimization.
[0041] 2-1 Overall Algorithm Idea
[0042] The bottom surface of the carrying unit is considered a two-dimensional placement plane. The set of cylindrical goods of different specifications to be loaded is defined as a group of goods numbered from 1 to n. The algorithm adopts a genetic encoding method of "permutation encoding + rule decoding": each chromosome is represented by an integer sequence, written as S=[s1, s2, …sk, …, sn], where sk represents the number of the k-th placed goods. The sequence S is generated by the genetic algorithm. The initial sequence is generated by diameter sorting and perturbation, and is subsequently updated iteratively through selection, crossover, and mutation. During decoding, goods are processed sequentially according to the order in the sequence S: priority is given to trying to place the current goods into an already opened carrying unit; if all opened carrying units cannot provide a feasible placement position that satisfies "no boundary crossing and no overlap", then a comparison and selection is made between the "forced placement scheme based on distance-type penalty" and the "new carrying unit placement scheme" to minimize the number of carrying units used.
[0043] The sequence S decoding process is as follows: For each item to be placed, the algorithm generates several candidate positions within the selected carrying unit based on the center coordinates of the currently placed item and the geometric boundary of the carrying unit. These positions are either close to the geometric boundary of the carrying unit or tangent to the bottom surface of the placed item. Among all candidate positions, the algorithm first selects from the set of candidate positions that satisfy the constraints of "no boundary crossing and no overlap," meaning that the bottom surface of the item does not exceed the geometric boundary of the carrying unit and does not geometrically overlap with the placed item. The algorithm prioritizes selecting the candidate position that minimizes the overall center of gravity offset of the carrying unit. When multiple candidate positions have the same or different center of gravity offsets below a preset threshold, the algorithm then selects the candidate position with the largest boundary fit gain. If no candidate position satisfies the constraints, the candidate position with the smallest distance penalty is selected from all candidates. The distance penalty is a weighted sum of the boundary depth and the overlap penetration depth: the boundary depth represents the maximum radial distance that the bottom circle of the cargo exceeds the geometric boundary of the bearing unit; the overlap penetration depth represents the difference when the center distance between the cargo to be placed and any already placed cargo is less than the sum of the radii of the two cargoes (i.e., the degree of penetration of the two circles along the line connecting their centers). The center distance is zero when it is greater than or equal to the sum of the radii of the two cargoes. If the cargo overlaps with multiple already placed cargoes at the same time, the overlap penetration depth is the sum of the overlap penetration amounts.
[0044] After fully decoding the sequence S using the above decoding rules, a layout solution L(S) can be obtained. This layout solution includes the number of carrier units used, the number and radius of the goods successfully placed in each carrier unit, and the corresponding set of center coordinates, as well as the correspondence between the goods and the carrier units.
[0045] To achieve multi-objective optimization, this implementation calculates three main objective metrics for each layout solution:
[0046] Space utilization rate: It is measured by the ratio of the sum of the base areas of all successfully placed cylinders to the sum of the base areas of the load-bearing units used. The higher the value, the more compact the loading.
[0047] Center of gravity balance: Calculate the distance between the overall center of gravity of the layout and the geometric center of the bearing unit for each bearing unit, and summarize and evaluate the distances of the bearing units used (the summarization can be done by methods including but not limited to taking the maximum value or the average value). The smaller the distance, the more uniform the force on the bearing unit.
[0048] Boundary fit: The ratio of the total cylindrical arc length of the contact bearing unit boundary to the perimeter of the bearing unit used. The larger the ratio, the more fully the boundary is utilized.
[0049] For layouts that violate constraints such as local overlap or boundary crossing, the extreme target index value is directly assigned, making such infeasible solutions significantly inferior in the evaluation of the three main target indices, thereby guiding the population iteration to converge toward the feasible region.
[0050] During population evolution, a multi-objective elite mechanism similar to NSGA-II is employed. Specifically:
[0051] First, the entire population is non-dominated and sorted according to three target indicators, and individuals are divided into several Pareto front layers.
[0052] Individuals in the front layer have higher priority in retention and selection. For individuals within the same front layer, to maintain solution diversity, this implementation calculates the crowding distance of each individual to measure its sparsity in the target space. Specifically, based on each target index, individuals in the current front layer are sorted according to their corresponding target values. The crowding distance of individuals located at the boundary of that target (i.e., those whose values on that target index reach the minimum or maximum value within the current front layer) is set to a maximum value to ensure that solutions at the front boundary are preferentially retained during subsequent truncation and screening. For the remaining individuals, their adjacent differences on each target index are calculated and normalized, i.e., by dividing "the target value of the next individual minus the target value of the previous individual" by the target value span of the current target index within the current front layer. Finally, the normalized differences on each target index are summed to obtain the total crowding distance of the current individual. A larger crowding distance indicates that the solutions around that individual are more sparse and representative. When truncation is required (in the process of constructing the next generation population or elite set, when individuals are added sequentially according to the priority of the front layer, the addition of individuals to a certain front layer will cause the total number of individuals to exceed the preset population size limit. At this time, it is necessary to select some individuals in the front layer to add to meet the size limit), individuals with larger crowding distances are prioritized for retention: sort the individuals in the current front layer from largest to smallest crowding distance, and select them sequentially until the required number of individuals is reached, to prevent the solution set from being concentrated in a certain local area too early.
[0053] Within the aforementioned permutation coding framework, selection, crossover, and mutation operations are performed on the chromosome sequence S to generate new individuals. Tournament selection is preferred as the selection operator, partial mapping crossover (PMX) and sequential crossover (OX) suitable for permutation coding are used as the crossover operator, and exchange mutation (randomly swapping two items) and insertion mutation (inserting one item into another position) are used as mutation operators, always maintaining a valid permutation of chromosomes with "no duplicate numbers and no missing numbers."
[0054] Through the aforementioned encoding and decoding framework of "permutation encoding + rule decoding", and the multi-objective elite genetic optimization process based on non-dominated sorting and crowding distance, this implementation method obtains a set of high-quality mixed loading layout schemes that correspond to the three objective indicators and are mutually non-dominated, through multiple generations of iterative evolution while ensuring global search capability. When the termination condition is met, the top several Pareto front solutions in the final population can be output as a set of candidate schemes, which is convenient for selection based on different preferences in actual engineering.
[0055] 2-2 Optimize Process Steps
[0056] The hybrid loading optimization process based on a multi-objective elite genetic algorithm includes the following steps (the step numbers are for illustrative purposes only and do not limit the specific order of the invention):
[0057] Data initialization: Based on the list of cylindrical goods to be loaded, obtain the diameter, height and quantity information of each type of goods; read the length, width and height of the carrying unit, and establish the coordinate system of the two-dimensional placement area of the carrying unit.
[0058] Population Initialization and Decoding: Several initial sequences are generated based on the cargo diameter sorting from largest to smallest and a random perturbation strategy, serving as the initial population. For each chromosome in the population, decoding is performed sequentially according to the sequence: First, the first carrying unit is activated; for the current cargo, candidate positions are generated sequentially within each activated carrying unit and placement is attempted. If a feasible candidate position exists that satisfies the constraints of "no boundary crossing and no overlap," the position is selected and placed according to priority rules. When multiple carrying units have feasible candidate positions, the first activated carrying unit is selected for placement. The priority rules are as follows: priority is given to selecting the candidate position that minimizes the overall center-of-gravity offset distance of the carrying unit; when multiple candidate positions have the same or different center-of-gravity offsets below a preset threshold, the candidate position with the largest boundary fit gain is selected from among them. If no feasible candidate locations exist for any of the already activated carrier units, the forced placement scheme with the minimum distance penalty is selected from all candidate locations and compared with the cost of "placing a new carrier unit" (where the cost of "placing a new carrier unit" is a preset new creation cost threshold P_open, used to represent the cost of adding a new carrier unit and compared on the same scale as the distance penalty, thereby controlling the number of carrier units and guiding the decoding process to prioritize the reuse of already activated carrier units): if the distance penalty of the forced placement scheme is less than the new creation cost threshold P_open, the forced placement scheme is selected; otherwise, a new carrier unit is created and placed within the new carrier unit according to priority rules. After all decoding is completed, the multi-carrier unit layout scheme corresponding to the chromosome and the set of coordinates of the cylinder centers within each carrier unit are obtained, and the correspondence between goods and carrier units is obtained.
[0059] Multi-objective fitness assessment: For each layout scheme, space utilization, center-of-gravity balance, and boundary fit are calculated. If a layout does not meet the "no boundary crossing, no overlap" constraints, the above target indicators are corrected by a penalty coefficient to make the evaluation results worse. Specifically, the penalty coefficient pv = 1 + λ*v (λ is a preset coefficient, v is a distance-based penalty) is used to make consistent corrections to all target indicator values: space utilization' = space utilization / pv, boundary fit' = boundary fit / pv, center-of-gravity balance' = center-of-gravity balance*pv. Then, a multi-objective non-dominated ranking is performed on all individuals (i.e., the layout schemes decoded from each chromosome in the population), dividing the individuals into several Pareto front layers, and calculating the crowding distance within each front layer.
[0060] Elite Preservation and Genetic Operations: Individuals are selected from the current population according to the Pareto front layer priority as elite individuals until the preset elite number limit is reached. If the number of elite individuals exceeds the limit when adding individuals to a front layer, individuals within that front layer are selected and added according to the crowding distance from largest to smallest. The remaining individuals are selected through a tournament to enter the mating pool, where crossover and mutation operations are performed to generate new offspring chromosomes. After merging elite individuals with offspring individuals, the front layer is obtained by non-dominance sorting (the sorting method is the same as described above, but the target is the merged set of "elite + offspring", used to construct the next generation), and is filled to the preset population size according to the front layer priority. When filling to a front layer would exceed the population size, individuals within that front layer are sorted and truncated according to the crowding distance from largest to smallest, forming a new population.
[0061] Termination and Solution Output: Determine if the preset termination conditions are met, such as reaching the upper limit of the evolutionary generation or the Pareto front (the result after each generation completes "elite preservation and genetic operation") showing no significant change over multiple generations. If not, proceed to the next generation loop, repeating the process of "multi-objective fitness evaluation" and "elite preservation and genetic operation" to continue iterating. Upon termination, perform non-dominated sorting on the final population, outputting the top several front layers as the final Pareto front solution set, and select one or more layout schemes according to specific business needs (e.g., when space utilization is more important, select the solution with the highest space utilization as the layout scheme; when stability is more important, select the solution with the highest space utilization among solutions whose center of gravity offset does not exceed a preset threshold as the layout scheme; or after normalizing the three objectives, select the solution with the highest score by weighting the scores according to preset weights as the layout scheme). This serves as the final loading layout result for cylindrical goods of different specifications within the carrying unit, for subsequent layout optimization at the transportation space level.
[0062] In one possible implementation, the optimization within the transport space is specifically achieved as follows:
[0063] This implementation solves the overall layout optimization problem of "loading carrier units into transportation space". Multiple carrier units are treated as items in a one-dimensional packing problem, and the transportation space is considered as boxes with fixed capacity. A deep reinforcement learning-based packing optimization algorithm is used. A sequence generation network directly outputs a suitable packing order, and heuristic algorithms (such as Best-Fit and First-Fit) are used to execute the specific packing. This minimizes the amount of transportation space and improves space utilization while ensuring constraints are met. (Carrier unit: can be a pallet, shelf module, or a pre-loaded unit module; Transportation space: can be a container, truck compartment, or logically divided loading area in a warehouse; In the length direction, the transportation space approximates a one-dimensional packing problem, with each carrier unit occupying a certain length, and the available length of the transportation space having a fixed capacity.)
[0064] 3-1 Problem Definition and Core Functions
[0065] The problem of loading carrier units into the transport space is abstracted as a single-stage one-dimensional packing problem. The input consists of carrier units of different sizes in the layout scheme. The length of each carrier unit is represented by an array w[1], w[2], ... w[i], ..., w[n], where w[i] represents the length occupied by the i-th carrier unit in the transport space. All transport spaces (boxes) have the same capacity, which is represented by a length value C, referring to the available length of the container or carriage.
[0066] The bin packing optimization algorithm employs a deep reinforcement learning order generation network and a heuristic bin packing execution module. The deep reinforcement learning order generation network outputs a bin packing order sequence `order=[o1, o2, ...oi, ..., on]`, where each `oi` is the number of a carrier unit. This bin packing order sequence is then executed by a heuristic algorithm (such as Best-Fit or First-Fit): the carrier units are sequentially packed into one or more transport spaces according to the `order` sequence, minimizing the number of transport spaces while satisfying capacity constraints and maximizing the average occupancy rate of each transport space, thereby reducing space waste. During the training phase, the average occupancy rate after heuristic bin packing execution is used as a reward signal, enabling the order generation network to provide a better loading order in different task instances.
[0067] 3-2 Technical Architecture and Model Structure
[0068] A deep reinforcement learning sequence generation network using the Actor-Critic architecture.
[0069] Actor: Employs a pointer network responsible for outputting the packing order based on the input carrier unit size sequence. The pointer network natively supports variable-length inputs, enabling it to handle varying numbers of carrier units.
[0070] Critic: A convolutional neural network (CNN) is used to encode the sequence of bearer unit sizes corresponding to the current task instance. Local patterns are extracted from it through position-wise convolution operations to generate a fixed-length feature vector sequence {f1,f2,...,fn} arranged according to the sequence position, where fi represents the feature vector of the i-th bearer unit after CNN encoding. Finally, the feature vector sequence is fed into a fully connected layer to output the value estimate V.
[0071] To further optimize the model's learning performance, the Critic network introduces a Self-Attention mechanism after obtaining the feature vector sequence output. Self-Attention uses the feature vector sequence {f1, f2, ..., fn} as the attention input, calculates the attention weight ai for each position's feature vector fi, and then weights and aggregates the feature vectors at each position in the sequence to obtain the global feature representation f_att. f_att is then fed into a fully connected layer to output the value estimate V. Through this mechanism, the model can adaptively increase its attention to the load-bearing units and their combination relationships that have a greater impact on the packing effect, thereby improving learning performance and loading efficiency.
[0072] 3-3 Training and Reward Design
[0073] Building the training dataset:
[0074] When training the deep reinforcement learning sequential generation network, sets of carrier units with different numbers and sizes are randomly generated, covering various scales and scenarios. The number of carrier units can take different ranges (e.g., 20–30, 50–60, around 100, etc.), and the size distribution can be set to uniform distribution or concentrated distribution within a certain size range (reflecting the situation of multiple specifications but biased towards a certain size), thereby verifying the applicability and generalization ability of the model under different scales and distribution conditions. The pointer network structure can naturally handle changes in input length, so the model can adapt to situations where the number of carrier units is not fixed in real tasks. In the data generation phase, the randomly generated bin packing instances are divided into training task sets and validation task sets according to a preset ratio. The training task set is used for parameter updates, while the validation task set does not participate in parameter updates and is only used to evaluate the model's performance and convergence on unseen instances.
[0075] Training configuration:
[0076] Training configuration parameters include, but are not limited to: the number of carrier units, the range of length values, and the transportation space capacity. The parameters are adjusted according to different vehicle models, container specifications, and business needs to train a strategy model that is suitable for the corresponding scenario.
[0077] Deep reinforcement learning training:
[0078] The training process employs a combination of "deep reinforcement learning output order + heuristic bin packing execution": the Actor network of the deep reinforcement learning model is responsible for generating the bin packing order, while the Critic network outputs value estimates to guide policy updates. During training, the bin packing order generated by the Actor is executed by a heuristic bin packing algorithm, which includes, but is not limited to, Best-Fit (BF) or First-Fit (FF).
[0079] Reward Design:
[0080] Assuming a packing operation utilizes B shipping spaces, the occupancy rate of each space is calculated: the sum of the lengths of all carrying units within that space divided by its capacity C. The occupancy rates of all shipping spaces are averaged to obtain a value between 0 and 1, which is taken as the average occupancy rate. In the reward design, this average occupancy rate is used as the reward value; the higher the average occupancy rate, the larger the reward.
[0081] Convergence criteria:
[0082] During training, the average occupancy rate on the validation set is continuously monitored. The model is considered converged when the average occupancy rate no longer significantly increases within a certain number of rounds, or when it reaches a preset threshold. After convergence, the model parameters for Actor and Critic are fixed.
[0083] By using this single-stage deep reinforcement learning bin packing algorithm, this implementation method retains the advantages of traditional heuristic algorithms in terms of efficiency and ease of implementation, while adding the ability to automatically learn the bin packing order optimization, which significantly improves the average occupancy rate of transportation space, reduces the amount of transportation space required, and improves the overall transportation efficiency.
[0084] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.
Claims
1. A loading optimization method for cylindrical cargo, characterized by the following steps: include: Step 1, Optimization within the load-bearing unit: For cylindrical goods that need to be loaded into a single load-bearing unit, different strategies are adopted for layout optimization based on whether their specifications are uniform. If the goods are of the same specifications, a precise search algorithm based on the enumeration of the center grid is used to obtain the layout scheme; If it is a mixed specification, a layout scheme is obtained by using a multi-objective elite genetic algorithm; Step 2, Optimization within the transportation space: Treat each loaded carrier unit in the obtained layout scheme as a whole, and use a packing optimization algorithm based on deep reinforcement learning to further optimize its placement scheme in the entire transportation space; Step 3, Final Solution Generation and Output: Integrate the optimization results of Step 1 and Step 2, complete the parameterized representation of the loading solution, output the final loading position and orientation of each cylindrical cargo, and provide detailed information on the loading solution in the form of structured data.
2. The loading optimization method for cylindrical cargo according to claim 1, characterized in that, The precise search algorithm based on circle center grid enumeration is implemented as follows: 1-1 Input parameters: Length L of the bearing unit, width W, radius r of the cylinder base; 1-2 Layout Pattern Definition: The core is to calculate the staggered placement pattern; let N be the number of goods in two rows along the length direction; 1-3 Geometric Modeling: Connect the centers of two adjacent cylinders in adjacent rows as the hypotenuse, and combine them with the right-angled sides in the length and width directions of the supporting unit to form a virtual right triangle; the hypotenuse of this triangle is 2r, and let the right-angled side in the length direction be d; the distance between the first and last centers of the N goods is L-2r, then the right-angled side d=(L-2r) / (N-1); according to the Pythagorean theorem, the side length of this right triangle in the width direction of the supporting unit is h=sqrt((2r)^2-d^2); the maximum number of goods that can be accommodated in the width direction is M=floor((W-2r) / h)+1; 1-4 Enumeration and Comparison: Traverse all possible rows of goods in the long direction (N+1)r<=L<=2Nr, calculate the total loading number P corresponding to each N value; if N is even, then P=N*M / 2; otherwise P=(N-1)*M / 2+floor((M+1) / 2); at the same time, traverse all possible rows in the wide direction (M+1)r<=W<=2Mr, if M is even, then P=M*N / 2; otherwise P=(M-1)*N / 2+floor((N+1) / 2); finally select the layout scheme that maximizes P.
3. The loading optimization method for cylindrical cargo according to claim 1, characterized in that, The specific implementation of the multi-objective elite genetic algorithm is as follows: The bottom surface of the carrying unit is considered as a two-dimensional placement plane; the set of cylindrical goods of different specifications to be loaded is defined as a group of goods numbered from 1 to n; the algorithm adopts a genetic coding method of "permutation encoding + rule decoding": each chromosome is represented by an integer sequence, written as S=[s1, s2, …sk, …, sn], where sk represents the number of the kth placed goods; the sequence S is generated by the genetic algorithm, the initial sequence is generated by diameter sorting and perturbation, and subsequently updated iteratively through selection, crossover and mutation; Through multiple generations of iterative evolution, a set of high-quality hybrid loading layout schemes that correspond to the target indicators and are not mutually dominant were obtained; During decoding, goods are processed sequentially according to the order in sequence S: priority is given to placing the current goods into an already opened carrying unit; if none of the opened carrying units can provide a feasible placement location that satisfies the requirements of "no boundary crossing and no overlap", then a comparison and selection is made between the "forced placement scheme based on distance-type penalty" and the "placement scheme of newly opened carrying unit".
4. The loading optimization method for cylindrical cargo according to claim 3, characterized in that, The sequence S decoding process is as follows: For each item to be placed, the algorithm generates several candidate positions within the selected carrying unit based on the center coordinates of the currently placed item and the geometric boundary of the carrying unit. These positions are either close to the geometric boundary of the carrying unit or tangent to the bottom surface of the placed item. Among all candidate positions, the algorithm first selects from the set of candidate positions that satisfy the "no boundary crossing, no overlap" constraint, whereby the bottom surface of the item does not exceed the geometric boundary of the carrying unit and does not geometrically overlap with the placed item. The algorithm prioritizes selecting the candidate position that minimizes the overall center of gravity offset of the carrying unit. When multiple center of gravity offsets are the same or differ from the preset value, the algorithm proceeds with further selection. When considering candidate positions for the threshold, the candidate position with the largest boundary fit gain is selected from among them. If no candidate position satisfies the constraint, the candidate position with the smallest distance penalty is selected from all candidates. The distance penalty is composed of a weighted sum of the boundary crossing depth and the overlap penetration depth: the boundary crossing depth represents the maximum radial distance that the bottom surface of the goods exceeds the geometric boundary of the bearing unit; the overlap penetration depth represents the difference when the center distance between the goods to be placed and any already placed goods is less than the sum of the radii of the two goods, and is zero when the center distance is greater than or equal to the sum of the radii of the two goods. If the goods overlap with multiple already placed goods at the same time, the overlap penetration depth is the sum of the overlap penetration amounts. After fully decoding the sequence S using the above decoding rules, a layout solution L(S) is obtained. This layout solution includes the number of carrier units used, the number and radius of the goods successfully placed in each carrier unit, and the corresponding set of center coordinates, as well as the correspondence between the goods and the carrier units.
5. The loading optimization method for cylindrical cargo according to claim 4, characterized in that, To achieve multi-objective optimization, three main objective metrics are calculated for each layout solution: Space utilization rate: It is measured by the ratio of the sum of the base areas of all successfully placed cylinders to the sum of the base areas of the load-bearing units used. The higher the value, the more compact the loading. Center of gravity balance: Calculate the distance between the overall center of gravity of the layout and the geometric center of the bearing unit for each bearing unit, and summarize and evaluate the distance of the bearing units used. The smaller the distance, the more uniform the force on the bearing unit. Boundary fit: The ratio of the total cylindrical arc length of the contact bearing unit boundary to the perimeter of the bearing unit used. The larger the ratio, the more fully the boundary is utilized. For layouts that violate constraints, a target index value of extreme error is directly assigned, making such infeasible solutions significantly inferior in the evaluation of the three main target indices, thereby guiding the population iteration to converge toward the feasible region.
6. The loading optimization method for cylindrical cargo according to claim 3, characterized in that, The hybrid loading optimization process based on a multi-objective elite genetic algorithm includes the following steps: Data initialization: Based on the list of cylindrical goods to be loaded, obtain the diameter, height, and quantity information of each type of goods; read the length, width, and height of the carrying unit, and establish the coordinate system of the two-dimensional placement area of the carrying unit; Population Initialization and Decoding: Several initial sequences are generated based on the cargo diameter sorting from largest to smallest and a random perturbation strategy, serving as the initial population. For each chromosome in the population, decoding is performed sequentially according to the sequence order: First, the first carrying unit is activated. For the current cargo, candidate positions are generated sequentially within each activated carrying unit and placement is attempted. If a feasible candidate position exists that satisfies the constraints of "no boundary crossing and no overlap," the position is selected and placed according to the priority rule. When multiple carrying units have feasible candidate positions, the first activated carrying unit is selected for placement. The priority rule specifically states: the candidate position that minimizes the overall center of gravity offset distance of the carrying unit is selected first; when multiple center of gravity offset distances exist... When moving candidate positions that are identical or have a difference below a preset threshold, the candidate position with the largest boundary fit gain is selected from among them. If no feasible candidate positions exist for all opened carrier units, the forced placement scheme with the smallest distance penalty is selected from all candidate positions and compared with the cost of "placing a new carrier unit": if the distance penalty of the forced placement scheme is less than the new cost threshold P_open, the forced placement scheme is selected; otherwise, a new carrier unit is opened and placed according to the priority rules in the new carrier unit. After all decoding is completed, the multi-carrier unit layout scheme corresponding to the chromosome and the set of cylinder center coordinates in each carrier unit are obtained, and the correspondence between goods and carrier units is obtained. Multi-objective fitness assessment: For each layout scheme, space utilization, center of gravity balance, and boundary fit are calculated. If the layout does not meet the "no boundary crossing, no overlap" constraints, the above target indicators are corrected by a penalty coefficient to make the evaluation results worse. Specifically, the penalty coefficient pv = 1 + λ*v, where λ is a preset coefficient and v is a distance-type penalty, is used to make consistency corrections for all target indicator values. Then, a multi-objective non-dominated ranking is performed on all individuals, dividing them into several Pareto front layers, and the crowding distance is calculated within each front layer. Elite Preservation and Genetic Operations: Individuals are selected from the current population according to the Pareto front layer priority as elite individuals until the preset elite number limit is reached. If the elite number limit is exceeded when adding individuals to a front layer, individuals within that front layer are selected according to the crowding distance from largest to smallest. The remaining individuals are selected through a tournament to enter the mating pool, where crossover and mutation operations are performed to generate new offspring chromosomes. After merging elite individuals with offspring individuals, the front layer is obtained by sorting according to non-dominance, and then filled to the preset population size according to the front layer priority. When filling a front layer would exceed the population size, individuals within that front layer are sorted according to the crowding distance from largest to smallest and truncated to form a new population. Termination and Solution Output: Determine whether the preset termination conditions have been met; if not, enter the next generation loop, that is, repeat the process of "multi-objective fitness evaluation" and "elite preservation and genetic operation" to continue iterating; when terminating, perform non-dominated sorting on the final population, output the top several front layers as the final Pareto front solution set, and select the layout scheme according to specific business needs.
7. The loading optimization method for cylindrical cargo according to claim 1, characterized in that, The bin packing optimization algorithm based on deep reinforcement learning is as follows: The problem of loading carrier units into the transportation space is abstracted into a single-stage one-dimensional packing problem. The input is carrier units of different sizes in the layout scheme. The length of each carrier unit is represented by the array w[1], w[2], ...w[i], ..., w[n], where w[i] represents the length occupied by the i-th carrier unit in the transportation space. All transportation spaces (boxes) have the same capacity, which is represented by a length value C, referring to the available length of the container or carriage. The bin packing optimization algorithm employs a deep reinforcement learning sequence generation network and a heuristic bin packing execution module. The deep reinforcement learning sequence generation network outputs a bin packing order sequence order=[o1, o2, ...oi, ..., on], where each oi is the number of the carrier unit. The bin packing order sequence is then executed by the heuristic algorithm: the carrier units are sequentially packed into one or more transport spaces according to the bin packing order sequence order. During the training phase, the average occupancy rate after the heuristic bin packing execution is used as the reward signal.
8. The loading optimization method for cylindrical cargo according to claim 7, characterized in that, A deep reinforcement learning sequential generation network employing an Actor-Critic architecture; Actor: Employs a pointer network to output the packing order based on the input sequence of carrier unit sizes; Critic: A convolutional neural network (CNN) is used to encode the sequence of bearer unit sizes corresponding to the current task instance. Local patterns are extracted from the sequence through position-wise convolution operations to generate a fixed-length feature vector sequence {f1,f2,...,fn} arranged according to the sequence position, where fi represents the feature vector of the i-th bearer unit after CNN encoding. Finally, the feature vector sequence is fed into a fully connected layer to output the value estimate V.
9. The loading optimization method for cylindrical cargo according to claim 8, characterized in that, After obtaining the feature vector sequence output, the Critic network introduces the Self-Attention mechanism: Self-Attention takes the feature vector sequence {f1,f2,...,fn} as attention input, calculates the attention weight ai of the feature vector fi at each position, and weights and aggregates the feature vectors at each position in the feature vector sequence to obtain the global feature representation f_att. Then, f_att is fed into the fully connected layer to output the value estimate V.
10. A loading optimization method for cylindrical cargo according to claim 8 or 9, characterized in that, The training and reward design of the bin packing optimization algorithm based on deep reinforcement learning is as follows: Building the training dataset: When training a deep reinforcement learning sequential generation network, a set of carrier units with different numbers and sizes is randomly generated to cover various scales and scenarios. During the data generation phase, the randomly generated binning instances are divided into training task sets and validation task sets according to a preset ratio. The training task set is used for parameter updates, while the validation task set does not participate in parameter updates and is only used to evaluate the performance and convergence of the model on unseen instances. Training configuration: Training configuration parameters include, but are not limited to: the number of carrying units, the range of length values, and the transportation space capacity. The parameters are adjusted according to different vehicle models, container specifications, and business needs to train a strategy model that is suitable for the corresponding scenario. Deep reinforcement learning training: The training process adopts a combination of "deep reinforcement learning output order + heuristic binning execution": the Actor network of the deep reinforcement learning model is responsible for generating the binning order, while the Critic network is used to output value estimates to guide policy updates; during training, the binning order generated by the Actor is executed by a heuristic binning algorithm, which includes, but is not limited to, Best-Fit or First-Fit. Reward Design: Suppose that a certain packing result uses B transport spaces, then calculate the occupancy rate of each transport space: that is, the sum of the lengths of all carrying units in the transport space divided by the capacity C; average the occupancy rates of all transport spaces to obtain a value between 0 and 1, which is used as the average occupancy rate; in the reward design, the average occupancy rate is used as the reward value, and the higher the average occupancy rate, the greater the reward. Convergence criteria: During training, the average occupancy rate on the validation set is continuously monitored; when the average occupancy rate no longer increases significantly within a certain number of rounds, or reaches a preset threshold, the model is considered to have converged; after convergence, the model parameters of Actor and Critic are fixed.