Encasement method, training method of sequence generation model and electronic equipment

Abstract

The invention provides a boxing method, a sequence generation model training method and electronic equipment. The boxing method comprises the following steps: acquiring attribute information of a plurality of to-be-loaded objects; based on the attribute information of the to-be-loaded objects, objects in the predicted loading sequence are determined step by step through a sequence generation model, a current loading sequence is obtained after the kth object is determined, and k is a positive integer smaller than the number of the to-be-loaded objects; determining an intermediate evaluation signal based on the current loading sequence; the intermediate evaluation signal indicates a volume utilization rate of the current loading sequence and a volume penalty determined based on a volume-to-weight relationship of the current loading sequence; adjusting a generation strategy of a sequence generation model based on the intermediate evaluation signal so as to improve the volume utilization rate and reduce the volume penalty in a subsequent sequence generation process; and based on the sequence generation model, continuously determining the (k + 1) th object in the predicted loading sequence through the adjusted generation strategy until a complete predicted loading sequence is obtained.

Description

Technical Field

[0001] This disclosure relates to the field of boxing technology, and in particular to a boxing method, a training method for a sequence generation model, and an electronic device. Background Technology

[0002] The packing methods in related technologies generate object loading sequences of poor quality, making it impossible to achieve optimal control of freight costs. Summary of the Invention

[0003] This disclosure provides a bin packing method, a training method for a sequence generation model, and an electronic device.

[0004] According to one aspect of this disclosure, a packing method is provided, comprising: acquiring attribute information of multiple objects to be loaded, the attribute information including volume information and weight information; based on the attribute information of the multiple objects to be loaded, progressively determining each object in a predicted loading sequence through a sequence generation model, and obtaining a current loading sequence after determining the k-th object, where k is a positive integer less than the number of multiple objects to be loaded; determining an intermediate evaluation signal based on the current loading sequence; the intermediate evaluation signal indicating the volume utilization rate of the current loading sequence and a volume penalty determined based on the volume-weight relationship of the current loading sequence; adjusting the generation strategy of the sequence generation model based on the intermediate evaluation signal to improve the volume utilization rate and reduce the volume penalty in subsequent sequence generation processes; and continuing to determine the (k+1)-th object in the predicted loading sequence through the adjusted generation strategy based on the sequence generation model until a complete predicted loading sequence is obtained.

[0005] According to embodiments of this disclosure, adjusting the generation strategy of the sequence generation model based on intermediate evaluation signals includes: reducing the sampling consistency of the sequence generation model in subsequent sequence generation processes when the volume utilization rate is less than a first threshold and the volume penalty is greater than or equal to a second threshold; and improving the sampling consistency of the sequence generation model in subsequent sequence generation processes when the volume utilization rate is greater than or equal to a third threshold and the volume penalty is less than a fourth threshold; wherein the first threshold is less than or equal to the third threshold and the second threshold is greater than or equal to the fourth threshold.

[0006] According to embodiments of this disclosure, determining intermediate evaluation signals based on the current loading sequence includes: performing a packing simulation based on the current loading sequence to determine the number of loading units consumed in the simulated packing state, as well as the loading weight and loading volume of each loading unit; determining the volume utilization rate based on the number of loading units consumed and the volume information of objects in the current loading sequence; and determining the volume penalty based on the loading weight and loading volume of each loading unit.

[0007] According to one aspect of this disclosure, a training method for a sequence generation model is provided, comprising: acquiring training data, the training data including attribute information of multiple objects to be loaded and corresponding target loading sequences; based on the attribute information, progressively determining each object in the predicted loading sequence through the sequence generation model, and obtaining the current loading sequence after determining the k-th object, where k is a positive integer less than the number of multiple objects to be loaded; determining an intermediate reward signal based on the current loading sequence and the target loading sequence; the intermediate reward signal characterizing the consistency between the current loading sequence and the target loading sequence; adjusting the parameters of the sequence generation model based on the intermediate reward signal to improve the consistency between the subsequently generated predicted loading sequence and the target loading sequence; and using the adjusted sequence generation model to continue determining the (k+1)-th object in the predicted loading sequence.

[0008] According to embodiments of this disclosure, determining an intermediate reward signal based on a current loading sequence and a target loading sequence includes: extracting objects that are common to the current loading sequence from the target loading sequence and maintaining the relative order of the objects in the target loading sequence to obtain a target comparison sequence; and determining an intermediate reward signal based on the degree of difference in the relative order between the current loading sequence and the target comparison sequence.

[0009] According to embodiments of this disclosure, determining an intermediate reward signal based on the degree of difference in the relative order between the current loading sequence and the target comparison sequence includes: determining at least one base sequence distance based on the current loading sequence and the target comparison sequence, wherein the base sequence distance characterizes the degree of difference in the relative order of objects in the current loading sequence and the target comparison sequence; obtaining the current sequence distance based on the at least one base sequence distance; and determining the intermediate reward signal based on the current sequence distance.

[0010] According to embodiments of this disclosure, at least one basic sequence distance includes: a first sequence distance, determined based on the number of pairs of objects with inconsistent relative orders in the current loading sequence and the target comparison sequence; a second sequence distance, determined based on the sum of the absolute values ​​of the position differences of each object in the current loading sequence and the target comparison sequence; a third sequence distance, determined based on measuring the relative order of objects in the current loading sequence and the target comparison sequence and calculating the number of operations required to make them consistent; a fourth sequence distance, determined based on the sum of the squares of the position differences of each object in the current loading sequence and the target comparison sequence; and a fifth sequence distance, determined based on the length of the longest common subsequence of the current loading sequence and the target comparison sequence.

[0011] According to embodiments of this disclosure, after iteratively adjusting the sequence generation model to obtain a complete predicted loading sequence, the method further includes: performing bin packing simulation based on the complete predicted loading sequence to obtain simulated bin packing results; determining the final reward signal based on the simulated bin packing results; and adjusting the parameters of the sequence generation model based on the final reward signal and the intermediate reward signals obtained during the generation of the predicted loading sequence. Specifically, this includes: evaluating the long-term value of each step in the sequence generation process based on the intermediate reward signals and the final reward signal to obtain a value assessment result; calculating the policy gradient of the output result of the sequence generation model at each step in the sequence generation process; scaling the policy gradient based on the value assessment result, and adjusting the parameters of the sequence generation model accordingly.

[0012] According to one aspect of this disclosure, a bin packing method is provided, which performs bin packing inference through a sequence generation model. The sequence generation model is trained according to the above-described training method. The bin packing method includes: obtaining attribute information of multiple objects to be loaded; and determining each object in the predicted loading sequence based on the attribute information of the multiple objects to be loaded using the sequence generation model.

[0013] Another aspect of this disclosure provides an electronic device comprising: at least one memory, a computer program stored in the memory; at least one processor; the processor executing the computer program to implement the steps of the method described above.

[0014] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0015] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein:

[0016] Figure 1 This is a flowchart of a packing method according to an embodiment of the present disclosure;

[0017] Figure 2 This is a flowchart of a training method for a sequence generation model according to an embodiment of the present disclosure;

[0018] Figure 3 This is a flowchart of a packing method according to another embodiment of the present disclosure;

[0019] Figure 4 This is a schematic diagram of the structure of an electronic device according to an embodiment of the present disclosure; and

[0020] Figure 5 This is a schematic block diagram of an example electronic device used to implement embodiments of the present disclosure. Detailed Implementation

[0021] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0022] In the technical solutions disclosed herein, the collection, storage, use, processing, transmission, provision, disclosure, and application of data (including but not limited to user personal information) comply with the provisions of relevant laws and regulations, necessary confidentiality measures have been taken, and they do not violate public order and good morals.

[0023] In the field of container shipping, there are special rules for freight calculation: when the volumetric weight of the pallet and the objects on it exceeds the actual weight, the freight will be charged based on the volumetric weight, and the resulting additional freight cost is called the volume penalty. Companies face a dual optimization objective when packing containers: on the one hand, they need to maximize pallet volume utilization to reduce the number of pallets; on the other hand, they need to control the volume penalty to reduce freight costs.

[0024] In some examples, when the number of objects is large, the quality of the object loading sequence generated by the packing method in related technologies is poor, making it difficult to quickly find the optimal solution and achieve optimal control of freight costs.

[0025] Figure 1 This is a flowchart of a packing method according to an embodiment of the present disclosure.

[0026] like Figure 1 As shown, the packing method of this embodiment includes operations S110-S150.

[0027] In operation S110, attribute information of multiple objects to be loaded is obtained, including volume information and weight information.

[0028] In the embodiments of this disclosure, attribute information is a set of data describing the physical characteristics of an object, including volume information and weight information. Volume information refers to the size of the object in three-dimensional space, which can be calculated by measuring the object's length, width, and height. For example, a regular cardboard box with dimensions of 0.8m × 0.6m × 0.5m has a volume of 0.24 cubic meters. Weight information refers to the object's mass, which can be obtained using a weighing device. For example, the cardboard box weighs 15 kilograms after being filled.

[0029] In the embodiments of this disclosure, attribute information of multiple objects to be loaded is obtained, including volume and weight information. The system acquires the basic physical parameters of all objects to be packed. In practical applications, the size data of the objects can be obtained through a scanning device, the weight data of the objects can be obtained through an electronic scale, and an attribute information database can be established to obtain the attribute information of multiple objects to be loaded.

[0030] For example, for 100 goods to be loaded, the system will record the unique number, volume information and weight information of each goods, forming a complete list of attribute information.

[0031] In operation S120, based on the attribute information of multiple objects to be loaded, each object in the predicted loading sequence is determined step by step through the sequence generation model. After determining the k-th object, the current loading sequence is obtained, where k is a positive integer less than the number of multiple objects to be loaded.

[0032] In the embodiments of this disclosure, the sequence generation model can be an artificial intelligence model that progressively outputs a sequence using an autoregressive approach, such as a neural network model based on the Transformer architecture. This model predicts the next most suitable object based on the generated sequence portion, effectively capturing complex dependencies between objects. Those skilled in the art should understand that any model capable of progressive sequence generation falls within the scope of this invention. This includes, but is not limited to, recurrent neural networks, search-based algorithms (such as greedy algorithms and bundle search), and other machine learning models (such as decision tree-based ranking models). These models can all receive object attribute information and progressively output predicted loading sequences based on specific strategies.

[0033] In embodiments of this disclosure, the predicted loading sequence refers to the object loading order output by the sequence generation model. The current loading sequence refers to the sequence consisting of the first k objects determined during the generation process.

[0034] In embodiments of this disclosure, the sequence generation model generates a complete sequence step by step, starting with the first object. At step k, the sequence generation model can calculate the probability distribution of the next object and select the optimal option based on the attributes of the k-1 selected objects and the remaining objects.

[0035] For example, for 100 goods to be loaded, when k=5, the current loading sequence contains the first 4 objects. The model will analyze the attributes of the remaining 96 objects and select the 5th object that best suits the current packing state.

[0036] In operation S130, an intermediate evaluation signal is determined based on the current loading sequence; the intermediate evaluation signal indicates the volume utilization rate of the current loading sequence and the volume penalty determined based on the volume-weight relationship of the current loading sequence.

[0037] In the embodiments of this disclosure, the intermediate evaluation signal is a quantitative evaluation value of the packing effect of the current loading sequence, and the intermediate evaluation signal may include two indicators: volume utilization rate and volume penalty.

[0038] In the embodiments of this disclosure, the volume utilization rate of the current loading sequence refers to the comprehensive characterization value of the total volume utilization rate of all loading units after simulating the loading of the currently sorted k objects into at least one loading unit in sequence. It reflects the overall spatial compactness of these k objects.

[0039] In the embodiments of this disclosure, the volume penalty of the current loading sequence refers to the comprehensive characterization value of the volume penalty generated by all loading units after simulating the loading of the currently sorted k objects into at least one loading unit in sequence. It reflects the overall risk of cost overrun that may result from the current sorting of these k objects.

[0040] For example, in an air transport scenario, when k=10, 10 objects are simulated to be loaded into at least one standard air transport pallet in sequence. The volume utilization rate of the current loading sequence is calculated to be 68%, and the volume penalty of the current loading sequence is 35 yuan based on the freight calculation rules.

[0041] In operation S140, the generation strategy of the sequence generation model is adjusted based on the intermediate evaluation signal to improve volume utilization and reduce volume penalty in subsequent sequence generation processes.

[0042] In the embodiments of this disclosure, the generation strategy refers to the decision rules or preferences that the sequence generation model follows when selecting the next object from the remaining unselected objects at each step. The generation strategy is not a specific parameter, but rather a general reflection of the model's internal decision-making logic. The generation strategy determines which characteristics the model is more inclined to choose when faced with multiple possibilities.

[0043] In embodiments of this disclosure, the system uses intermediate evaluation signals as feedback information to modify the generation strategy of the sequence generation model. If the intermediate evaluation signal indicates that the current sequence is not ideal (e.g., low volume utilization and high volume penalty), it means that the model's current "selection criteria" is incorrect and needs to be adjusted immediately to make better decisions in subsequent selections.

[0044] For example, assuming the current loading sequence contains k=15 objects, after simulating loading 3 pallets, the calculated overall volume utilization rate is low (50%), and the overall volume penalty is high (80 yuan). This indicates that the current sequence suffers from serious space waste and an excessively high proportion of low-density goods.

[0045] Before the adjustment, the model may have implicitly followed a strategy of "prioritizing heavy objects" because heavy goods can meet the pallet's weight limit more quickly, but this may result in the pallet space not being filled and low volume utilization.

[0046] The generation strategy of the sequence generation model can be adjusted based on intermediate evaluation signals. For example, the weight given to the "object volume" feature can be increased: when selecting the 16th object, the model will consider the object's size more. It will be more inclined to select a large-volume object that can fit perfectly into the remaining space of the current tray, rather than a heavy, small object. This will improve the current tray's fill rate, thereby improving the overall volume utilization.

[0047] For example, a preference for "object density" (volume / weight) can be introduced: to avoid increasing volume penalties by selecting low-density goods again, the strategy will be adjusted to prioritize high-density objects (i.e., heavy but relatively small objects). Such objects can increase pallet weight without significantly increasing volume, thus tilting freight calculations towards "weight-based billing" and reducing volume penalties.

[0048] In the embodiments of this disclosure, the generation strategy of the sequence generation model is adjusted by intermediate evaluation signals, and the macro-optimization objectives (improving utilization and reducing costs) are converted into micro-decision instructions for the model when selecting the next object in real time and operablely, thereby ensuring that the final predicted loading sequence achieves a better level in terms of both volume utilization and cost control.

[0049] In operation S150, based on the sequence generation model, the k+1th object in the predicted loading sequence is determined through the adjusted generation strategy until the complete predicted loading sequence is obtained.

[0050] In the embodiments of this disclosure, the steps described herein depict iterative execution and closed-loop feedback. This step follows the generation strategy adjustment completed in operation S140, applying the adjusted generation strategy to the subsequent operation of the sequence generation model. Its core lies in the fact that the model no longer uses the old, potentially inefficient decision-making method, but instead, based on the latest optimization instructions, starts from the current interruption point (after the k-th object) and continues to construct the sequence based on the new generation strategy. This process is not a one-time action, but a cycle of "generation-evaluation-adjustment-continue generation" until the sequence is completed, yielding a complete predicted loading sequence.

[0051] Through the embodiments of this disclosure, by evaluating the volume utilization rate and volume penalty index in the sequence generation process in real time, and adjusting the generation strategy of the sequence generation model based on intermediate evaluation signals, the optimization direction can be adjusted in a timely manner during the generation process, significantly improving the quality and practicality of the packing method. Through a cyclical evaluation and adjustment mechanism, a better balance between volume utilization rate and freight cost control is ensured in the final loading sequence, effectively improving packing quality.

[0052] In some embodiments, adjusting the generation strategy of the sequence generation model based on intermediate evaluation signals includes: reducing the sampling consistency of the sequence generation model in subsequent sequence generation processes when the volume utilization rate is less than a first threshold and the volume penalty is greater than or equal to a second threshold; and increasing the sampling consistency of the sequence generation model in subsequent sequence generation processes when the volume utilization rate is greater than or equal to a third threshold and the volume penalty is less than a fourth threshold; wherein the first threshold is less than or equal to the third threshold and the second threshold is greater than or equal to the fourth threshold.

[0053] In embodiments of this disclosure, the first threshold and the third threshold are benchmark reference values ​​for volume utilization, used to determine whether the space utilization of the current loading sequence meets the standard. For example, the first threshold can be set to 60%, and the third threshold can be set to 75%.

[0054] In embodiments of this disclosure, the second threshold and the fourth threshold are benchmark reference values ​​for volume penalty, used to evaluate the cost control effect of the current loading sequence. For example, the second threshold can be set to 50 yuan, and the fourth threshold can be set to 30 yuan.

[0055] In the embodiments of this disclosure, sampling consistency refers to the stability and predictability of the sequence generation model across multiple selections. Sampling consistency reflects the model's confidence in the current optimal solution. Improving sampling consistency means enhancing the determinism and focus of the model's decision-making, enabling the model to firmly select the solution with the best current evaluation effect and reducing random exploration. Decreasing sampling consistency means increasing the randomness and exploration range of the model's decision-making, causing the model to not only consider the currently considered optimal choice but also try suboptimal solutions with a certain probability.

[0056] For example, suppose a sequence generation model needs to select the next object from 10 candidate objects. When the sampling consistency is high, the model has a 90% probability of selecting the first-ranked object and a 10% probability of randomly selecting another object. When the sampling consistency is low, the probability of the model selecting the first-ranked object drops to 50%, and the remaining probability is evenly distributed among the other candidate objects.

[0057] In the embodiments of this disclosure, the system monitors two key indicators in real time: volume utilization and volume penalty. When the combination of indicators reaches a preset threshold, the corresponding sampling consistency adjustment is automatically triggered.

[0058] In embodiments of this disclosure, when the volume utilization rate is less than a first threshold and the volume penalty is greater than or equal to a second threshold, the sequence generation model reduces the consistency of sampling during subsequent sequence generation.

[0059] For example, when generating the 25th object, the virtual binning simulation shows a volume utilization rate of 55% (less than the first threshold of 60%) and a volume penalty of 65 yuan (greater than the second threshold of 50 yuan). The system determines that the current path quality is poor and performs a "reduce sampling consistency" operation. Specifically, for example, the sampling temperature parameter can be increased from the baseline value of 1.0 to 1.8. After this adjustment, when selecting the 26th object, the probability of selecting the object with the highest probability decreases from 70% to 40%, while the probabilities of selecting the second and third most probable objects increase from 15% and 8% to 25% and 18%, respectively. The remaining probabilities are allocated to other candidate objects. This adjustment allows the model to escape potentially local optima and explore more diverse object combination schemes.

[0060] In embodiments of this disclosure, when the volume utilization rate is greater than or equal to a third threshold and the volume penalty is less than a fourth threshold, the consistency of sampling is improved in the sequence generation model during subsequent sequence generation.

[0061] For example, when generating the 40th object, the virtual binning simulation shows a volume utilization rate of 78% (greater than the third threshold of 75%) and a volume penalty of 25 yuan (less than the fourth threshold of 30 yuan). The system determines that the current path is effective and executes the "improve sampling consistency" operation. Specifically, this can be achieved by reducing the sampling temperature parameter from 1.0 to 0.3. After this adjustment, when selecting the 41st object, the probability of selecting the object with the highest probability increases from 60% to 90%, while the total probability of selecting other candidate objects is reduced from 40% to 10%. This adjustment allows the model to continue along the currently successful path, avoiding unnecessary random perturbations and quickly converging to a high-quality solution.

[0062] Through the embodiments of this disclosure, dynamic adjustment of sampling consistency is achieved by establishing a quantified threshold judgment system. When poor sequence quality is detected, the exploratory nature is automatically enhanced to find new paths; when the sequence quality is good, the determinism is automatically enhanced to achieve rapid convergence. The method of this embodiment significantly improves the efficiency of sequence search and the quality of the final generated loading sequence.

[0063] In some embodiments, determining intermediate evaluation signals based on the current loading sequence includes: performing a packing simulation based on the current loading sequence to determine the number of loading units consumed in the simulated packing state of the current loading sequence, as well as the loading weight and loading volume of each loading unit; determining the volume utilization rate based on the number of loading units consumed and the volume information of objects in the current loading sequence; and determining the volume penalty based on the loading weight and loading volume of each loading unit.

[0064] In embodiments of this disclosure, a packing simulation is performed based on the current loading sequence to determine the number of loading units consumed in the simulated packing state, as well as the loading weight and volume of each loading unit. Packing simulation refers to the process of virtually loading a currently sorted sequence of objects into standard loading units in a computer environment according to preset loading rules. This process simulates actual packing operations but does not involve physical handling.

[0065] In the embodiments of this disclosure, the number of loading units consumed in the simulated loading state refers to the total number of loading units (such as pallets, containers) consumed to complete the virtual loading of all objects in the current loading sequence. The loading weight of each loading unit refers to the sum of the weights of all objects within a single loading unit, reflecting the weight load of that unit. The loading volume of each loading unit refers to the total volume of all objects within a single loading unit, reflecting the space occupancy of that unit.

[0066] For example, in an air transport loading scenario, assuming the current loading sequence contains 35 objects, the system performs a packing simulation: starting with the first object, it attempts to place it onto existing pallets sequentially; if the pallet cannot fit, a new pallet is used. After the simulation completes, it may consume 3 standard air cargo pallets (1.5m³ in volume). 3 (Load capacity 100kg). Pallet 1 has a loading capacity of 85kg and a loading volume of 1.3m³. 3 Pallet 2 has a loading weight of 78kg and a loading volume of 1.4m³. 3 Pallet 3 has a loading weight of 45kg and a loading volume of 0.9m³. 3 .

[0067] In embodiments of this disclosure, volume utilization is determined based on the number of loading units consumed and the volume information of objects in the current loading sequence. The ratio of the total volume of all objects in the current loading sequence to the total volume of the loading units consumed can reflect space utilization.

[0068] For example, pallet 1 has a volume of 1.5 m³ and actually loads 1.3 m³. Pallet 2 has a volume of 1.5 m³ and actually loads 1.4 m³. Pallet 3 has a volume of 1.5 m³ and actually loads 0.9 m³. Therefore, the volume utilization rate can be calculated as (1.3 + 1.4 + 0.9) / (1.5 + 1.5 + 1.5) * 100% = 80%.

[0069] In embodiments of this disclosure, a volume penalty is determined based on the loading weight and volume of each loading unit. This is an additional charge incurred when the volume-based freight cost exceeds the weight-based freight cost, based on freight calculation rules. The volume penalty can be calculated on a pallet-by-pallet basis and then accumulated.

[0070] For example, for each loading unit.

[0071] Calculate volumetric weight = loading volume × conversion factor (e.g., 167 kg / m³).

[0072] Calculate volumetric shipping cost = max(actual weight, volumetric weight) × rate.

[0073] Calculate freight cost by weight = actual weight × rate.

[0074] Single pallet volume penalty = max(0, volume freight - weight freight).

[0075] Continuing from the previous example (assuming a rate of 5 yuan / kg and a conversion factor of 167 kg / m³).

[0076] Pallet 1: Volumetric weight = 1.3 × 167 = 217 kg > Actual weight 85 kg, Volume penalty = (217 - 85) × 5 = 660 yuan.

[0077] Pallet 2: Volumetric weight = 1.4 × 167 = 234 kg > Actual weight 78 kg, Volume penalty = (234 - 78) × 5 = 780 yuan.

[0078] Pallet 3: Volumetric weight = 0.9 × 167 = 150 kg > Actual weight 45 kg, Volume penalty = (150 - 45) × 5 = 525 yuan.

[0079] Total volume penalty = 660 + 780 + 525 = 1965 yuan.

[0080] Through the embodiments of this disclosure, abstract loading effects are transformed into precise volume utilization and volume penalty indicators through specific packing simulation and quantitative calculation. For multi-loading unit scenarios, the accuracy and representativeness of the evaluation signal can be ensured.

[0081] Figure 2 This is a flowchart of a training method for a sequence generation model according to an embodiment of the present disclosure.

[0082] like Figure 2 As shown, the training method of this embodiment includes operations S210-S240.

[0083] In operation S210, training data is acquired, which includes attribute information of multiple objects to be loaded and their corresponding target loading sequences.

[0084] In some examples, traditional sequence generation models face the problem of reward sparsity during training, as reward signals are only obtained after the complete sequence is generated, resulting in low training efficiency and a tendency to get trapped in local optima.

[0085] The sequence generation model obtained by the training method of the sequence generation model based on the embodiments of this disclosure can be used in the packing method of the foregoing embodiments.

[0086] In embodiments of this disclosure, the training data is a sample set used to train the sequence generation model, which includes input features (attribute information of multiple objects to be loaded) and target output results (target loading sequence).

[0087] In the embodiments of this disclosure, attribute information refers to feature data describing the physical characteristics of the object to be loaded, including but not limited to attributes such as the object's size, weight, volume, shape, center of gravity, and fragility. Attribute information constitutes the input feature space for model decision-making.

[0088] In embodiments of this disclosure, the target loading sequence is a standard loading order, representing the ideal arrangement of objects under the current optimization objective. The target loading sequence serves as an optimization benchmark during model training. The target loading sequence can be provided by an optimization algorithm or a domain expert, or it can be determined based on historical loading sequence data.

[0089] In operation S220, based on attribute information, each object in the predicted loading sequence is determined step by step through the sequence generation model. After determining the k-th object, the current loading sequence is obtained, where k is a positive integer less than the number of multiple objects to be loaded.

[0090] In embodiments of this disclosure, the sequence generation model can be a neural network model that generates sequences stepwise using an autoregressive approach, such as a generative model based on the Transformer architecture. The sequence generation model captures complex dependencies between objects through an attention mechanism.

[0091] Those skilled in the art will understand that any model capable of generating a step-by-step sequence falls within the scope of this invention. This includes, but is not limited to, recurrent neural networks, search-based algorithms (such as greedy algorithms and beam search), and other machine learning models (such as decision tree-based ranking models). These models can all receive object attribute information and output predicted loading sequences step-by-step based on specific strategies.

[0092] In the embodiments of this disclosure, the predicted loading sequence refers to the loading order output by the sequence generation model after sorting the input objects according to the currently learned strategy. The current loading sequence refers to the partial sequence result output by the model in the intermediate stage of the generation process.

[0093] In the embodiments of this disclosure, the sequence generation model generates sequences in a stepwise manner, with each step using the generated partial sequence as historical information and combining it with the features of the remaining objects to predict the next most suitable object.

[0094] For example, assuming there are 30 objects to be loaded, when k=5, the sequence generation model has generated the sequence of the first 5 objects: [object D, object A, object C, object H, object M]. Now we need to determine the 6th object. Based on the generated current loading sequence and the attribute information of the remaining 25 objects, the sequence generation model calculates the selection probability of each candidate object. The model selects the object F with the highest probability as the 6th object and updates the current loading sequence to: [object D, object A, object C, object H, object M, object F].

[0095] In operation S230, an intermediate reward signal is determined based on the current loading sequence and the target loading sequence; the intermediate reward signal represents the consistency between the current loading sequence and the target loading sequence.

[0096] In the embodiments of this disclosure, the intermediate reward signal refers to a quantitative evaluation value of the similarity between the current loading sequence and the target loading sequence during the sequence generation process. The intermediate reward signal is used to measure the similarity between the current loading sequence and the target loading sequence in the order of object arrangement. The intermediate reward signal focuses not on the absolute position of objects, but on whether the sequential relationship between pairs of objects is consistent.

[0097] For example, in the target loading sequence, object B precedes object C. If, in the current loading sequence, both object B and object C appear, and object B still precedes object C, then consistency is maintained for this pair of objects.

[0098] In operation S240, the parameters of the sequence generation model are adjusted based on the intermediate reward signal to improve the consistency between the subsequently generated predicted loading sequence and the target loading sequence; the adjusted sequence generation model is used to continue to determine the (k+1)th object in the predicted loading sequence.

[0099] In embodiments of this disclosure, the parameters of the sequence generation model are adjusted based on intermediate reward signals. Adjusting the model parameters refers to changing the values ​​of learnable variables within the sequence generation model. These parameters determine the mapping relationship between the model's input (generated sequences, object attribute information) and its output (the probability distribution of the next object).

[0100] For example, in neural networks, adjusting model parameters includes the attention weight matrix in the Transformer, the weights and biases of the feedforward neural layers, and the parameters of the layer normalization layer.

[0101] In the embodiments of this disclosure, the parameters of the sequence generation model are adjusted based on intermediate reward signals to improve the consistency between the subsequently generated predicted loading sequence and the target loading sequence. Through parameter adjustment, the model parameters converge to a set of optimal values. When the model possesses this set of optimal parameters, its inherent decision-making logic will cause it to tend to make choices more consistent with the target sequence at each step of sequence generation.

[0102] In the embodiments of this disclosure, the training process can be implemented using the policy gradient algorithm in reinforcement learning. If the model receives a high reward after taking an action in a certain state, the parameters are fine-tuned to increase the probability of the model taking that action in a similar state in the future. Conversely, if a low reward is obtained, the probability of that action is reduced.

[0103] In embodiments of this disclosure, the adjusted sequence generation model is used to continue determining the (k+1)th object in the predicted loading sequence. That is, the above training and adjustment process continues until a complete sequence is generated. Through multiple iterations of sequence generation and parameter adjustment, the parameters of the sequence generation model will be optimized so that when the sequence generation model starts autoregressively from an empty sequence, it selects the object with the highest probability at each step, resulting in a sequence that is very close to the target loading sequence.

[0104] The embodiments of this disclosure effectively solve the reward sparsity problem in sequence generation model training by introducing an intermediate reward signal mechanism. By providing real-time consistency feedback during the generation process, the model can adjust its generation strategy in a timely manner, significantly improving training efficiency and convergence speed.

[0105] In some embodiments, determining an intermediate reward signal based on the current loading sequence and the target loading sequence includes: extracting objects that are common to the current loading sequence from the target loading sequence and maintaining the relative order of the objects in the target loading sequence to obtain a target comparison sequence; and determining the intermediate reward signal based on the degree of difference in the relative order between the current loading sequence and the target comparison sequence.

[0106] In embodiments of this disclosure, objects commonly contained in both the target loading sequence and the current loading sequence are extracted, while maintaining the relative order of the objects in the target loading sequence, to obtain a target comparison sequence. Since the current loading sequence is a partial sequence, it may not contain all objects from the target loading sequence, making a direct comparison of the complete sequences unreasonable. Therefore, it is necessary to determine the target comparison sequence based on the objects commonly contained in both sequences. Commonly contained objects refer to the set of objects that appear simultaneously in both the current loading sequence and the target loading sequence.

[0107] For example, suppose the target loading sequence is [A, B, C, D, E, F], and the current loading sequence is [A, D, C, B]. Then the objects that are commonly included are {A, B, C, D}. Objects E and F are in the target loading sequence, but have not yet appeared in the current loading sequence, so they are not included in the comparison in this step.

[0108] In embodiments of this disclosure, relative order refers to the order in which objects appear in a sequence. Comparing relative order refers to comparing whether the order of the same pair of objects is consistent in two sequences.

[0109] For example, for an object pair (B, C), B precedes C in the target loading sequence, but C precedes B in the current loading sequence. This is inconsistent in the relative order of the objects.

[0110] In embodiments of this disclosure, the target comparison sequence refers to a new sequence formed by extracting objects that are commonly contained in the current loading sequence from the target loading sequence and maintaining the original relative order of these objects in the target loading sequence.

[0111] Continuing from the previous example, extract the common objects {A, B, C, D} from the target loading sequence [A, B, C, D] and maintain their relative order in the target loading sequence (A→B→C→D) to obtain the target comparison sequence [A, B, C, D].

[0112] In embodiments of this disclosure, an intermediate reward signal is determined based on the degree of difference in the relative order between the currently loaded sequence and the target comparison sequence. After obtaining comparable sequences, the intermediate reward signal is determined based on the degree of difference in the relative order between the two sequences. The degree of difference is typically negatively correlated with the reward signal.

[0113] Through the embodiments of this disclosure, the reward signal calculation based on the degree of relative order difference can accurately reflect the degree of consistency between the current loading sequence and the target loading sequence. The method of this embodiment improves the efficiency and accuracy of determining intermediate reward signals.

[0114] In some embodiments, determining an intermediate reward signal based on the degree of difference in the relative order between the current loading sequence and the target comparison sequence includes: determining at least one base sequence distance based on the current loading sequence and the target comparison sequence, wherein the base sequence distance characterizes the degree of difference in the relative order of objects in the current loading sequence and the target comparison sequence; obtaining the current sequence distance based on the at least one base sequence distance; and determining the intermediate reward signal based on the current sequence distance.

[0115] In the embodiments of this disclosure, different basic sequence distances are quantitative indicators that measure the degree of relative order difference between two sequences from different mathematical perspectives. Each basic sequence distance reflects a certain aspect of the inconsistency between the two sequences through a specific calculation principle.

[0116] In embodiments of this disclosure, the current sequence distance is obtained based on at least one base sequence distance. The current sequence distance can be a single quantified value representing the overall degree of difference between two sequences, obtained by combining one or more base sequence distances. The current sequence distance serves as a direct input for calculating the reward signal.

[0117] For example, the distance to the current sequence can be obtained by taking the average, weighted sum, or maximum value of various basic distances.

[0118] For example, based on the current loaded sequence and the target comparison sequence, the distance between the first base sequence and the second base sequence are determined. The average of the first base sequence distance and the second base sequence distance is then taken as the current sequence distance.

[0119] For example, based on the current loaded sequence and the target comparison sequence, the distance between the first base sequence and the second base sequence is determined, and the larger of the first base sequence distance and the second base sequence distance is selected as the current sequence distance.

[0120] In embodiments of this disclosure, an intermediate reward signal is determined based on the current sequence distance. The steps of this embodiment map the sequence distance to a reward signal usable for reinforcement learning. A negative correlation mapping can be used, where a smaller distance results in a higher reward, encouraging the sequence generation model to reduce the difference between itself and the target loaded sequence.

[0121] Through the embodiments of this disclosure, a stable and robust intermediate reward signal is provided for the subsequent training and adjustment process by comprehensively evaluating multi-dimensional sequence distances. The hierarchical calculation method of this embodiment enhances the accuracy and guidance of the intermediate reward signal, effectively improving the efficiency and quality of model training.

[0122] In some embodiments, at least one basic sequence distance includes: a first sequence distance, determined based on the number of pairs of objects in inconsistent relative order in the current loading sequence and the target comparison sequence; a second sequence distance, determined based on the sum of the absolute values ​​of the positional differences of each object in the current loading sequence and the target comparison sequence; a third sequence distance, determined based on measuring the relative order of objects in the current loading sequence and the target comparison sequence and calculating the number of operations required to make them consistent; a fourth sequence distance, determined based on the sum of the squares of the positional differences of each object in the current loading sequence and the target comparison sequence; and a fifth sequence distance, determined based on the length of the longest common subsequence of the current loading sequence and the target comparison sequence.

[0123] In embodiments of this disclosure, the first sequence distance is determined based on the number of object pairs whose relative order is inconsistent in the current loading sequence and the target comparison sequence. Each inconsistent object pair refers to the same pair of objects in reverse order in the two sequences.

[0124] For example, given the current loading sequence [A, D, C, B], the target comparison sequence [A, B, C, D], checking all object pairs, (B, C), (C, D), and (B, D) are three inversion pairs, and the first sequence distance is 3.

[0125] In embodiments of this disclosure, the second sequence distance is determined based on the sum of the absolute values ​​of the positional differences of each object in the current loading sequence and the target comparison sequence. The second sequence distance is measured by calculating the sum of the absolute values ​​of the positional differences of each object in the two sequences.

[0126] For example, given the current loading sequence [A, D, C, B], with object positions A-1, D-2, C-3, B-4, and the target comparison sequence [A, B, C, D], with object positions A-1, B-2, C-3, D-4, calculate the absolute value of the position difference for each object. A: |1−1| = 0. B: |2−4| = 2. C: |3−3| = 0. D: |4−2| = 2. The second sequence distance = 0 + 2 + 0 + 2 = 4.

[0127] In embodiments of this disclosure, the third sequence distance, based on a measure of the relative order of objects in the current loaded sequence and the target comparison sequence, is determined by calculating the number of operations required to make them consistent. The third sequence distance is measured by calculating the minimum number of adjacent swaps required to transform one sequence into another.

[0128] For example, given the current loading sequence [A, D, C, B], with object positions A-1, D-2, C-3, B-4, and the target comparison sequence [A, B, C, D], with object positions A-1, B-2, C-3, D-4, after swapping B and D in the first step, the two sequences are identical, and the distance between the three sequences is 1.

[0129] In embodiments of this disclosure, the fourth sequence distance is determined based on the sum of squared differences in the positions of each object in the current loading sequence and the target comparison sequence. The fourth sequence distance is measured by calculating the sum of squared differences in the positions of each object in the two sequences.

[0130] For example, given the current loading sequence [A, D, C, B], with object positions A-1, D-2, C-3, B-4, and the target comparison sequence [A, B, C, D], with object positions A-1, B-2, C-3, D-4, calculate the absolute value of the position difference for each object. A: (1-1) 2 =0. B: (2−4) 2 =4. C: (3−3)2 =0. D: (4−2) 2 =4. The distance of the fourth sequence = 0 + 4 + 0 + 4 = 8.

[0131] In embodiments of this disclosure, the fifth sequence distance is determined based on the length of the longest common subsequence of the currently loaded sequence and the target comparison sequence. The fifth sequence distance is typically expressed as the difference between the total sequence length and the length of the longest common subsequence.

[0132] For example, given the current loading sequence [A, D, C, B], with object positions A-1, D-2, C-3, B-4, and the target comparison sequence [A, B, C, D], with object positions A-1, B-2, C-3, D-4, the longest common subsequence is [A]. The total sequence length is 4, and the distance to the fifth sequence is 4-1=3.

[0133] The embodiments of this disclosure provide five specific sequence distance measurement methods to quantify sequence differences from different mathematical perspectives. These multiple distance measurement methods can be used individually or in combination, enhancing the comprehensiveness and robustness of intermediate reward signal evaluation and effectively improving the stability and convergence of model training.

[0134] In some embodiments, after iteratively adjusting the sequence generation model to obtain a complete predicted loading sequence, the method further includes: performing bin packing simulation based on the complete predicted loading sequence to obtain simulated bin packing results; determining the final reward signal based on the simulated bin packing results; and adjusting the parameters of the sequence generation model based on the final reward signal and the intermediate reward signals obtained during the generation of the predicted loading sequence. Specifically, this includes: evaluating the long-term value of each step in the sequence generation process based on the intermediate reward signals and the final reward signal to obtain a value assessment result; calculating the policy gradient of the output result of the sequence generation model at each step in the sequence generation process; scaling the policy gradient based on the value assessment result, and adjusting the parameters of the sequence generation model accordingly.

[0135] In the embodiments of this disclosure, a bin-packing simulation is performed based on the complete predicted loading sequence to obtain the simulated bin-packing result. The simulated bin-packing result refers to the quantitative evaluation result obtained after performing a bin-packing operation on the complete predicted loading sequence in a virtual environment, which may include indicators such as space utilization, number of loading units, and weight distribution.

[0136] In embodiments of this disclosure, a final reward signal is determined based on the simulated binning results. The final reward signal is a reward value calculated based on the simulated binning results, reflecting the overall quality of the complete sequence. This signal evaluates the final outcome of the sequence generation task.

[0137] For example, the complete predicted loading sequence is input into the packing simulator, and virtual loading is performed according to the actual loading rules and constraints. Key performance indicators during the loading process are collected, and a reward function is designed according to business needs to transform the performance indicators into the final reward signal.

[0138] For example, the final reward signal could be a combination of a positive reward for space utilization and a negative reward for volume penalty.

[0139] In the embodiments of this disclosure, the long-term value of each step in the sequence generation process is evaluated based on intermediate and final reward signals to obtain a value assessment result. Long-term value refers to the assessment of the cumulative rewards that a state or decision may obtain in the future, taking into account the long-term impact of the current decision, rather than just the immediate reward.

[0140] In the embodiments of this disclosure, for each step k in the sequence generation process, the cumulative discounted reward from the start of that step to the end of the sequence can be calculated. The discount factor is used to balance the importance of immediate rewards and future rewards, and the value assessment result reflects the degree of contribution of each decision to the final result.

[0141] For example, suppose a sequence of length 4 is generated, with a discount factor γ = 0.9. The intermediate reward signals at each step are: r1 = 0.2, r2 = 0.3, r3 = 0.1, r4 (final reward) = -15. The long-run value of the first step is: R1 = r1 + γ × r2 + γ 2 ×r3+γ 3 ×r4, we can calculate R1=0.2+0.9×0.3+0.81×0.1+0.729×(-15)≈-10.5.

[0142] In embodiments of this disclosure, the policy gradient of the output result of the sequence generation model at each step during sequence generation is calculated. The steps in this embodiment calculate the gradient of the model parameters with respect to the decision probabilities, indicating how the model parameters should be adjusted to increase (or decrease) the probability of certain decisions occurring. For each step in the generated sequence, the probability of the output action at that step is calculated, and the gradient of that probability with respect to the model parameters is calculated. The gradient direction indicates the direction of parameter adjustment to increase the probability of that action.

[0143] For example, suppose in the second step, the model selects object A with an 80% probability. Calculate the corresponding policy gradient. This gradient is used to indicate how the model parameters should be adjusted to increase the probability of selecting A in similar situations.

[0144] In embodiments of this disclosure, the policy gradient is scaled based on the value assessment results, and the parameters of the sequence generation model are adjusted accordingly. The steps of this embodiment use long-term value assessment results as weights to scale the policy gradient, and then update the parameters. Decisions with higher value (higher long-term returns) receive a larger update magnitude for their gradients.

[0145] Through embodiments of this disclosure, by introducing a gradient scaling mechanism based on long-term value, the contribution of each decision to the final result can be accurately evaluated, significantly improving training efficiency and model performance.

[0146] In some embodiments, a method for training a sequence generation model is provided, with the aim of training the sequence generation model (such as a Transformer-based neural network) to generate high-quality object loading sequences.

[0147] In the embodiments of this disclosure, there are n objects to be loaded, numbered as follows: The target loading sequence is... Let be the reference optimal order for n objects to be loaded. A predicted loading sequence is then output step-by-step through a sequence generation model. The k-th object is selected from the set of remaining unselected objects. .

[0148] In embodiments of this disclosure, the training process for the sequence generation model is an autoregressive loop process in which the following steps are performed at each step k (from 1 to n).

[0149] Step 1: Determine the state s at step k. k Includes the current load sequence and the feature vectors of all objects. This state encapsulates the "decision history" up to this point and all available object information, providing the model with all the context needed to make the next decision.

[0150] Step 2, Determine the motion space That is, the set of all remaining unselected objects.

[0151] Step 3: Input the current loaded sequence and the feature vectors of all objects into the sequence to generate the model, obtain the selection probability calculated by the model for each remaining object in the action space, and form a probability distribution. Where W is a learnable weight matrix and TransformerDec is the Transformer decoder function. The k-th object is determined by sampling or greedy selection (selecting the object with the highest probability).

[0152] Step four: Calculate the intermediate reward signal. Immediately after generating an object in each step k, calculate the intermediate reward signal. Obtain the agent's current prefix. This refers to the partial sequence already generated by the model. Obtain the optimal reference prefix. Since the order in which the model is generated may differ from the optimal order, the objects contained in the two prefixes may not be exactly the same. Direct comparison is unfair and meaningless; therefore, "projection" is needed to determine the set of objects commonly contained in the two prefix sequences. .

[0153] Step 5: Calculate the normalized sequence distance. Here, we take the first sequence distance from the aforementioned embodiment as an example for calculation. Calculate the normalized first sequence distance. .in, The distance between the first sequence is calculated as the number of inversion pairs (object pairs with opposite relative order) in the two projected sequences. It is the number of all possible object pairs in the common element set, used to normalize the distance to the interval [0, 1].

[0154] Step six: Identify intermediate reward signals. A negative reward for distance means that the sequence generated by the model is more similar to the optimal sequence (distance). The smaller the value, the higher the reward ( The larger the value, the better. This encourages the model's behavior to align closer to the target loading sequence.

[0155] Furthermore, after the model generates the complete sequence π^ (i.e., k=n), a final evaluation is performed to calculate the final reward signal. Simulated binning is then conducted, performing a virtual binning operation according to the order of the generated complete sequence π^. The loss value is calculated based on the binning result. This loss comprehensively reflects the quality of the packaging.

[0156] For example, loading unit space utilization (the lower the better), volume penalty (the higher the worse).

[0157] The number of loading units used (the more, the worse).

[0158] Furthermore, define the final step reward. The negative sign is also used here to encourage the model to generate sequences that result in low binning loss (i.e., high-quality binning).

[0159] Further, reward calculation and policy gradient update. This is the end of the training loop, where the model parameters are updated using the rewards generated throughout the entire round (from step 1 to step n). The discounted reward is calculated; for each step k, the total reward obtainable from the start of that step to the end of the sequence is calculated. Typically, the discount factor γ is set to 1, meaning that the decay of future rewards is not considered. This represents the overall quality of the model's subsequent decisions starting from step k.

[0160] Furthermore, the model parameters θ are updated using a policy gradient algorithm. If a certain action sequence yields a high reward, the model parameters are adjusted to increase the probability of generating that action sequence in the future. .

[0161] in, The policy gradient indicates the direction in which the parameter θ should be fine-tuned to increase the efficiency of the state. Select action The logarithmic probability. Represents the dominance function. It refers to the actual returns received. It is a baseline (e.g., a moving average of returns) used to reduce the variance of gradient estimates. If A value greater than 0 indicates that the performance of this step is better than average, and this action should be encouraged. Therefore, the positive gradient direction is reasonable.

[0162] By introducing intermediate rewards based on ranking distance metrics through embodiments of this disclosure, the sparse final reward signal is densified, providing the model with immediate and guided feedback at each step in generating long sequences. The method of this embodiment alleviates the credit allocation problem and sparse reward problem faced by reinforcement learning in sequence generation tasks, significantly improving training efficiency and enabling the model to learn near-optimal bin packing strategies more effectively.

[0163] Figure 3 This is a flowchart of a packing method according to another embodiment of the present disclosure.

[0164] like Figure 3 As shown, the bin packing method of this embodiment performs bin packing inference through a sequence generation model. The sequence generation model is trained according to the training method of the aforementioned embodiment. The bin packing method includes operations S310-S320.

[0165] In operation S310, attribute information of multiple objects to be loaded is obtained.

[0166] In the embodiments of this disclosure, the steps are the input preparation stage of the reasoning process. It is necessary to obtain the attribute information of the object to be loaded.

[0167] For example, attribute information for each object can be obtained through sensor scanning, database queries, or manual data entry. This attribute information can then be encoded into feature vectors, and a mapping relationship between object identifiers and attribute features can be established.

[0168] In operation S320, based on the attribute information of multiple objects to be loaded, the sequence generation model is used to determine each object in the predicted loading sequence.

[0169] In the embodiments of this disclosure, the prepared attribute information is input into a trained sequence generation model, and the model outputs the complete predicted loading sequence step by step in an autoregressive manner.

[0170] For example, the model initializes from an empty sequence. At each step, the model calculates the probability distribution of the next object based on the currently generated sequence portion and all object attributes. The object with the highest probability is selected and added to the sequence using a greedy strategy or sampling. This process is repeated until all objects are sorted, resulting in a complete prediction loading sequence.

[0171] Through the embodiments of this disclosure, the trained sequence generation model is applied to actual bin packing reasoning, enabling the rapid generation of high-quality loading sequences. The method of this embodiment achieves real-time and intelligent bin packing decisions, significantly improving bin packing efficiency.

[0172] Figure 4 This is a schematic diagram of the structure of an electronic device according to an embodiment of the present disclosure.

[0173] In this embodiment of the disclosure, the electronic device 400 includes: at least one memory 410, a computer program stored on the memory 410; at least one processor 420; the processor 420 executes the computer program to implement the steps of the method of the foregoing embodiments.

[0174] In this embodiment of the disclosure, processor 420 can execute the bin packing method described above. Processor 420 can also execute the training method of the sequence generation model described above.

[0175] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0176] Figure 5 This is a schematic block diagram of an example electronic device used to implement embodiments of the present disclosure.

[0177] like Figure 5As shown, an electronic device 500 according to an embodiment of this application includes a processor 501, which can perform various appropriate actions and processes according to a program stored in a read-only memory 502 or a program loaded from a storage portion 508 into a random access memory 503. The processor 501 may include, for example, a general-purpose microprocessor, an instruction set processor and / or an associated chipset and / or a dedicated microprocessor. The processor 501 may also include onboard memory for caching purposes. The processor 501 may include a single processing unit or multiple processing units for executing different steps of the method flow according to an embodiment of this application.

[0178] Random access memory 503 stores various programs and data required for the operation of electronic device 500. Processor 501, read-only memory 502, and random access memory 503 are interconnected via bus 504. Processor 501 executes various steps of the method flow according to embodiments of this application by executing programs stored in read-only memory 502 and / or random access memory 503. It should be noted that programs may also be stored in one or more memories other than read-only memory 502 and random access memory 503. Processor 501 may also execute various steps of the method flow according to embodiments of this application by executing programs stored in one or more memories.

[0179] According to embodiments of this application, the electronic device 500 may further include an input / output interface 505, which is also connected to a bus 504. The electronic device 500 may also include one or more of the following components connected to the input / output interface 505: an input section 506 including a keyboard, mouse, etc.; an output section 507 including a cathode ray tube, liquid crystal display, etc., and a speaker, etc.; a storage section 508 including a hard disk, etc.; and a communication section 509 including a network interface card, such as a local area network card, modem, etc. The communication section 509 performs communication processing via a network such as the Internet. A drive 510 is also connected to the input / output interface 505 as needed. A removable medium 511, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 510 as needed so that computer programs read from it can be installed into the storage section 508 as needed.

[0180] Embodiments of this application also provide a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or it may exist independently and not assembled into the device / apparatus / system. The computer-readable storage medium carries one or more programs, which, when executed, implement the method according to the embodiments of this application.

[0181] According to embodiments of this application, the computer-readable storage medium can be a non-volatile computer-readable storage medium, such as including but not limited to: portable computer disks, hard disks, random access memory, read-only memory, erasable programmable read-only memory, portable compact disk read-only memory, optical storage devices, magnetic storage devices, or any suitable combination thereof. In embodiments of this application, the computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. For example, according to embodiments of this application, the computer-readable storage medium may include the read-only memory 502, and / or random access memory 503, and / or one or more memories other than read-only memory 502 and random access memory 503 described above.

[0182] Embodiments of this application also include a computer program product comprising a computer program containing program code for performing the methods shown in the flowchart. When the computer program product is run on a computer system, the program code is used to cause the computer system to implement the methods provided in the embodiments of this application.

[0183] In one embodiment, the computer program may rely on a tangible storage medium such as an optical storage device or a magnetic storage device. In another embodiment, the computer program may also be transmitted and distributed in the form of signals over a network medium, and may be downloaded and installed via the communication section 509, and / or installed from a removable medium 511. The program code contained in the computer program can be transmitted using any suitable network medium, including but not limited to: wireless, wired, etc., or any suitable combination thereof.

[0184] In embodiments of this application, the computer program can be downloaded and installed from a network via communication section 509, and / or installed from removable medium 511. When the computer program is executed by processor 501, it performs the functions defined in the system of this application embodiment. According to embodiments of this application, the systems, devices, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0185] According to embodiments of this application, program code for executing the computer programs provided in the embodiments of this application can be written in any combination of one or more programming languages. Specifically, these computational programs can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. The program code can be executed entirely on the user's computing device, partially on the user's device, partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0186] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0187] Those skilled in the art will understand that the features described in the various embodiments of this application can be combined and / or combined in various ways, even if such combinations or combinations are not explicitly described in this application. In particular, the features described in the various embodiments of this application can be combined and / or combined in various ways without departing from the spirit and teachings of this application. All such combinations and / or combinations fall within the scope of this application.

Claims

1. A packing method, comprising: Obtain attribute information of multiple objects to be loaded, including volume information and weight information; Based on the attribute information of the multiple objects to be loaded, each object in the predicted loading sequence is determined step by step through the sequence generation model. After determining the k-th object, the current loading sequence is obtained, where k is a positive integer less than the number of multiple objects to be loaded. Based on the current loading sequence, determine the intermediate evaluation signal; The intermediate evaluation signal indicates the volume utilization rate of the current loading sequence and the volume penalty determined based on the volume-weight relationship of the current loading sequence; The generation strategy of the sequence generation model is adjusted based on the intermediate evaluation signal to improve volume utilization and reduce volume penalty in subsequent sequence generation processes. Based on the sequence generation model, the k+1th object in the predicted loading sequence is determined through the adjusted generation strategy until the complete predicted loading sequence is obtained.

2. The method according to claim 1, wherein adjusting the generation strategy of the sequence generation model based on the intermediate evaluation signal comprises: If the volume utilization rate is less than a first threshold and the volume penalty is greater than or equal to a second threshold, the sequence generation model reduces the consistency of sampling in subsequent sequence generation processes. When the volume utilization rate is greater than or equal to the third threshold and the volume penalty is less than the fourth threshold, the sequence generation model improves the consistency of sampling in subsequent sequence generation processes. Wherein, the first threshold is less than or equal to the third threshold, and the second threshold is greater than or equal to the fourth threshold.

3. The method according to claim 1, wherein determining the intermediate evaluation signal based on the current loading sequence comprises: Based on the current loading sequence, a packing simulation is performed to determine the number of loading units consumed by the current loading sequence in the simulated packing state, as well as the loading weight and loading volume of each loading unit; The volume utilization rate is determined based on the number of loading units consumed and the volume information of objects in the current loading sequence. The volume penalty is determined based on the loading weight and loading volume of each loading unit.

4. A training method for a sequence generation model, comprising: Acquire training data, which includes attribute information of multiple objects to be loaded and corresponding target loading sequences; Based on the attribute information, each object in the predicted loading sequence is determined step by step through the sequence generation model. After determining the k-th object, the current loading sequence is obtained, where k is a positive integer less than the number of the multiple objects to be loaded. An intermediate reward signal is determined based on the current loading sequence and the target loading sequence; the intermediate reward signal represents the consistency between the current loading sequence and the target loading sequence. The parameters of the sequence generation model are adjusted based on the intermediate reward signal to improve the consistency between the subsequently generated predicted loading sequence and the target loading sequence; the adjusted sequence generation model is used to continue to determine the (k+1)th object in the predicted loading sequence.

5. The method according to claim 4, wherein determining the intermediate reward signal based on the current loading sequence and the target loading sequence comprises: Extract objects that are common to the current loading sequence from the target loading sequence, while maintaining the relative order of the objects in the target loading sequence, to obtain the target comparison sequence; The intermediate reward signal is determined based on the degree of difference in the relative order between the current loading sequence and the target comparison sequence.

6. The method according to claim 5, wherein determining the intermediate reward signal based on the degree of difference in relative order between the currently loaded sequence and the target comparison sequence comprises: Based on the current loading sequence and the target comparison sequence, at least one basic sequence distance is determined, the basic sequence distance characterizing the degree of difference in the relative order of objects in the current loading sequence and the target comparison sequence; The current sequence distance is obtained based on at least one basic sequence distance; The intermediate reward signal is determined based on the current sequence distance.

7. The method according to claim 6, wherein the at least one basic sequence distance comprises: The first sequence distance is determined based on the number of object pairs whose relative order is inconsistent in the current loading sequence and the target comparison sequence; The second sequence distance is determined based on the sum of the absolute values ​​of the position differences of each object in the current loading sequence and the target comparison sequence; The third sequence distance is determined by measuring the relative order of objects in the current loading sequence and the target comparison sequence and calculating the number of operations required to make them consistent. The fourth sequence distance is determined based on the sum of squares of the position differences of each object in the current loading sequence and the target comparison sequence; The fifth sequence distance is determined based on the length of the longest common subsequence of the currently loaded sequence and the target comparison sequence.

8. The method according to claim 4, after completing the iterative adjustment of the sequence generation model to obtain the complete predicted loading sequence, the method further includes: Based on the complete predicted loading sequence, a packing simulation is performed to obtain the simulated packing results; Based on the simulated packing results, the final reward signal is determined; Based on the final reward signal and the intermediate reward signals obtained during the generation of the predicted loading sequence, the parameters of the sequence generation model are adjusted; specifically including: Based on the intermediate reward signal and the final reward signal, the long-term value of each step in the sequence generation process is evaluated to obtain the value evaluation result. Calculate the policy gradient of the output result of the sequence generation model at each step in the sequence generation process; The policy gradient is scaled based on the value assessment results, and the parameters of the sequence generation model are adjusted accordingly.

9. A bin packing method, comprising bin packing inference using a sequence generation model, wherein the sequence generation model is trained according to any one of claims 4-8, the bin packing method comprising: Obtain attribute information for multiple objects to be loaded; Based on the attribute information of the multiple objects to be loaded, the sequence generation model determines each object in the predicted loading sequence.

10. An electronic device, the electronic device comprising: At least one memory, and a computer program stored in the memory; At least one processor; The processor executes the computer program to implement the steps of the method as described in any one of claims 1-9.

Images