Container positioning screening method and system, storage medium and electronic equipment

Abstract

The application provides a container positioning screening method and system, a storage medium and an electronic device. The method constructs a 3D yard model of a container, iteratively trains the 3D yard model and a deep learning model, receives a loading and unloading instruction of a container to be positioned, determines an optimal positioning and an operation path of the container to be positioned according to the trained deep learning model, and drives a work mechanism to move the container to be positioned to the optimal positioning according to the operation path. The container positioning screening method fully considers the stacking condition of the container in the current yard, and trains the deep learning model by evaluating key indicators, so that the determined positioning and operation path are accurate and have a higher optimization effect on the actual container turnover rate, and the method has the feasibility of practical production application.

Description

Technical Field

[0001] This invention relates to the field of port terminal yard management technology, specifically to a container placement and screening method, system, storage medium, and electronic equipment. Background Technology

[0002] Container shipping, as the most common mode of maritime transport, relies on infrastructure such as port terminals, yards, and warehouses to achieve efficient operations. In recent years, the throughput of port terminals has experienced explosive growth. Faced with increasing pressure on yard management, the problems of low operational efficiency and high operating costs resulting from haphazard loading and unloading plans and scheduling decisions lacking data support have become increasingly prominent. The most representative problem is the persistently high turnover rate of container yards, which brings many inconveniences and high costs to both yard operators and transport drivers involved in the operations.

[0003] The core issue of container overturning is the mismatch between the container yard's stacking status and the actual container pickup sequence. Since it's difficult to predict the subsequent pickup sequence when containers enter the yard, yard planners can only rely on experience-based rules for stacking, inevitably leading to container overturning during pickup. Furthermore, if inappropriate obstruction points are chosen during overturning, it can further cause secondary or even more overturning in subsequent pickups. Therefore, optimizing the stacking and overturning placement is a crucial way to reduce container overturning in container yards and remains a challenging research area.

[0004] Currently, the main technical methods used to solve this problem fall into the following four categories:

[0005] (1) Optimal placement method for container turning based on set strategies. The formulation of strategies for this type of method is mainly based on the experience rules of the planning and scheduling personnel in the yard. The optimization strategies mainly include: the closest stacking strategy in the same yard, the lowest stacking strategy in the same yard, the highest stacking strategy in the same yard, stacking with the same bill of lading and combinations of strategies, etc.

[0006] (2) Optimal container placement method based on recursion or branch and bound. This type of method is usually based on the known container picking order in the bay, and uses the idea of ​​recursion or branch and bound to calculate the total number of container flips at all possible placement positions in the bay, and selects the placement position with the minimum total number of container flips.

[0007] (3) Heuristic-based method for optimal container placement. This type of method uses the container distribution information and container picking order information within the container bay to construct the expected container handling volume (approximate value) for each candidate container bay, and selects the container placement location based on the principle of minimizing the expected container handling volume.

[0008] The above methods have the following drawbacks:

[0009] The problem scale of the above-mentioned optimization methods is all single-bay problem scale, which lacks practical significance compared to the problem scale of actual production operations. Most of the methods assume that the container picking order is determined and do not consider the situation of containers being stacked during loading and unloading, that is, they do not consider the stacking location, which is inconsistent with actual operations. Since the problem scale only considers a single bay and does not include the movement distance of the operating mechanism equipment in the optimization objective, considering only the container turning volume is not rigorous enough. The status of most yards and the attribute status of containers are simplified, such as considering standard containers for container types, which is inconsistent with the actual situation of yards and containers and lacks practical application scenarios. Summary of the Invention

[0010] Therefore, a technical solution for container placement and screening is needed to address the shortcomings of current container placement and screening strategies that rely on a predetermined container pick-up order, resulting in inaccurate placement screening and poor applicability in various scenarios.

[0011] To achieve the above objectives, in a first aspect, the present invention provides a container placement screening method based on deep learning, the method comprising the following steps:

[0012] S1: Construct a 3D container yard model;

[0013] S2: Obtain historical yard data and historical operation data of the container yard, and determine the parameter information in the deep learning model based on the historical yard data and historical operation data;

[0014] S3: Combine the 3D stockpile model and the deep learning model for iterative training, compare the evaluation indicators of the training process with the reference operation data, and when the error between the two is within the preset error range, the deep learning model training is completed.

[0015] S4: Receive the loading and unloading instructions for the container to be placed, determine the optimal placement position and operation path of the container to be placed according to the deep learning model trained in step S3, and drive the operating mechanism to move the container to be placed to the optimal placement position according to the operation path.

[0016] Further, step S1 includes:

[0017] S11: Determine the size of the container area;

[0018] S12: Render each container currently stacked in the yard based on the container data of the yard; the container data includes the stacking position size and the container size;

[0019] S13: Based on the container stacking rules, determine the set of permitted stacking or unstacking locations within the current yard;

[0020] S14: Statistically analyze the key indicators of each yard and display them in the 3D yard model; the key indicators include any one or more of the following: yard utilization rate, number of containers currently entering and leaving the yard, total number of containers turned over, number of containers turned over per container, and distance of the operating mechanism from the yard position.

[0021] Furthermore, the parameter information includes yard state characteristics, action state characteristics, and reward function;

[0022] The yard status characteristics are determined based on the yard's attribute characteristics and the container's attribute characteristics;

[0023] The action state feature is the set of operation actions required to place the current container in a certain pile position in the yard;

[0024] The reward function is used to calculate the reward score, which is determined based on minimizing the amount of box turning, the amount of secondary box turning, and the movement distance of the operating machinery.

[0025] Furthermore, the evaluation indicators include one or more of the following: the total reward points for each iteration, the number of boxes flipped, the number of boxes flipped twice, and the distance of movement of the work structure.

[0026] Further, step S3 includes:

[0027] S31: Initialize the state space S at time t0, which includes setting the current state of the container yard and the container operation to be performed, and setting the maximum number of erroneous operations allowed per step;

[0028] S32: Based on the environmental state s at time t t Select an optional action a in action space A. t Determine if the action meets the business validation rules. If not, save the current state and record the reward r(s). t ,a t The single-step error count is incremented by 1, and the next action is selected for evaluation until the single-step error count reaches the maximum allowed error count, at which point the current iteration ends. If the currently selected action matches the container status of the upcoming operation and meets the business validation rules, the reward r(s) is recorded. t ,a t) Enter time t+1 and update the environment state to S. t+1 ;

[0029] S33: By setting different number of iterations or model parameters, the model is trained, key points in the training process are statistically analyzed, and the key evaluation indicators are compared with reference job data. When the error between the two is within the preset error range, the deep learning model training is completed.

[0030] Furthermore, the method also includes:

[0031] The deep learning model trained in step S3 is put into the planning and scheduling decision module of the terminal operating system. The planning and scheduling decision module is used to receive loading and unloading instructions for containers to be placed, and output the optimal stacking and container flipping placement through the trained deep learning model.

[0032] Furthermore, the method also includes:

[0033] When the current box-flipping rate is determined to be higher than the preset value, the optimal stacking and box-flipping positions output by the model are recorded, and the recorded data is re-inputted into the trained model for optimization and iteration.

[0034] In a second aspect, the present invention also provides a storage medium storing a computer program that, when executed, implements the method described in the first aspect of the present invention.

[0035] In a third aspect, the present invention also provides a server, comprising:

[0036] The storage medium is the storage medium described in the second aspect of the present invention;

[0037] A processor, electrically connected to the storage medium, is configured to execute a computer program stored in the storage medium to implement the method as described in the first aspect of the invention.

[0038] In a fourth aspect, the present invention provides a container placement and sorting system based on deep learning, comprising:

[0039] The electronic device is the electronic device as described in the third aspect of the present invention;

[0040] The operating mechanism is used to receive control commands from the electronic equipment and move the container to be placed to the optimal placement position according to the control commands.

[0041] Unlike existing technologies, this invention has the following characteristics:

[0042] This invention provides a container placement and screening method, system, storage medium, and electronic device based on deep reinforcement learning. The method constructs a 3D container yard model, iteratively trains the 3D yard model and the deep learning model, receives loading and unloading instructions from containers to be placed, determines the optimal placement and operation path of the container based on the trained deep learning model, and drives the operating mechanism to move the container to the optimal placement location according to the operation path. This container placement and screening method fully considers the current container stacking situation in the yard and trains the deep learning model by evaluating key indicators, ensuring accurate determination of placement and operation paths. It achieves a higher optimization effect than the actual container turnover rate and is feasible for practical production applications. Attached Figure Description

[0043] Figure 1 This is a flowchart of a container placement and screening method based on deep reinforcement learning according to the first embodiment of the present invention;

[0044] Figure 2 This is a flowchart of a container placement and screening method based on deep reinforcement learning according to the second embodiment of the present invention;

[0045] Figure 3 This is a flowchart of a container placement screening method based on deep reinforcement learning according to the third embodiment of the present invention;

[0046] Figure 4 A schematic diagram of the Markov decision process for optimal placement of the stack and the container.

[0047] Figure 5 This is a schematic diagram of the state space;

[0048] Figure 6 A schematic diagram showing the possible placement options for container repositioning in container yards;

[0049] Figure 7 A flowchart for model training iterations;

[0050] Figure 8 This is a diagram illustrating the initialization state of container stacking within a container container according to an embodiment of the present invention.

[0051] Figure 9 This is a graph showing the decrease in the amount of box turning as a function of iterations according to an embodiment of the present invention;

[0052] Figure 10 This is a graph showing the decrease in the movement distance of the vehicle as a function of iterations according to an embodiment of the present invention;

[0053] Figure 11 This is a diagram showing the container stacking status within the initialization area according to another embodiment of the present invention;

[0054] Figure 12 This is a graph showing the decrease in the amount of boxes turned over with iterations according to another embodiment of the present invention;

[0055] Figure 13 This is a graph showing the decrease in the amount of material flipped during the second iteration according to another embodiment of the present invention;

[0056] Figure 14 This is a graph showing the decrease in the distance traveled by the vehicle as a function of iterations, according to another embodiment of the present invention.

[0057] Figure 15 This is a graph showing the decrease in the vehicle's movement distance with iteration according to another embodiment of the present invention;

[0058] Figure 16 This is a schematic diagram of an electronic device according to an embodiment of the present invention;

[0059] Figure 17 This is a schematic diagram of a container placement and screening system based on deep reinforcement learning according to an embodiment of the present invention. Detailed Implementation

[0060] To illustrate the possible application scenarios, technical principles, implementable specific solutions, and achievable objectives and effects of this application in detail, the following description, in conjunction with the listed specific embodiments and accompanying drawings, provides a detailed explanation. The embodiments described herein are merely illustrative of the technical solutions of this application and are therefore intended to limit the scope of protection of this application.

[0061] In this document, the term "embodiment" means that a specific feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The term "embodiment" appearing in various places throughout the specification does not necessarily refer to the same embodiment, nor does it specifically limit its independence or connection with other embodiments. In principle, in this application, as long as there are no technical contradictions or conflicts, the technical features mentioned in each embodiment can be combined in any way to form corresponding implementable technical solutions.

[0062] Unless otherwise defined, the technical terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains; the use of related terms herein is merely for the purpose of describing particular embodiments and is not intended to limit this application.

[0063] In the description of this application, the term "and / or" is used to describe the logical relationship between objects, indicating that three relationships can exist. For example, A and / or B means: A exists, B exists, and A and B exist simultaneously. Additionally, the character " / " in this document generally indicates that the preceding and following objects have an "or" logical relationship.

[0064] In this application, terms such as “first” and “second” are used only to distinguish one entity or operation from another, and do not necessarily require or imply any actual quantity, hierarchy or order relationship between these entities or operations.

[0065] Unless otherwise specified, the use of terms such as “comprising,” “including,” “having,” or other similar expressions in this application is intended to cover non-exclusive inclusion, which does not exclude the presence of additional elements in a process, method, or product that includes elements, such that a process, method, or product that includes a series of elements may include not only those defined elements but also other elements not expressly listed, or elements inherent to such process, method, or product.

[0066] Similar to the understanding in the Examination Guidelines, in this application, expressions such as "greater than," "less than," and "exceeding" are understood to exclude the stated number; expressions such as "above," "below," and "within" are understood to include the stated number. Furthermore, in the description of the embodiments in this application, "multiple" means two or more (including two), and similar expressions related to "multiple" are also understood in this way, such as "multiple groups" and "multiple times," unless otherwise explicitly specified.

[0067] For ease of subsequent explanation, some terms involved in this invention are defined below:

[0068] Deep learning: The deep learning involved in this application preferably refers to deep reinforcement learning. Deep reinforcement learning is an artificial intelligence method that combines the perception ability of deep learning with the decision-making ability of reinforcement learning. It consists of an agent and a training environment. The agent takes actions based on the observed state of the environment and receives rewards. The goal of the agent is to discover the optimal policy in each discrete state to maximize the expected discounted reward. Deep reinforcement learning solves the problems of traditional reinforcement learning (such as Q-learning) in being unable to describe complex state spaces and the poor generalization ability of the model to new states.

[0069] rummaging through boxes: such as Figure 6 As shown, when retrieving a container, another container must be moved first. The container to be retrieved is called the target container, and the container that must be moved is called the obstruction container. The operation of moving the obstruction container is called container flipping.

[0070] Secondary box flipping: Because the box flipping operation has already created an obstruction box that was flipped once, and then the box flipping operation is performed again because it obstructs the retrieval of other target boxes, it is recorded as a secondary box flipping.

[0071] Box turning rate: As shown in the formula below, the box turning rate is the ratio of the number of boxes turned to the number of boxes picked up.

[0072]

[0073] Where R r P represents the rate of box turnover. n Number of boxes carried, P r This represents the number of times the box has been searched.

[0074] Container yard stacking and repositioning optimization issues: Container stacking optimization is the issue of optimizing the placement of containers based on the current state of the yard when they enter the yard; repositioning optimization refers to the issue of optimizing the placement of obstructing containers when retrieving containers that are not on the top layer in a multi-layered container yard.

[0075] like Figure 1 As shown, this invention provides a container placement screening method based on deep learning, the method comprising the following steps:

[0076] S1: Construct a 3D container yard model;

[0077] S2: Obtain historical yard data and historical operation data of the container yard, and determine the parameter information in the deep learning model based on the historical yard data and historical operation data;

[0078] S3: Combine the 3D stockpile model and the deep learning model for iterative training, compare the evaluation indicators of the training process with the reference operation data, and when the error between the two is within the preset error range, the deep learning model training is completed.

[0079] S4: Receive the loading and unloading instructions for the container to be placed, determine the optimal placement position and operation path of the container to be placed according to the deep learning model trained in step S3, and drive the operating mechanism to move the container to be placed to the optimal placement position according to the operation path.

[0080] The above scheme fully considers the current container stacking situation in the yard when screening container placement, and trains the deep learning model by evaluating key indicators, which can make the determined placement and operation path accurate and have a higher optimization effect than the actual container turnover rate, and has the feasibility of practical production application.

[0081] like Figure 2 As shown, step S1 includes:

[0082] S11: Determine the size of the container area;

[0083] S12: Render each container currently stacked in the yard based on the container data of the yard; the container data includes the stacking position size and the container size;

[0084] S13: Based on the container stacking rules, determine the set of permitted stacking or unstacking locations within the current yard;

[0085] S14: Statistically analyze the key indicators of each storage yard and display them in the 3D storage yard model.

[0086] In step S11, assuming the size of the container area is 15×16×4, it means that the container area has a total of 15 rows, 16 bays, and a floor height limit of 4 layers.

[0087] In step S13, the stacking rules include that loaded containers and empty containers cannot be stacked together, that import and export containers cannot be stacked together when the yard utilization rate is less than 80%, and that containers of different sizes cannot be stacked together. By setting the stacking rules, the set of allowed stacking or container flipping locations can be determined each time.

[0088] In step S14, the key indicators include one or more of the following: yard utilization rate, number of containers currently entering and leaving the yard, total number of containers turned over, number of containers turned over per container, and distance of the operating mechanism moving to the stack position.

[0089] Through steps S11-S14, the current status of containers stored in the yard and the attribute characteristics of each storage location can be displayed through the model, which facilitates subsequent calculation and processing.

[0090] In some embodiments, step S2 further includes the steps of: determining a Markov decision process model for the optimal container repositioning problem and collecting historical yard data and historical operation data of the container yard, and determining parameter information in the deep learning model based on the historical yard data and historical operation data.

[0091] Preferably, the parameter information includes yard status characteristics, action status characteristics, and a reward function; the yard status characteristics are determined based on the attribute characteristics of the yard and the attribute characteristics of the container; the action status characteristics are the set of operation actions required to place the current container in a certain pile position in the yard; the reward function is used to calculate the reward score, and the reward score is determined based on minimizing the container turning amount, the secondary container turning amount, and the movement distance of the operating machinery.

[0092] Furthermore, the specific implementation of step S2 is as follows:

[0093] S21: First, the relevant symbols are defined as follows:

[0094] B, R, LT: represent the total number of bays, the total number of rows, and the maximum height limit for storage in the storage yard, respectively. For example, for a 4×4×4 storage yard space, B=4, R=4, LT=4;

[0095] C i : Represents the i-th container waiting to be processed, i = {1, 2, 3, ..., Nc};

[0096] C ij : Represents the operation type of the i-th container waiting for operation, j = {1, 2, 3}, where j = 1 represents stacking operation, j = 2 represents departure operation, and j = 3 represents container flipping operation within the same container;

[0097] C k : Represents containers in the yard, k = {1, 2, 3, ..., R × B × LT};

[0098] C kb : Represents the bay position of the k-th container in the yard, 1≤C kb For a simple B problem, Ckb ≤ 2 × B⁻¹, Ckb ≡ 2.

[0099] C kr : Represents the row number of the k-th container in the yard, 1≤C kr ≤R;

[0100] C kt : Represents the height of the k-th container in the yard, 1≤C kr ≤LT;

[0101] DL Ci : Represents the distance the trolley travels after the i-th container operation is completed;

[0102] DSCi: Represents the distance the trolley moves after the container operation of the i-th operation is completed;

[0103] M Ci : Represents the amount of container turnover generated by the i-th container waiting for operation;

[0104] H Ci : Represents how many containers are currently being stacked, i = {1, 2, 3};

[0105] C k : Represents containers in the yard, k = {1, 2, 3, ..., R × B × LT};

[0106] C km : Represents the number of times the k-th container in the yard has been turned over. If Ckm is greater than 1, it means that the container has been turned over two or more times.

[0107] LD Cior LD Ck This indicates how many days remain until the predicted storage days for the container in operation or on-site.

[0108] RD Ci or RD Ck : Represents the number of days since the operation or container in the yard has been stored;

[0109] PD Ci or PD Ck : Represents the predicted number of days of container storage in the yard or during operations;

[0110] s0, s t s i : Represent the initial state, final state, and current state of the environment, respectively, t≤Nc;

[0111] a0, a t a i : These represent the actions chosen in the initial state, the final state, and the current state, respectively, where t≤Nc;

[0112] S22: According to the definition of the Markov property, when a stochastic process is given a present state and all past states, the conditional probability distribution of its future state depends only on the current state. Since the current state of the container yard is conditionally independent of its past states (i.e., the historical path of the process), the operation process of the container yard can be considered to possess the Markov property. This process is a Markov decision (S, A, {Psa}, {Rsa}, V) process. A schematic diagram of the Markov decision process is attached. Figure 4 As shown.

[0113] S23: Deep reinforcement learning state space features (i.e., yard state features) and action state features. The selection of state space features can be based on features related to the container pickup sequence. For example, based on certain container pickup sequence-related features, the following table 1 and appendix were determined. Figure 5 The state-space characteristics shown in Figure 2 and the action-space characteristics shown in Table 2 are as follows:

[0114]

[0115]

[0116] Table 1. Spatial Characteristics of the Stockyard

[0117] The determination of actions within the action space characteristics is mainly related to the type of operation and the specifications of the storage yard, as shown in the appendix. Figure 6The diagram illustrates a 4×4×4 yard space, consisting of 4 rows, 4 bays, and 4 tiers with a height limit. When all yard stack heights are below the maximum height limit, considering the container type, enumerating the placement options for a single container provides the maximum number of choices: 2×4×4 = 32 placement positions. Further considering the types of operational actions, such as stacking, moving, and leaving, the total number of selectable actions reaches 3×32 = 96. The specific selection of action space features is as follows:

[0118] Container-related characteristics Range of values The container stacking belongs to the shell [1,2×B-1] The stack to which the container is placed [1,R] Operation type 0,1,2,3

[0119] Table 2 Characteristics of Action Space

[0120] In this embodiment, the evaluation index includes one or more of the following: the total reward points for each iteration, the amount of box flipping, the amount of secondary box flipping, and the movement distance of the working structure. Therefore, after step S23, step S24 can proceed: designing a reward function based on minimizing the amount of box flipping, the amount of secondary box flipping, and the movement distance of the working machinery. (In state s) t In this case, choose action a. t Then, the reward for environmental feedback is represented by r(s). t ,a t This indicates that the score reflects the quality of the current action.

[0121] The single-step reward is the sum of all rule rewards:

[0122] r(s t a t )=∑reward

[0123] The rules for obtaining environmental rewards are as follows, and can be any one or more of the following nine rules:

[0124] (1) Based on the Manhattan distance of the vehicle's movement, the process is divided into two stages, with the reward and penalty scores calculated at the quarter-point of the B / s ratio, as shown in formula (1):

[0125]

[0126] (2) Based on the Manhattan distance of the car's movement, the process is divided into two stages. The reward and penalty scores are calculated at the halfway point of the row count, as shown in formula (2):

[0127]

[0128] (3) The reward and penalty score is calculated based on the number of times the box is searched, as shown in formula (3):

[0129]

[0130] (4) Calculate the reward and penalty score based on the number of stacked cabinets. For example, in gantry crane mechanical operation, the maximum stacking is 4 layers, so the maximum number of stacked cabinets is 3. The fewer the stacked cabinets, the higher the score, as shown in formula (4):

[0131]

[0132] (5) Calculate the reward / penalty score based on the remaining days of the predicted storage period, as shown in formula (5):

[0133]

[0134] (6) Calculate the reward and penalty scores based on the actual number of days the pressurized cabinets are stored, as shown in formula (6):

[0135]

[0136] (7) Calculate the score based on the import / export and mixed storage of domestic and foreign trade containers. If there is no mixed storage, add 6 points (different values ​​can be set for different application scenarios); otherwise, deduct 6 points (different values ​​can be set for different application scenarios). For example, if domestic and foreign trade containers are stacked in the same building in the yard, the reward function will deduct points.

[0137] (8) Depending on whether the container has been booked, bound, processed, detained, inspected, or designated, add 3 points if it has not been booked (different values ​​can be set for different application scenarios), otherwise deduct 3 points (different values ​​can be set for different application scenarios).

[0138] (9) Completing the operation of container entry and exit records adds 40 points (different values ​​can be set for different application scenarios), and completing all operation records adds 1000 points (different values ​​can be set for different application scenarios). For the problem scale of the entire container area, the overall reward is relatively sparse. Increasing the reward for single-step operation and the reward for complete operation can prevent the agent from getting trapped in local optima.

[0139] In some embodiments, such as Figure 3 As shown, step S3 includes:

[0140] S31: Initialize the state space S at time t0, which includes setting the current state of the container yard and the container operation to be performed, and setting the maximum number of erroneous operations allowed per step;

[0141] S32: Based on the environmental state s at time t t Select an optional action a in action space A. t Determine if the action meets the business validation rules. If not, save the current state and record the reward r(s). t ,a tThe single-step error count is incremented by 1, and the next action is selected for evaluation until the single-step error count reaches the maximum allowed error count, at which point the current iteration ends. If the currently selected action matches the container status of the upcoming operation and meets the business validation rules, the reward r(s) is recorded. t ,a t) Enter time t+1 and update the environment state to S. t+1 ;

[0142] S33: By setting different number of iterations or model parameters, the model is trained, key points in the training process are statistically analyzed, and the key evaluation indicators are compared with reference job data. When the error between the two is within the preset error range, the deep learning model training is completed.

[0143] During model training in step S33, key reward data and business data are statistically analyzed. The key reward data includes the reward data for a single iteration, and the business data includes specific business metrics such as the number of boxes flipped, the distance traveled by the large vehicle, and the distance traveled by the small vehicle. Statistical analysis of the key reward data and business data facilitates the subsequent plotting of the reward function and business metric curves.

[0144] Furthermore, the deep learning model trained in step S3 is placed into the planning and scheduling decision module of the terminal operating system. The planning and scheduling decision module is used to receive loading and unloading instructions for containers to be placed, and output the optimal stacking and container flipping placement through the trained deep learning model.

[0145] Preferably, the method further includes: when it is determined that the current box-flipping rate is higher than a preset value, recording the optimal stacking and box-flipping positions output by the model, and re-inputting the recorded data into the trained model for optimization iteration.

[0146] The technical solution designed by this invention can achieve the following technical effects: by modeling the actual production operations of container yards, considering the complete container area, the presence of inbound and outbound containers, and various factors such as container size, it has the potential to be implemented in actual production operations; based on deep reinforcement learning algorithms, it solves the limitations of existing algorithms in solving large-scale problems; it proposes multi-objective optimization by combining container turning and mechanical movement distance, which is more practical; and by comparing actual production operation decisions, the container turning rate is optimized by a large proportion.

[0147] In a second aspect, the present invention also provides a storage medium storing a computer program that, when executed, implements the method described in the first aspect of the present invention.

[0148] In the third aspect, such as Figure 16As shown, the present invention also provides an electronic device 10, including a storage medium 102 and a processor 101, wherein the storage medium 102 is the storage medium as described in the second aspect; the processor 101 is electrically connected to the storage medium 102 and is used to execute a computer program stored in the storage medium to implement the method as described in the first aspect.

[0149] In the fourth aspect, such as Figure 17 As shown, the present invention also provides a deep learning-based container placement and screening system, including an electronic device 10 and a working mechanism 20. The working mechanism 20 is used to receive control commands from the electronic device and move the container to be placed to the optimal placement position according to the control commands.

[0150] Preferably, the electronic device can be a computer device, including but not limited to: personal computers, servers, general-purpose computers, special-purpose computers, network devices, embedded devices, programmable devices, smart mobile terminals, smart home devices, wearable smart devices, and in-vehicle smart devices. Storage media include but are not limited to: RAM, ROM, magnetic disks, magnetic tapes, optical discs, flash memory, USB flash drives, portable hard drives, memory cards, memory sticks, network server storage, and network cloud storage. Processors include but are not limited to CPUs (Central Processing Units), GPUs (Graphics Processing Units), and MCUs (Microprocessors).

[0151] The training process of the agent in deep reinforcement learning of this application will be described in detail below with reference to two embodiments.

[0152] Example 1: Solving the size problem of a single Bayesian problem

[0153] The technical solution of this application can solve not only the problem of the scale of the entire container area, but also the problem of the scale of a single bay. In this embodiment, the actual stacking and departure operation records of the 01 container area and 01 bay of a Class II terminal in a coastal area within one week are selected, totaling 68 records, of which 34 are departure records and 34 are stacking records, with a distribution of 1:1. Since actual operations can flip containers across bays, but the scale of the single bay problem does not allow obstructing containers to be flipped outside the bay, the optimization effect of this example cannot be compared with actual production data. It can be compared with the commonly used formula for predicting the total number of containers flipped (without considering the stacking of containers):

[0154]

[0155] In the above formula, 'a' represents the number of container stacks, which is equal to the number of rows in a single bay. 'c' represents the rated stacking height of the containers, also known as the maximum number of tiers. For the scale of bay 01 in container area 01, which is 15 rows × 4 tiers, the estimated total container turnover can be calculated as follows:

[0156]

[0157] This embodiment focuses on the 3D model of the initial state of a single container yard, as shown below. Figure 8 The image shows a 15x4 container bay, consisting of 15 rows and a height limit of 4 tiers. The bottom right corner shows the container operation requiring a decision. Figure 8 The current image shows a container departure operation.

[0158] For the above case, the following method uses deep reinforcement learning-based optimization of heap and bin-flipping placement to optimize the job decision instructions within this time period. It can be seen that as the number of iterations increases, Figure 9 The curves shown show the amount of box turning and Figure 10 The mean curves of the movement distance of the operating mechanisms (such as trolleys) shown all show a downward trend. The number of containers turned converges to about 42, and the movement distance of the trolleys converges to about 900, which is about 9% better than the predicted value. Considering that there are containers entering the stack, this is a very good optimization for a stacker yard with a utilization rate of nearly 80%.

[0159] Example 2: Solving the size problem of a complete container area

[0160] Similarly, a total of 102 operation records of actual container entry and exit from the F warehouse of a coastal terminal were selected within a week. Among them, there were 59 exit records and 43 entry records. The distribution of the two was close to 6:4. The actual number of containers turned over was 33, and the turnover rate was about 56%, which is at a relatively high level.

[0161] The 3D model of the container yard area in this embodiment is as follows: Figure 11 The image shows a container area with dimensions of 15×16×4, which is a container area with 15 rows, 16 bays, and a height limit of 4 tiers. The bottom right corner shows the container operation that requires a decision. Figure 11 The image shows a box-moving operation that obstructs the movement of a box.

[0162] For the above case, the following method uses deep reinforcement learning-based optimization of heap insertion and bin flipping placement to optimize the job decision instructions within this time period. As can be seen, Figure 12 shown, Figure 13 The amount of the second-round box shown Figure 14 The travel distance of the operating mechanism (such as the trolley) shown, and Figure 15The average curves of the movement distances of the operating mechanisms (such as trolleys) shown all show a downward trend. The number of container flippings converges to around 18, the number of secondary container flippings converges to 2.75, the movement distance of the main trolley converges to around 470, and the movement distance of the trolley converges to around 875. The number of container flippings and the flipping rate are optimized by nearly 45% compared with the actual values. For a terminal with a monthly throughput of 100,000 TEUs, every 2% reduction in the flipping rate can save RMB 600,000 in container flipping operation costs and 1,000 hours of additional machine time per year (the container flipping cost is calculated at RMB 25 / TEU, and the container flipping machine time is calculated at 2.5 minutes / TEU). Therefore, the technical solution of this application can save the terminal nearly RMB 13.5 million in container flipping costs and 22,500 hours of container flipping machine time per year.

[0163] Although the above embodiments have been described, those skilled in the art, once they understand the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the above are merely embodiments of the present invention and do not limit the scope of patent protection of the present invention. Any equivalent structural or procedural transformations made using the content of the present invention's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of the present invention.

Claims

1. A deep learning-based container positioning screening method, characterized in that, The method includes the following steps: S1: Construct a 3D container yard model; S2: Obtain historical yard data and historical operation data of the container yard, and determine the parameter information in the deep learning model based on the historical yard data and historical operation data; S3: Combine the 3D stockpile model and the deep learning model for iterative training, compare the evaluation indicators of the training process with the reference operation data, and when the error between the two is within the preset error range, the deep learning model training is completed. S4: Receive the loading and unloading instructions for the container to be placed, determine the optimal placement position and operation path of the container to be placed according to the deep learning model trained in step S3, and drive the operating mechanism to move the container to be placed to the optimal placement position according to the operation path. Step S1 includes: S11: Determine the size of the container area; S12: Render each container currently stacked in the yard based on the container data of the yard; the container data includes the stacking position size and the container size; S13: Based on the container stacking rules, determine the set of permitted stacking or unstacking locations within the current yard; S14: Statistically analyze the key indicators of each yard and display them in the 3D yard model; the key indicators include any one or more of the following: yard utilization rate, number of containers currently entering and leaving the yard, total number of containers turned over, number of containers turned over per container, and distance of the operating mechanism to the stack position; The parameter information includes yard status characteristics, action status characteristics, and reward function; The yard status characteristics are determined based on the yard's attribute characteristics and the container's attribute characteristics; The action state feature is the set of operation actions required to place the current container in a certain stack location in the yard; The reward function is used to calculate the reward score, which is determined based on minimizing the box-turning amount, the secondary box-turning amount, and the movement distance of the operating machinery. Step S3 includes: S31: Initialize the state space S at time t0, which includes setting the current state of the container yard and the container operation to be performed, and setting the maximum number of erroneous operations allowed per step; S32: select a certain optional action a in action space A according to the environment state s at t moment t t t t t t) t+1 ;​​​​​​ S33: The model is trained by setting different number of iterations or model parameters, key evaluation indicators are statistically analyzed during the training process, and the key evaluation indicators are compared with reference job data. When the error between the two is within the preset error range, the deep learning model training is completed. 2.The deep learning-based container drop-off screening method of claim 1, wherein, The evaluation indicators include one or more of the following: the total reward points for each iteration, the number of boxes flipped, the number of boxes flipped twice, and the distance of movement of the work structure. 3.The deep learning-based container drop-off screening method of claim 1, wherein, The method further includes: The deep learning model trained in step S3 is put into the planning and scheduling decision module of the terminal operating system. The planning and scheduling decision module is used to receive loading and unloading instructions for containers to be placed, and output the optimal stacking and container flipping placement through the trained deep learning model. 4.The deep learning-based container drop-off screening method of claim 3, wherein, The method further includes: When the current box-flipping rate is determined to be higher than the preset value, the optimal stacking and box-flipping positions output by the model are recorded, and the recorded data is re-inputted into the trained model for optimization and iteration.

5. A storage medium, characterized by The storage medium stores a computer program that, when executed, implements the method as described in any one of claims 1 to 4.

6. An electronic device, comprising: include: The storage medium is the storage medium as described in claim 5; A processor, electrically connected to the storage medium, is configured to execute a computer program stored in the storage medium to implement the method as described in any one of claims 1 to 4. 7.A deep learning based container positioning screening system, characterized in that, include: The electronic device is the electronic device as described in claim 6; The operating mechanism is used to receive control commands from the electronic equipment and move the container to be placed to the optimal placement position according to the control commands.

Images