A Port Scheduling Method and Device Based on AEO Digital Twin

By using a port scheduling method based on AEO digital twins and employing a multi-agent reinforcement learning algorithm to generate and verify scheduling schemes, the problem of realizing and improving the efficiency of priority passage rights for AEO enterprises in customs ports has been solved. This ensures the priority passage rights of high-credit-rating enterprises and reduces waiting time and operational risks.

CN123070533APending Publication Date: 2026-11-10NAT LNFORMATION CENT OF GACC NAT E CLEARANCE CENT OF GACC
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202611136657.X
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-07-29
Publication Date
2026-11-10

AI Technical Summary

Technical Problem

The existing physical interaction links at customs ports, such as checkpoint passage, cargo inspection, and container loading and unloading, lack a holistic perspective, cannot guarantee the priority passage rights of AEO enterprises, and lack pre-simulation and verification methods, resulting in slow response speed, high trial and error costs, and an inability to quantify and evaluate the implementation effect of priority measures.

Method used

A port scheduling method based on AEO digital twin is adopted. By acquiring real-time status data, a digital twin model is constructed and AEO regulatory rules are embedded. A multi-agent reinforcement learning algorithm is used to generate a scheduling scheme, which is then pre-run and verified in the digital twin model. After ensuring that the scheme meets the constraints, it is issued for execution and the decision parameters are updated.

Benefits of technology

This ensured the accurate implementation of AEO priority measures, reduced waiting times for AEO enterprises and cargo clearance times, improved overall port efficiency, guaranteed absolute priority passage rights for high-credit-rating enterprises, and reduced operational risks.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123070533A_ABST
    Figure CN123070533A_ABST
Patent Text Reader

Abstract

This invention provides a port scheduling method, apparatus, equipment, and readable storage medium based on AEO digital twin, relating to the field of customs port supervision technology. The method includes updating a digital twin model based on real-time status data and embedding AEO supervision rules as constraints within the model. Using the updated digital twin model as a simulation environment, a multi-agent reinforcement learning algorithm is employed to generate scheduling schemes for each means of transport and cargo. The scheduling schemes are then input into the digital twin model for pre-simulation, and the pre-simulation results are used to determine whether the schemes meet the constraints. If they do, the scheduling schemes are distributed to port physical equipment for execution. The decision parameters of the multi-agent reinforcement learning algorithm are updated based on the difference between the executed status data and the real-time status data. This invention addresses the technical problem in existing technologies where the effectiveness of AEO priority measures is not quantitatively evaluated, leading to slow response times and high trial-and-error costs.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the field of customs port supervision technology, and more specifically, to a port scheduling method and apparatus based on AEO digital twin. Background Technology

[0002] According to General Administration of Customs Order No. 282, AEO (Authorized Economic Operator) companies enjoy priority inspection and release for their goods. Currently, the physical interactions at customs ports, including checkpoint clearance, cargo inspection, and container loading / unloading, generally employ a first-come, first-served scheduling approach, which cannot guarantee priority passage for AEO companies. While some ports have introduced automated equipment, the scheduling of each stage is independent, lacking a holistic perspective and easily creating operational bottlenecks. When equipment malfunctions or vehicle congestion occur, the lack of pre-simulation and verification methods makes it difficult for dispatchers to promptly assess the effectiveness of adjustments, resulting in slow response times and high trial-and-error costs. Furthermore, existing technology does not provide quantitative evaluation of the effectiveness of AEO priority measures, failing to support continuous optimization of scheduling strategies. Summary of the Invention

[0003] The purpose of this invention is to provide a port scheduling method, apparatus, device, and readable storage medium based on AEO digital twins to improve the aforementioned problems. To achieve the above objective, the technical solution adopted by this invention is as follows: Firstly, this application provides a port scheduling method based on AEO digital twins, including: Obtain real-time status data of physical entities at the port, wherein the real-time status data includes at least the credit rating and cumulative waiting time of each means of transport and cargo; The digital twin model is updated based on the real-time status data, and AEO regulatory rules are embedded in the digital twin model as constraints. Using the updated digital twin model as the simulation environment, a multi-agent reinforcement learning algorithm is used to generate scheduling schemes for each means of transport and goods based on the credit rating and cumulative waiting time. The scheduling scheme is input into the digital twin model for pre-simulation, and the pre-simulation results are used to determine whether the scheduling scheme meets the constraints. If the conditions are met, the scheduling scheme will be sent to the port physical equipment for execution; Obtain the post-execution state data, and update the decision parameters of the multi-agent reinforcement learning algorithm based on the difference between the post-execution state data and the real-time state data.

[0004] Secondly, this application also provides a port scheduling device based on AEO digital twin, comprising: The data acquisition module is used to acquire real-time status data of physical entities at the port. The real-time status data includes at least the credit rating and cumulative waiting time of each means of transport and cargo. The model update module is used to update the digital twin model according to the real-time status data and embed AEO regulatory rules as constraints in the digital twin model. The scheduling generation module is used to generate scheduling schemes for each means of transport and goods based on the updated digital twin model as a simulation environment, according to the credit rating and cumulative waiting time, using a multi-agent reinforcement learning algorithm. The pre-simulation judgment module is used to input the scheduling scheme into the digital twin model for pre-simulation, and judge whether the scheduling scheme meets the constraints based on the pre-simulation results; The instruction issuing module is used to issue the scheduling scheme to the port physical equipment for execution when the scheduling scheme meets the constraints. The parameter update module is used to acquire the state data after execution and update the decision parameters of the multi-agent reinforcement learning algorithm based on the difference between the state data after execution and the real-time state data.

[0005] Thirdly, this application also provides a port scheduling device based on AEO digital twin, including: Memory, used to store computer programs; A processor is used to implement the steps of the port scheduling method based on AEO digital twin when executing the computer program.

[0006] Fourthly, this application also provides a readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the above-described port scheduling method based on AEO digital twin.

[0007] The beneficial effects of this invention are as follows: 1. This invention embeds AEO regulatory rules as core constraints into a digital twin model, ensuring that scheduling decisions are rigidly constrained by credit rating rules even in the simulation environment, thus fundamentally guaranteeing the accurate execution of AEO priority measures. Simultaneously, a multi-agent reinforcement learning algorithm generates scheduling schemes based on credit rating and cumulative waiting time, and through a hierarchically weighted reward function, significantly increases the waiting penalty weight for high-credit-rating enterprises compared to ordinary enterprises, thereby guaranteeing the absolute priority passage rights of high-credit-rating enterprises.

[0008] 2. After the scheduling plan is generated, it is first pre-tested and verified in the digital twin model. Only the plan that meets the AEO constraints is issued for execution, avoiding the operational risks of trial and error at the physical port. Compared with traditional solutions, this invention can significantly reduce the average waiting time at checkpoints and the average customs clearance time for AEO enterprises, while improving the overall port throughput efficiency.

[0009] Other features and advantages of the invention will be set forth in the following description, and will be apparent in part from the description, or may be learned by practicing embodiments of the invention. The objects and other advantages of the invention may be realized and obtained by means of the structures particularly pointed out in the written description, claims, and drawings. Attached Figure Description

[0010] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0011] Figure 1 This is a schematic diagram of the overall system architecture of the port scheduling method based on AEO digital twin as described in this embodiment of the invention; Figure 2 This is a schematic diagram of the port scheduling method based on AEO digital twin as described in an embodiment of the present invention; Figure 3 This is a schematic diagram of the port scheduling device based on AEO digital twin as described in an embodiment of the present invention; Figure 4 This is a schematic diagram of the port scheduling equipment based on AEO digital twin as described in an embodiment of the present invention.

[0012] Marked in the image: 800. Port scheduling equipment based on AEO digital twin; 801. Processor; 802. Memory; 803. Multimedia component; 804. I / O interface; 805. Communication component. Detailed Implementation

[0013] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0014] Example 1: This embodiment provides a port scheduling method based on AEO digital twin, which is applicable to intelligent scheduling scenarios for physical interaction operations such as checkpoint passage, cargo inspection, and container loading and unloading at customs ports.

[0015] See Figure 1 and Figure 2 , Figure 1 The overall architecture of the method in this embodiment is shown. Figure 2 The steps of this method are shown.

[0016] Specifically, the overall framework architecture includes, from bottom to top, a data access layer, a digital twin layer, a reinforcement learning layer, and a decision application layer.

[0017] The data access layer serves as the port's perception data entry point, receiving real-time data from multiple sources such as checkpoints, inspection stations, logistics systems, AEO credit databases, and IoT devices, providing dynamic and real-time input data sources for the upper-layer model.

[0018] The digital twin layer serves as the core foundation for simulation verification, constructing a full-element digital twin model of the port, embedding AEO regulatory rules into the twin constraints, and achieving millisecond-level state synchronization between the physical world and the digital twin, pre-simulation and verification of scheduling schemes, simulation and deduction of abnormal scenarios, and visualization of operational status.

[0019] The reinforcement learning layer serves as the global scheduling decision engine. Based on a multi-agent reinforcement learning algorithm with AEO hard constraints, it generates a global collaborative scheduling scheme with AEO priority as the core objective, supporting online inference and closed-loop iterative optimization.

[0020] The decision application layer serves as the exit point for scheduling execution and management, providing functions such as issuing scheduling instructions, controlling AEO priority passage, automatically handling anomalies, evaluating operational performance, and iteratively optimizing scheduling strategies.

[0021] The various layers interact with each other in real time via standardized data interfaces to exchange status and commands. The following section will combine... Figure 2 Each step of this method is explained in detail.

[0022] S1. Obtain real-time status data of the physical entities at the port, wherein the real-time status data includes at least the credit rating and cumulative waiting time of each means of transport and cargo; Specifically, in step S1, real-time data from multiple sources, including port checkpoints, inspection stations, logistics systems, AEO credit databases, and IoT devices, is accessed through the data access layer to obtain real-time status data of all physical entities at the port. These physical entities include checkpoint channels, inspection stations, containers, transport vehicles, and loading / unloading robots. In this embodiment, the transport vehicle specifically refers to a transport vehicle, and the cargo specifically refers to a container. The AEO credit rating of the transport vehicle and cargo corresponds to the credit rating coefficient of its respective enterprise. ,in, Indicates general credit. Indicates general authentication. This indicates advanced certification.

[0023] Furthermore, based on the real-time status data, the status of each port physical entity is represented in the form of real-time status data, defined as follows: ; In the formula, For the first The state vector of each port physical entity For the first The spatial location of each port physical entity For the first The speed of movement of physical entities at the port. For the first The operational status of each port physical entity, including idle, in operation, and waiting. For the first AEO credit rating of each port physical entity For the first The cumulative waiting time for each physical entity at the port.

[0024] Based on the above embodiments, this method further includes: S2. Update the digital twin model based on the real-time status data, and embed AEO regulatory rules as constraints in the digital twin model; Specifically, in step S2, before updating the digital twin model based on the real-time status data, the construction of the digital twin model is also included: S21. Construct a physical entity model, which includes real-time state data. ; Specifically, the physical entity model utilizes real-time state data. Model the entire set of physical entities at the port and assign credit rating coefficients. Embedded as an entity attribute in the physical entity model, the digital twin model can distinguish the scheduling priority of entities with different credit ratings.

[0025] S22. Construct a business process model, which includes the sequential dependencies of checkpoint passage operations, inspection operations, and loading and unloading operations; Specifically, the entire process of checkpoint access, inspection operations, and container loading and unloading is modeled using directed graphs: ; In the formula, For business process models, For a set of process nodes, This represents the set of sequential dependency edges between nodes. In this embodiment, each process node is configured with an AEO priority channel attribute, supporting high-credit enterprises to jump the queue and undergo parallel processing, breaking the fixed logic of traditional serial queuing.

[0026] S23. Construct a regulatory rule model, and set the AEO regulatory rule as a constraint condition in the regulatory rule model; Specifically, the AEO regulatory rules include the following constraints: a. Checkpoint allocation constraint: In the same checkpoint queue, the cumulative waiting time of a higher credit rating vehicle shall not exceed the cumulative waiting time of a lower credit rating vehicle; Specifically, the constraint conditions are expressed as follows: ; In the formula, These represent the cumulative waiting times for high-credit, general-credit, and general-credit transport vehicles within the same checkpoint queue. These constraints ensure that high-credit enterprises enjoy priority access at checkpoints.

[0027] b. Inspection order constraint: AEO high-level goods in the same inspection queue are opened and inspected first.

[0028] c. Resource scheduling constraints: vehicles or goods with higher credit ratings are given priority in the allocation of available operational resources; Specifically, the resource scheduling constraints ensure that available operational resources such as inspection equipment and loading and unloading machinery are prioritized for allocation to enterprises with high AEO credit ratings when conflicts occur, thus guaranteeing the implementation of AEO facilitation measures from a resource perspective.

[0029] S24. The digital twin model is composed of the physical entity model, the business process model, and the regulatory rule model; Specifically, the digital twin model is defined as follows: ; In the formula, For the digital twin model of the port, For physical entity models, For business process models, For regulatory rules models.

[0030] In this embodiment, the AEO credit rules are embedded as the core constraint into the twin base to achieve a 1:1 twin mapping between the regulatory system and the physical process, providing a simulation verification environment for priority scheduling.

[0031] Preferably, by accessing IoT sensing data in real time, millisecond-level state synchronization between the physical world and the digital twin is achieved, with the synchronization equation being: ; In the formula, for The port digital twin model after a synchronization time interval This represents the full-scale sensing data at time t. This is a state update mapping function. In this embodiment, the synchronization time interval... The time is no more than 100 milliseconds, ensuring the real-time requirements of the digital twin model and the physical world.

[0032] Based on the above embodiments, before system deployment, the multi-agent reinforcement learning algorithm is trained offline in a digital twin environment to enable it to have preliminary scheduling and decision-making capabilities. The specific steps are as follows: S25. Set corresponding waiting time weights according to the credit rating of each means of transport and cargo, where the higher the credit rating, the greater the corresponding waiting time weight value; Specifically, a three-tiered waiting time weighting system is set based on AEO credit rating. The higher the level, the greater the penalty for longer waiting time. The weighting ratio can be set according to policy requirements, for example: This indicates that the waiting time penalty for advanced certified companies is 5 times that of ordinary credit companies.

[0033] S26. Input the scheduling scheme into the digital twin model for simulation operation to generate the simulated waiting time of each means of transport and goods in the digital twin model; S27. Calculate the reward value of the scheduling scheme based on the simulated waiting time of each means of transport and cargo and its corresponding waiting time weight: ; In the formula, The total reward value of the scheduling scheme. As a priority reward item for AEO, This is a traffic efficiency incentive item. To ensure fairness, the reward items are subject to constraints. For hard constraints, there are penalties for violations. Let each reward item have a weight, and satisfy the following conditions: , For hard constraint violation penalty coefficient and This ensures the rigidity of the AEO priority rules.

[0034] Specifically, the AEO priority reward item The following calculations are made based on the simulated waiting time of the goods and its corresponding waiting time weight: ; In the formula, As a priority reward item for AEO, The waiting time weights are respectively assigned to enterprises with advanced certification, general certification, and general credit rating. For advanced certified transport vehicles and cargo entities in The sum of simulated waiting times at each moment. This is the sum of simulated waiting times for generally certified transport vehicles and cargo entities. This is the sum of simulated waiting times for general credit transport vehicles and cargo entities.

[0035] Specifically, the traffic efficiency reward item The goal is to minimize the total customs clearance time at the port. ; In the formula, This is a traffic efficiency incentive item. Total customs clearance time at the port. For the first The entire customs clearance time for goods. The total number of goods. The aforementioned traffic efficiency bonus item. This ensures overall operational efficiency and avoids a decline in overall port throughput efficiency due to simply prioritizing AEO (Authorized Economic Operator) status.

[0036] Specifically, the fairness constraint reward item The penalty is the variance of the waiting time for typical businesses: ; In the formula, To ensure fairness, the reward items are subject to constraints. This represents the variance of the waiting time for general credit enterprises. The number of general credit enterprises. For the first Waiting time for advanced certification companies The average waiting time for all general creditworthy enterprises. The fairness constraint reward item. This indicates the degree to which the waiting time for advanced certified companies deviates from the average waiting time for general credit companies. In other words, if the waiting time for advanced certified companies is significantly higher than the average for general companies... A large negative value; if the waiting time for advanced certified companies is close to the average level for ordinary companies, Close to 0.

[0037] Specifically, the penalty for violating the hard constraint The calculation formula is: ; In the formula, For hard constraints, there are penalties for violations. Credit rating The statutory maximum waiting time threshold, 、 These are the statutory maximum waiting time thresholds for general certification and advanced certification, respectively. This represents the penalty coefficient corresponding to credit rating C. The penalty term for violating the hard constraint is used to determine the penalty. High penalties will be imposed when the waiting time of high-credit enterprises exceeds the statutory threshold.

[0038] By combining the above four rewards and / or penalties, the reward function... It takes into account four dimensions: AEO priority, traffic efficiency, fairness, and constraint rigidity, and can balance multiple objectives during the training process.

[0039] S28. Adjust the policy network parameters corresponding to each agent according to the reward value. Specifically, the policy network parameters are adjusted using an AEO credit-weighted gradient descent strategy: ; In the formula, Let the policy objective function be... For policy network parameters, For the first AEO credit weights corresponding to each sample Let be the policy function, representing the state. Select action The probability, Let be the dominance function, representing the state . Select action The degree of advantage relative to the average action level in the corresponding state is used to measure the relative merit of the decision.

[0040] Through the above adjustment strategy, training samples related to high-credit enterprises occupy a higher weight in gradient updates, making the policy network pay more attention to the scheduling effect of high-credit enterprises during iterative training.

[0041] Based on the above embodiments, this method further includes: S3. Using the updated digital twin model as the simulation environment, and based on the credit rating and cumulative waiting time, a multi-agent reinforcement learning algorithm is used to generate scheduling schemes for each means of transport and goods; Specifically, step S3 includes: S31. Obtain global state information in the digital twin model, the global state information including the queue length of each checkpoint, the queue length of each inspection area, the queue length of each loading and unloading area, the credit rating distribution of transport vehicles and goods in each queue, and the operating status of each operating equipment; Specifically, the global state information consists of the state vectors of each physical entity at the port. It is assembled from parts, and is defined as follows: ; In the formula, for Global state information at any given moment. These are the queue length vectors for the checkpoint, inspection area, and loading / unloading area, respectively. Let AEO credit rating vector represent all transport vehicles and goods in the queue. This is the operating state vector for all operating equipment.

[0042] S32. Using the global state information as the input of the multi-agent reinforcement learning algorithm, the multi-agent reinforcement learning algorithm selects three types of scheduling actions from the scheduling action set. The three types of scheduling actions include channel allocation actions, inspection and sorting actions, and loading and unloading equipment scheduling actions. S33. Determine the execution order of scheduling actions corresponding to each means of transport and cargo based on the credit rating, and generate a scheduling scheme that includes the three types of scheduling actions and their execution order; Specifically, the action execution priority of AEO enterprises meets the following requirements: ; In the formula, Credit rating The entity performs the scheduling action Priority is determined by the fact that actions of higher-level certified enterprises take unconditional priority over those of lower-level enterprises, thus guaranteeing priority rights at the action level.

[0043] The scheduling actions generated by each intelligent agent are arranged in descending order of credit rating to form a complete scheduling scheme. In the scheduling scheme, the channel allocation, verification sorting, and equipment scheduling actions of advanced certified enterprises are given priority in the execution queue, followed by those of general certified enterprises, and those of general credit rating enterprises are executed last.

[0044] Based on the above embodiments, this method further includes: S4. Input the scheduling scheme into the digital twin model for pre-simulation, and determine whether the scheduling scheme meets the constraints based on the pre-simulation results; If the pre-simulation results do not meet the constraints, return to step S3 to regenerate the scheduling scheme.

[0045] Based on the above embodiments, this method further includes: S5. If satisfied, the scheduling scheme will be sent to the port physical equipment for execution; Specifically, dispatch instructions will be issued to port physical equipment such as checkpoint control systems, inspection station dispatch terminals, and loading and unloading equipment controllers for execution.

[0046] Based on the above embodiments, this method further includes: S6. Obtain the state data after execution, and update the decision parameters of the multi-agent reinforcement learning algorithm based on the difference between the state data after execution and the real-time state data; Specifically, step S6 includes: S61. Calculate the actual reward value of the scheduling scheme based on the actual customs clearance time after the physical equipment at each port executes the scheduling scheme; Specifically, the actual completion time and waiting time after execution are substituted into the reward function. In the process, the actual reward value is obtained.

[0047] S62. Combine the real-time status data, the scheduling scheme, the actual reward value, and the post-execution status data into a training sample set; Specifically, each training sample is a quadruple. ,in, For real-time status data, The dispatch plan was issued. This is the actual reward value. This is the state data after execution. Multiple training samples are combined to form an online training sample set, which is used for online updates of the policy network parameters.

[0048] S63. Set sample weights for each training sample according to its credit rating, and adjust the policy network parameters of the multi-agent reinforcement learning algorithm based on each training sample and its corresponding sample weights.

[0049] Based on the above embodiments, this method further includes: S7. Obtain real-time status data during the execution of the port physical equipment, and update the current status of the digital twin model based on the real-time status data; S8. Calculate the probability of an abnormal event occurring within a preset time window using the current state; Specifically, based on the current state of the digital twin model, future time windows are predicted using an anomaly prediction model. Probability of an abnormal event occurring within: ; In the formula, for The predicted probability of an abnormal event occurring at any given time. For anomaly prediction models, These are the parameters for the anomaly prediction model. In this embodiment, the anomaly events include equipment failure, traffic congestion, and sudden control checks.

[0050] S9. When the probability exceeds a preset alarm threshold, obtain the total amount of available job resources; Specifically, when When this occurs, the exception handling process is triggered, in which... A preset alarm threshold is set. At this time, the total amount of available operational resources at the port is obtained, including available inspection stations, loading and unloading equipment, and checkpoint channels.

[0051] S10. Calculate the amount of resources allocated to each credit level according to the resource allocation weight corresponding to each credit level, the number of vehicles and goods in each credit level, and the total amount of currently available operational resources, and allocate the allocated resources to the vehicles or goods in the corresponding credit level.

[0052] Specifically, the AEO credit-weighted resource allocation algorithm is used to calculate the amount of resources allocated to each credit rating: ; In the formula, Credit rating The amount of resources allocated. This represents the total amount of currently available job resources. Credit rating The corresponding resource allocation weights, Credit rating The number of transport vehicles and goods. Credit rating The number of transport vehicles and goods.

[0053] This embodiment uses real-time status synchronization and anomaly prediction through a digital twin model to initiate the handling process before an abnormal event occurs; and prioritizes the convenience rights of AEO enterprises when resources are scarce, avoiding customs clearance delays for high-credit enterprises due to abnormal situations.

[0054] Example 2: like Figure 3 As shown, this embodiment provides a port scheduling device based on AEO digital twin, the device comprising: The data acquisition module is used to acquire real-time status data of physical entities at the port. The real-time status data includes at least the credit rating and cumulative waiting time of each means of transport and cargo. The model update module is used to update the digital twin model according to the real-time status data and embed AEO regulatory rules as constraints in the digital twin model. The scheduling generation module is used to generate scheduling schemes for each means of transport and goods based on the updated digital twin model as a simulation environment, according to the credit rating and cumulative waiting time, using a multi-agent reinforcement learning algorithm. The pre-simulation judgment module is used to input the scheduling scheme into the digital twin model for pre-simulation, and judge whether the scheduling scheme meets the constraints based on the pre-simulation results; The instruction issuing module is used to issue the scheduling scheme to the port physical equipment for execution when the scheduling scheme meets the constraints. The parameter update module is used to acquire the state data after execution and update the decision parameters of the multi-agent reinforcement learning algorithm based on the difference between the state data after execution and the real-time state data.

[0055] Based on the above embodiments, the model update module further includes: A physical entity model building unit is used to build a physical entity model, which includes real-time state data. A business process model building unit is used to build a business process model, which includes the sequential dependencies of checkpoint passage operations, inspection operations, and loading and unloading operations. The regulatory rule model construction unit is used to construct a regulatory rule model and set the AEO regulatory rule as a constraint condition in the regulatory rule model; The model component unit is used to compose the digital twin model from the physical entity model, the business process model, and the regulatory rule model.

[0056] Based on the above embodiments, the scheduling generation module includes: The global status acquisition unit is used to acquire global status information in the digital twin model. The global status information includes the queue length of each checkpoint, the queue length of each inspection area, the queue length of each loading and unloading area, the credit rating distribution of transport vehicles and goods in each queue, and the operating status of each operating device. The scheduling action selection unit is used to select three types of scheduling actions from the scheduling action set by taking the global state information as the input of the multi-agent reinforcement learning algorithm. The three types of scheduling actions include channel allocation action, inspection and sorting action, and loading and unloading equipment scheduling action. The scheduling scheme generation unit is used to determine the execution order of scheduling actions corresponding to each means of transport and cargo based on the credit rating, and generate a scheduling scheme that includes the three types of scheduling actions and their execution order.

[0057] Based on the above embodiments, a training module is also included, the training module comprising: The weight setting unit is used to set the corresponding waiting time weight according to the credit rating of each means of transport and cargo. The higher the credit rating, the greater the corresponding waiting time weight value. The simulation operation unit is used to input the scheduling scheme into the digital twin model for simulation operation and generate the simulated waiting time of each means of transport and goods in the digital twin model; The reward calculation unit is used to calculate the reward value of the scheduling scheme based on the simulated waiting time of each means of transport and cargo and its corresponding waiting time weight; The strategy adjustment unit is used to adjust the policy network parameters corresponding to each agent according to the reward value.

[0058] Based on the above embodiments, an exception handling module is also included, the exception handling module comprising: An execution status acquisition unit is used to acquire real-time status data of the execution process of the port physical equipment and update the current status of the digital twin model based on the real-time status data. An anomaly probability calculation unit is used to calculate the probability of an anomaly event occurring within a preset time window using the current state; The resource acquisition unit is used to acquire the total amount of currently available job resources when the probability exceeds a preset alarm threshold. The resource allocation unit is used to calculate the amount of resources allocated to each credit level according to the resource allocation weight corresponding to each credit level, the number of vehicles and goods of each credit level, and the total amount of currently available operational resources, and to allocate the allocated resources to the vehicles or goods of the corresponding credit level.

[0059] Based on the above embodiments, the parameter update module includes: The actual reward calculation unit is used to calculate the actual reward value of the scheduling scheme based on the actual clearance time of each port's physical equipment after executing the scheduling scheme; The training sample combination unit is used to combine the real-time status data, the scheduling scheme, the actual reward value, and the post-execution status data into a training sample set. The network parameter adjustment unit is used to set sample weights for each training sample according to the credit level of each training sample, and adjust the policy network parameters of the multi-agent reinforcement learning algorithm according to each training sample and its corresponding sample weight.

[0060] It should be noted that the specific manner in which each module performs its operation in the apparatus described in the above embodiments has been described in detail in the embodiments of the method, and will not be elaborated here.

[0061] Example 3: Corresponding to the above method embodiments, this embodiment also provides a port scheduling device based on AEO digital twin. The port scheduling device based on AEO digital twin described below and the port scheduling method based on AEO digital twin described above can be referred to in correspondence.

[0062] Figure 4 This is a block diagram illustrating a port scheduling device 800 based on an AEO digital twin, according to an exemplary embodiment. Figure 4 As shown, the port scheduling device 800 based on AEO digital twin may include: a processor 801 and a memory 802. The port scheduling device 800 based on AEO digital twin may also include one or more of the following: a multimedia component 803, an I / O interface 804, and a communication component 805.

[0063] The processor 801 controls the overall operation of the AEO-based digital twin port scheduling device 800 to complete all or part of the steps in the AEO-based digital twin port scheduling method described above. The memory 802 stores various types of data to support the operation of the AEO-based digital twin port scheduling device 800. This data may include, for example, instructions for any application or method operating on the AEO-based digital twin port scheduling device 800, and application-related data such as contact data, sent and received messages, pictures, audio and video, etc. The memory 802 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The multimedia component 803 may include a screen and audio / video components. The screen may be, for example, a touchscreen, and the audio / video components are used to output and / or input audio and video signals. For example, the audio / video component may include a microphone, camera, etc., for receiving external audio / video signals. The received audio / video signals may be further stored in memory 802 or transmitted via communication component 805. The audio / video component also includes at least one speaker for outputting audio signals. I / O interface 804 provides an interface between processor 801 and other interface modules, such as a keyboard, mouse, buttons, etc. These buttons may be virtual or physical buttons. Communication component 805 is used for wired or wireless communication between the AEO-based digital twin port dispatching device 800 and other devices. Wireless communication includes, for example, Wi-Fi, Bluetooth, Near Field Communication (NFC), 4G, 5G, 6G, or one or more combinations thereof; therefore, the corresponding communication component 805 may include a Wi-Fi module, a Bluetooth module, and an NFC module.

[0064] In one exemplary embodiment, the port scheduling device 800 based on AEO digital twin may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the aforementioned port scheduling method based on AEO digital twin.

[0065] In another exemplary embodiment, a computer-readable storage medium including program instructions is also provided. When executed by a processor, these program instructions implement the steps of the AEO-based digital twin-based port scheduling method described above. For example, the computer-readable storage medium may be the memory 802 including the program instructions described above. These program instructions may be executed by the processor 801 of the AEO-based digital twin-based port scheduling device 800 to complete the AEO-based digital twin-based port scheduling method described above.

[0066] Example 4: Corresponding to the above method embodiments, this embodiment also provides a readable storage medium. The readable storage medium described below can be referred to in conjunction with the port scheduling method based on AEO digital twin described above.

[0067] A readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the port scheduling method based on AEO digital twin described in the above method embodiments.

[0068] The readable storage medium can specifically be a USB flash drive, external hard drive, read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk, or any other readable storage medium capable of storing program code.

[0069] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

[0070] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A port scheduling method based on AEO digital twin, characterized in that, include: Obtain real-time status data of physical entities at the port, wherein the real-time status data includes at least the credit rating and cumulative waiting time of each means of transport and cargo; The digital twin model is updated based on the real-time status data, and AEO regulatory rules are embedded in the digital twin model as constraints. Using the updated digital twin model as the simulation environment, a multi-agent reinforcement learning algorithm is used to generate scheduling schemes for each means of transport and goods based on the credit rating and cumulative waiting time. The scheduling scheme is input into the digital twin model for pre-simulation, and the pre-simulation results are used to determine whether the scheduling scheme meets the constraints. If the conditions are met, the scheduling scheme will be sent to the port physical equipment for execution; Obtain the post-execution state data, and update the decision parameters of the multi-agent reinforcement learning algorithm based on the difference between the post-execution state data and the real-time state data.

2. The port scheduling method based on AEO digital twin according to claim 1, characterized in that, Before updating the digital twin model based on the real-time status data, the process also includes constructing the digital twin model: Construct a physical entity model, which includes real-time state data; Construct a business process model, which includes the sequential dependencies of checkpoint passage operations, inspection operations, and loading and unloading operations; Construct a regulatory rule model, and set the AEO regulatory rule as a constraint condition in the regulatory rule model; The digital twin model is composed of the physical entity model, the business process model, and the regulatory rule model.

3. The port scheduling method based on AEO digital twin according to claim 1, characterized in that, A multi-agent reinforcement learning algorithm is used to generate scheduling schemes for various transportation vehicles and goods, including: Obtain global state information from the digital twin model, which includes the queue lengths of each checkpoint, each inspection area, each loading and unloading area, the credit rating distribution of transport vehicles and goods in each queue, and the operating status of each piece of equipment. The global state information is used as the input of the multi-agent reinforcement learning algorithm. The multi-agent reinforcement learning algorithm selects three types of scheduling actions from the scheduling action set. The three types of scheduling actions include channel allocation action, inspection and sorting action, and loading and unloading equipment scheduling action. Based on the credit rating, the execution order of the scheduling actions corresponding to each means of transport and cargo is determined, and a scheduling scheme containing the three types of scheduling actions and their execution order is generated.

4. The port scheduling method based on AEO digital twin according to claim 1, characterized in that, It also includes training the multi-agent reinforcement learning algorithm: The waiting time weight is set according to the credit rating of each means of transport and cargo. The higher the credit rating, the greater the corresponding waiting time weight value. The scheduling scheme is input into the digital twin model for simulation, generating simulated waiting times for each means of transport and goods in the digital twin model; The reward value of the scheduling scheme is calculated based on the simulated waiting time of each means of transport and cargo and its corresponding waiting time weight. Adjust the policy network parameters corresponding to each agent based on the reward value.

5. The port scheduling method based on AEO digital twin according to claim 1, characterized in that, After the step of distributing the scheduling plan to the port physical equipment for execution, the method further includes: Acquire real-time status data of the port physical equipment during its execution process, and update the current status of the digital twin model based on the real-time status data; Calculate the probability of an abnormal event occurring within a preset time window using the current state; When the probability exceeds the preset alarm threshold, obtain the total amount of currently available job resources; Based on the resource allocation weight corresponding to each credit rating, the number of vehicles and goods for each credit rating, and the total amount of currently available operational resources, the amount of resources allocated to each credit rating is calculated, and the allocated resources are then distributed to the vehicles or goods of the corresponding credit rating.

6. A port scheduling device based on AEO digital twin, characterized in that, include: The data acquisition module is used to acquire real-time status data of physical entities at the port. The real-time status data includes at least the credit rating and cumulative waiting time of each means of transport and cargo. The model update module is used to update the digital twin model according to the real-time status data and embed AEO regulatory rules as constraints in the digital twin model. The scheduling generation module is used to generate scheduling schemes for each means of transport and goods based on the updated digital twin model as a simulation environment, according to the credit rating and cumulative waiting time, using a multi-agent reinforcement learning algorithm. The pre-simulation judgment module is used to input the scheduling scheme into the digital twin model for pre-simulation, and judge whether the scheduling scheme meets the constraints based on the pre-simulation results; The instruction issuing module is used to issue the scheduling scheme to the port physical equipment for execution when the scheduling scheme meets the constraints. The parameter update module is used to acquire the state data after execution and update the decision parameters of the multi-agent reinforcement learning algorithm based on the difference between the state data after execution and the real-time state data.

7. The port scheduling device based on AEO digital twin according to claim 6, characterized in that, The model update module also includes: A physical entity model building unit is used to build a physical entity model, which includes real-time state data. A business process model building unit is used to build a business process model, which includes the sequential dependencies of checkpoint passage operations, inspection operations, and loading and unloading operations. The regulatory rule model construction unit is used to construct a regulatory rule model and set the AEO regulatory rule as a constraint condition in the regulatory rule model; The model component unit is used to compose the digital twin model from the physical entity model, the business process model, and the regulatory rule model.

8. The port scheduling device based on AEO digital twin according to claim 6, characterized in that, The scheduling generation module includes: The global status acquisition unit is used to acquire global status information in the digital twin model. The global status information includes the queue length of each checkpoint, the queue length of each inspection area, the queue length of each loading and unloading area, the credit rating distribution of transport vehicles and goods in each queue, and the operating status of each operating device. The scheduling action selection unit is used to select three types of scheduling actions from the scheduling action set by taking the global state information as the input of the multi-agent reinforcement learning algorithm. The three types of scheduling actions include channel allocation action, inspection and sorting action, and loading and unloading equipment scheduling action. The scheduling scheme generation unit is used to determine the execution order of scheduling actions corresponding to each means of transport and cargo based on the credit rating, and generate a scheduling scheme that includes the three types of scheduling actions and their execution order.

9. The port scheduling device based on AEO digital twin according to claim 6, characterized in that, It also includes a training module, which includes: The weight setting unit is used to set the corresponding waiting time weight according to the credit rating of each means of transport and cargo. The higher the credit rating, the greater the corresponding waiting time weight value. The simulation operation unit is used to input the scheduling scheme into the digital twin model for simulation operation and generate the simulated waiting time of each means of transport and goods in the digital twin model; The reward calculation unit is used to calculate the reward value of the scheduling scheme based on the simulated waiting time of each means of transport and cargo and its corresponding waiting time weight; The strategy adjustment unit is used to adjust the policy network parameters corresponding to each agent according to the reward value.

10. The port scheduling device based on AEO digital twin according to claim 6, characterized in that, It also includes an exception handling module, which includes: An execution status acquisition unit is used to acquire real-time status data of the port physical equipment during the execution process and update the current status of the digital twin model based on the real-time status data. An anomaly probability calculation unit is used to calculate the probability of an anomaly event occurring within a preset time window using the current state; The resource acquisition unit is used to acquire the total amount of currently available job resources when the probability exceeds a preset alarm threshold. The resource allocation unit is used to calculate the amount of resources allocated to each credit level according to the resource allocation weight corresponding to each credit level, the number of vehicles and goods of each credit level, and the total amount of currently available operational resources, and to allocate the allocated resources to the vehicles or goods of the corresponding credit level.