An intelligent logistics full-link dynamic scheduling method and system based on digital twinning

Abstract

The application discloses an intelligent logistics full-link dynamic scheduling method and system based on digital twinning, and relates to the technical field of logistics scheduling. Physical operation data and business process data of a logistics full scene are collected through a physical perception layer and uploaded to a data fusion layer; the data fusion layer performs cleaning, format conversion, correlation matching and time axis alignment processing on the received data, forms a standard data set and synchronizes the standard data set to a digital twinning layer and an algorithm scheduling layer; the digital twinning layer constructs a multi-level virtual mapping model based on static modeling data and the standard data set, realizes bidirectional real-time synchronization of a physical space and a virtual space, and performs simulation deduction based on the virtual model, and synchronizes the deduction result to the algorithm scheduling layer; the algorithm scheduling layer takes the standard data set and the deduction result as input, and generates executable scheduling instructions covering all links of transportation, warehousing and terminal distribution based on a reinforcement learning algorithm. The application improves the logistics full-link scheduling efficiency and intelligent level.

Description

Technical Field

[0001] This invention relates to the field of logistics scheduling technology, specifically to a method and system for intelligent logistics end-to-end dynamic scheduling based on digital twins. Background Technology

[0002] In recent years, the application of digital twin technology in the logistics field has gradually matured, and digital twin and AI scheduling systems have become core hotspots in industry research and application. Currently, the scheduling management of the logistics industry has gradually evolved from traditional manual scheduling and single-stage information scheduling to multi-stage digital and intelligent scheduling. Technologies such as IoT devices, GPS positioning, WMS warehouse management systems, and TMS transportation management systems have been initially applied in various stages of logistics.

[0003] The closest existing technical solution to this invention is a logistics partial-link scheduling system that combines digital twin technology. This type of system mainly achieves status monitoring and simple scheduling optimization for a single link in warehousing or transportation by constructing a digital twin model of that link. It also integrates data from sources such as IoT and GPS, and issues instructions through preset scheduling rules. While some large logistics companies' high-end scheduling systems can achieve full-link digital twin scheduling, their complex system architecture and high deployment costs mean they are only suitable for the standardized business scenarios of large enterprises.

[0004] However, the aforementioned existing technical solutions still have the following shortcomings: poor scenario adaptability; existing high-end system architectures are complex and costly, failing to meet the low-cost, lightweight application needs of small and medium-sized logistics enterprises. Furthermore, the existing systems of small and medium-sized enterprises are mostly single-link information systems, making it difficult to achieve end-to-end scheduling; rigid scheduling mechanisms; existing systems are mostly based on preset rules, lacking self-learning and self-optimization capabilities, resulting in delayed adjustments to scheduling strategies in the face of emergencies; insufficient data fusion and collaboration; existing systems have weak integration capabilities for multi-source heterogeneous data, and there is a delay in data synchronization between physical and virtual spaces, leading to low collaboration efficiency across links and poor end-to-end scheduling accuracy; and an imperfect management system; existing systems struggle to achieve closed-loop management of prediction, monitoring, scheduling, and feedback, lacking accurate prediction of the operational status of logistics nodes and intelligent risk warning capabilities, resulting in weak supply chain emergency response capabilities.

[0005] Therefore, there is an urgent need in this field for an intelligent scheduling method and system that can achieve efficient integration of multi-source data across the entire logistics chain, possess autonomous learning and optimization capabilities, and support rapid response to emergencies, in order to meet the pressing needs of small and medium-sized logistics enterprises for low-cost, lightweight, and intelligent scheduling across the entire chain. Summary of the Invention

[0006] To address the problems of poor end-to-end coordination, delayed scheduling response, low scenario adaptability, and weak risk prediction capability in existing logistics scheduling systems, this invention provides an intelligent logistics end-to-end dynamic scheduling method and system based on digital twins to solve the aforementioned technical deficiencies.

[0007] This invention proposes a dynamic scheduling method for the entire intelligent logistics chain based on digital twins, comprising the following steps: S1. Real-time collection of physical operation data and business process data across the entire logistics scenario through sensing devices deployed in the physical sensing layer, and uploading of physical operation data and business process data to the data fusion layer via communication protocols; S2. The data fusion layer uses ETL tools to clean, convert, match, and align the received physical operation data and business process data to form a unified standard dataset, and then synchronizes the standard dataset to the digital twin layer and the algorithm scheduling layer respectively. S3, the digital twin layer acquires static modeling data of the logistics scenario. Based on the static modeling data and standard dataset, a multi-level virtual mapping model of the entire logistics scenario is constructed using a hierarchical modeling method. A real-time data synchronization protocol is used to achieve bidirectional real-time data synchronization and accurate mapping between the physical space and the virtual space to obtain a virtual model. At the same time, scenario simulation is performed based on the virtual model to obtain simulation results, and the simulation results are synchronized to the algorithm scheduling layer. S4, the algorithm scheduling layer takes the standard dataset and simulation results as input, and transforms the logistics end-to-end scheduling problem into a Markov decision process based on the reinforcement learning algorithm. With logistics operation efficiency and operating cost as objective functions, it iteratively optimizes the global scheduling strategy and generates executable scheduling instructions covering the entire process of transportation, warehousing and last-mile delivery.

[0008] This invention also proposes a digital twin-based intelligent logistics end-to-end dynamic scheduling system to implement the method described above. The system includes: The physical sensing layer is configured to collect physical operation data and business process data of the entire logistics scenario in real time through deployed sensing devices, and upload the physical operation data and business process data to the data fusion layer through communication protocols. The data fusion layer is configured to use ETL tools to clean, convert, match, and align the received physical operation data and business process data to form a unified standard dataset, and then synchronize the standard dataset to the digital twin layer and the algorithm scheduling layer respectively. The digital twin layer is configured to acquire static modeling data of logistics scenarios. Based on the static modeling data and standard datasets, a multi-level virtual mapping model of the entire logistics scenario is constructed using a hierarchical modeling method. A real-time data synchronization protocol is used to achieve bidirectional real-time data synchronization and accurate mapping between physical space and virtual space to obtain a virtual model. Simultaneously, scenario simulation is performed based on the virtual model to obtain simulation results, which are then synchronized to the algorithm scheduling layer. The algorithm scheduling layer is configured to take standard datasets and simulation results as inputs, and transform the logistics end-to-end scheduling problem into a Markov decision process based on reinforcement learning algorithms. With logistics operation efficiency and operating cost as objective functions, it iteratively optimizes the global scheduling strategy and generates executable scheduling instructions covering all aspects of transportation, warehousing and last-mile delivery.

[0009] The present invention also proposes a terminal device, including a memory, a processor, and a computer program stored in the memory and capable of running on the processor. When the processor executes the computer program, it implements the steps of the above-described intelligent logistics end-to-end dynamic scheduling method based on digital twins.

[0010] The present invention also proposes a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the above-described intelligent logistics end-to-end dynamic scheduling method based on digital twins.

[0011] Compared with the prior art, the beneficial effects of the present invention are as follows: (1) This invention achieves deep fusion of multi-source heterogeneous data through two-level associative primary key design and multi-frequency time axis alignment method, effectively solving the technical problem of insufficient data fusion and collaboration in existing systems. Specifically, a globally unique order number is used as the primary key to connect inbound and outbound documents from the WMS warehouse management system, warehouse inventory data, transportation waybill data from the TMS transportation management system, last-mile delivery dispatch data, and GPS transportation trajectory data. A unique device ID is used as the secondary key to bind real-time data collected by IoT sensors, GPS positioning terminals, warehouse barcode scanners, and vehicle ECUs to transportation vehicles, warehouse equipment, and delivery terminals. Then, the primary key bound to the device is used to connect back to the business data, enabling deep integration of real-time data at the device layer and order data at the business layer under a unified data standard. This is combined with a multi-frequency time alignment method that uses GPS collection timestamps as the global reference time axis, linear interpolation for high-frequency continuous data, nearest neighbor matching for low-frequency business data, and timestamp anchoring for event-triggered data. This eliminates semantic conflicts and time misalignments caused by differences in sources and reporting frequencies between multiple data sources. As a result, the multi-source heterogeneous data collected by the physical sensing layer becomes a standard dataset with a unified spatiotemporal reference before entering the digital twin layer and algorithm scheduling layer, significantly improving the data consistency and accuracy of the entire logistics chain scheduling.

[0012] (2) This invention constructs a multi-layered virtual mapping model for the entire logistics scenario through a layered modeling method, and adopts a real-time synchronous update mode that combines timed triggering and event triggering mechanisms. This effectively solves the technical problems of data synchronization delay between physical space and virtual space and weak scenario adaptability in existing systems. Specifically, the geometric model replicates the spatial structure and equipment shape of the physical scenario, the behavioral model simulates the operation process of each link based on the operation process information and interaction relationship information, and the physical model continuously receives real-time data driven by the logistics equipment operating parameters and cargo physical characteristics information in the standard dataset. The three-layer model completely depicts the entire logistics scenario from the three dimensions of spatial structure, operation logic, and physical characteristics. Under normal operating conditions, the basic data is synchronized according to the preset synchronization cycle to ensure the consistency of the basic state of the virtual model and the physical entity. When an abnormal event is detected, the event-driven high-priority update is immediately triggered to synchronize the abnormal data to the virtual model in real time. Through the synergistic cooperation of the two mechanisms, the basic consistency between the virtual model and the physical entity under normal conditions is guaranteed, and the timely perception and accurate mapping of the sudden situation under abnormal conditions are realized, which effectively improves the system's adaptability to complex and ever-changing logistics scenarios.

[0013] (3) This invention constructs a full-link dynamic scheduling algorithm through the reinforcement learning DQN algorithm framework and realizes the autonomous optimization of the algorithm model based on the experience replay mechanism, effectively solving the technical problems of rigid scheduling mechanism and lack of autonomous learning ability in the existing system. Specifically, the logistics full-link scheduling problem is transformed into a Markov decision process. By constructing a state space containing logistics equipment operating parameters, vehicle conditions, delivery node progress, warehouse inventory data and cargo physical characteristics information, as well as an action space covering all executable scheduling operations in transportation, warehousing and last-mile delivery, the logistics operation efficiency and operating cost are quantified into a reward function to realize the iterative optimization of the scheduling strategy. On this basis, the instruction execution results and feedback data of the physical execution layer are pushed to the experience replay pool of the algorithm scheduling layer after being standardized by the data fusion layer. The Q-value network parameters are iteratively updated using the experience replay mechanism and gradient descent method, so that the decision-making ability of the algorithm model can continuously reflect the efficiency and cost trade-off in the real scheduling scenario, forming a complete closed-loop link of collection, fusion, modeling, scheduling, execution, feedback and optimization, realizing the continuous evolution and adaptive optimization of the scheduling strategy.

[0014] (4) This invention constructs a complete technical link from physical space data acquisition to virtual space model mapping, from intelligent scheduling decision to physical equipment execution, and from execution result feedback to algorithm model optimization through the collaborative work of the physical perception layer, data fusion layer, digital twin layer and algorithm scheduling layer. Each layer achieves efficient collaboration through standardized data interfaces and communication protocols, enabling the system to have the ability to view the status in real time, manage resources dynamically, control strategies autonomously, and predict risks in advance throughout the entire link. This significantly improves the collaborative scheduling efficiency and intelligent management level of the entire logistics link, and meets the urgent needs of small and medium-sized logistics enterprises for low-cost, lightweight, and intelligent scheduling throughout the entire link. Attached Figure Description

[0015] Other features, objects, and advantages of the present invention will become more apparent from the following detailed description of non-limiting embodiments, taken with reference to the accompanying drawings: Figure 1 This is a flowchart of a dynamic scheduling method for the entire intelligent logistics chain based on digital twins; Figure 2 This is a structural diagram of a digital twin-based intelligent logistics end-to-end dynamic scheduling system; Figure 3 This is a schematic diagram of the structure of a computer system suitable for implementing the embodiments of the present invention. Detailed Implementation

[0016] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. Furthermore, it should be noted that, for ease of description, only the parts relevant to the invention are shown in the accompanying drawings.

[0017] It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other. The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0018] Figure 1 A flowchart illustrating a dynamic scheduling method for the entire intelligent logistics chain based on digital twins is shown. Figure 1 As shown, the method includes the following steps: Step S1: Collect physical operation data and business process data of the entire logistics scenario in real time through sensing devices deployed in the physical sensing layer, and upload the physical operation data and business process data to the data fusion layer through communication protocols.

[0019] The physical sensing layer is equipped with various sensing devices, including IoT sensors, GPS positioning terminals, warehouse barcode scanning devices, and delivery check-in devices. These sensing devices collect data from the entire logistics scenario in real time. The collected data is divided into two categories: one is physical operation data, including the location of transport vehicles, vehicle operating conditions, warehouse cargo status, delivery node progress, and logistics equipment operating parameters; the other is business process data, including business data from the WMS warehouse management system and the TMS transportation management system.

[0020] The IoT sensors and GPS positioning terminals in the physical sensing layer use the MQTT protocol to report the collected physical operation data to the data fusion layer. The data fusion layer uses the OPC UA protocol to interface with the WMS warehouse management system and TMS transportation management system to obtain business process data and upload it to the data fusion layer. The digital twin layer and the visualization and control layer use the WebSocket protocol for full-duplex real-time data push, with the visualization and control layer used for the visualization and issuance of scheduling instructions in subsequent steps.

[0021] The physical sensing layer supports concurrent device access of 10,000 points or more, ensuring large-scale data collection capabilities. In one specific implementation, IoT sensors are deployed in the cargo boxes of transport vehicles and warehousing equipment to monitor parameters such as temperature and humidity of the transportation environment, vibration and impact during vehicle operation, and operating current and temperature of warehousing loading and unloading equipment in real time; GPS positioning terminals report the location coordinates, driving direction, instantaneous speed, and other position and motion parameters of the transport vehicles in real time; warehouse barcode scanning devices collect barcode or RFID tag information of goods during goods entering, leaving, and transferring; and delivery check-in devices are used by last-mile delivery personnel to check in upon completion of delivery, recording the delivery completion time and node location.

[0022] Step S2: The data fusion layer uses ETL tools to clean, convert, match, and align the received physical operation data and business process data to form a unified standard dataset, and then synchronizes the standard dataset to the digital twin layer and the algorithm scheduling layer respectively.

[0023] The data fusion layer establishes multi-source heterogeneous data integration interfaces, connecting to device data collected by the physical sensing layer and business data from the enterprise's existing WMS warehouse management system and TMS transportation management system. ETL tools complete the entire process of extraction, transformation, and loading. The ETL tool specifically executes the following sub-steps: S21. The data fusion layer cleans the received physical operation data and business process data, removes missing fields and abnormal data, and obtains cleaned data.

[0024] In one specific implementation, physical operation data and business process data come from different data sources and acquisition channels. The ETL tool performs independent cleaning processes for the physical operation data and business process data respectively. For physical operation data reported via the MQTT protocol, protocol parsing and format verification are performed first, followed by missing value detection and outlier screening according to their respective data dictionaries, removing invalid data points that exceed the threshold range or fail verification. For business process data obtained through OPC UA protocol integration, the integrity of business fields is checked, records with missing key fields are supplemented or removed, and logically conflicting data entries are deleted. After the cleaning process is completed, the two types of cleaned data are merged into a unified temporary dataset to constitute the cleaned data.

[0025] S22. Convert the format of the cleaned data to convert heterogeneous data from different sources into a unified data format to obtain unified format data.

[0026] In one specific implementation, the cleaned data includes real-time data collected from physical devices such as IoT sensors, GPS positioning terminals, warehouse barcode scanners, and vehicle ECUs, as well as business data from WMS (Warehouse Management System) and TMS (Transportation Management System). Data from different sources differ in data structure, field naming, data type, and encoding methods, necessitating format standardization. This includes standardizing timestamp formats, device numbering rules, data types for status fields, and geographic coordinate reference systems to ensure all data can be processed under the same data standard.

[0027] S23. Perform association matching on data in a unified format, specifically including two-level association primary key design.

[0028] The primary key is the globally unique order number. Using the globally unique order number as the primary key, the inbound and outbound documents and warehouse inventory data of the WMS warehouse management system, the transportation waybill data of the TMS transportation management system, the last-mile delivery dispatch data, and the transportation trajectory data of the GPS positioning terminal are linked through the primary key to realize the connection of data for a single order from warehousing, transportation to last-mile delivery.

[0029] The secondary primary key is the device's unique ID. Using this unique device ID as the secondary primary key, real-time data collected by IoT sensors, GPS positioning terminals, warehouse barcode scanners, and vehicle ECUs are bound to their corresponding transport vehicles, warehouse equipment, and delivery terminals. Then, the primary primary key bound to the device is used to complete the association and matching with business process data, obtaining the data that has been successfully associated. This step enables the linkage between device data and business data, allowing for deep integration of real-time data at the device layer and order data at the business layer under a unified data standard.

[0030] S24. Using the collection timestamp of the GPS positioning terminal as the global reference time axis, the data that has been associated and matched is time-aligned on the global reference time axis to obtain the time-aligned data and form a unified standard dataset.

[0031] In one specific implementation, an alignment method combining a reference timestamp and nearest neighbor matching is used to address the differences in reporting frequencies from different data sources. Specifically, the collection timestamp of the GPS positioning terminal is used as the global reference time axis, with a time granularity of 1 second, to construct a continuous time series. For high-frequency reported continuous device data, linear interpolation is used to fill discrete reported data into each time granularity of the global reference time axis, ensuring data continuity. For low-frequency reported business data, a nearest neighbor matching method is used. For any time point on the global reference time axis, the business data value whose timestamp is closest to and earlier than that time point is taken as the alignment data for that time point. For event-triggered reported discrete data, an event timestamp anchoring method is used to accurately map event data to the corresponding time point on the global reference time axis, while retaining associated device data within 10 seconds before and after the event, achieving linkage alignment between event data and device status data.

[0032] Through the aforementioned cleaning, format conversion, correlation matching, and timeline alignment processes, the multi-source heterogeneous data eliminates semantic conflicts and temporal misalignments caused by differences in source and reporting frequency, forming a standard dataset with a unified spatiotemporal benchmark. This standard dataset contains physical operation data and business process data that have undergone correlation matching and time alignment. The physical operation data specifically includes logistics equipment operating parameters and vehicle operating conditions, while the business process data specifically includes delivery node progress, warehouse inventory data, and cargo physical characteristic information.

[0033] Step S3: The digital twin layer acquires static modeling data of the logistics scenario. Based on the static modeling data and standard dataset, a multi-level virtual mapping model of the entire logistics scenario is constructed using a hierarchical modeling method. A real-time data synchronization protocol is used to achieve bidirectional real-time data synchronization and accurate mapping between the physical space and the virtual space to obtain a virtual model. At the same time, scenario simulation is performed based on the virtual model to obtain simulation results, and the simulation results are synchronized to the algorithm scheduling layer.

[0034] The digital twin layer employs a layered modeling approach, constructing geometric, physical, and behavioral models of the logistics scenario. The geometric model replicates the spatial structure and equipment shape of the physical scenario, the behavioral model simulates the operational processes and interactions of each stage of logistics, and the physical model simulates the operating parameters of logistics equipment and the physical characteristics of goods. The basic data required for layered modeling is imported during the system initialization phase. Based on static modeling data and standard datasets, the digital twin layer uses a layered modeling approach to construct a multi-layered virtual mapping model of the entire logistics scenario, specifically including the following sub-steps: S31. The digital twin layer acquires static modeling data of the logistics scenario. The static modeling data includes spatial structure information, equipment shape information, operation process information and interaction relationship information.

[0035] In one specific implementation, the spatial structure information comes from the CAD design drawings of the logistics operation site, including the building floor plan, the division of functional areas, the direction of passageways, and the location of access control; the equipment shape information comes from the equipment 3D model or equipment specification manual, including the shelf dimensions, the width and length of the conveyor belt, the shape of the sorting machine, and the external dimensions of the vehicle; the operation process information and interaction relationship information come from the operation process specification document, including the complete flow path of goods from warehousing to outbound, the connection relationship between each link, and the collaborative working logic between equipment.

[0036] S32. Construct a geometric model based on the spatial structure information and equipment shape information in the static modeling data.

[0037] The geometric model replicates the spatial structure and equipment shape of the physical scene. In one specific implementation, the geometric model is constructed using 3D modeling software. A building structure model is established based on the site CAD design drawings, and geometric models of equipment such as shelves, conveyor belts, sorting machines, and vehicles are established based on the 3D models or specifications of the equipment. All models are then assembled according to their actual spatial relationships to form a virtual scene that is completely consistent with the physical scene in terms of geometric shape and spatial layout.

[0038] S33. Construct a behavioral model based on the work process information and interaction relationship information in the static modeling data.

[0039] Behavioral models simulate the operational processes and interactions of various stages in logistics. In one specific implementation, the behavioral model defines the complete operational process of goods from warehousing, storage, picking, outbound to transportation and last-mile delivery, as well as the triggering conditions and data transmission rules between each stage.

[0040] S34. Construct a physical model based on the operating parameters of logistics equipment and the physical characteristics of goods in the standard dataset.

[0041] The physical model simulates the operating parameters of logistics equipment and the physical characteristics of goods. In one specific implementation, the physical model continuously receives real-time data driven by the operating parameters of logistics equipment and the physical characteristics of goods from a unified standard dataset. The operating parameters of logistics equipment include equipment operating condition data collected by IoT sensors, vehicle operating parameters collected by the vehicle ECU, and location and motion data collected by GPS positioning terminals; the physical characteristics of goods include the weight, volume, temperature and humidity requirements, and storage location of goods in the WMS warehouse management system. The physical model continuously receives incremental updates from the standard dataset through a real-time data synchronization protocol, ensuring that the operating status of logistics equipment and the physical state of goods in the virtual space remain consistent with the physical space in real time.

[0042] S35. Overlay the geometric model, behavioral model and physical model to obtain a multi-level virtual mapping model of the entire logistics scenario.

[0043] In one specific implementation, the virtual mapping model formed by superimposing the three-level models includes not only the geometric shape and spatial layout information of the physical scene, but also the behavioral logic information of the logistics operation process, as well as the real-time status information of equipment operating parameters and cargo physical characteristics, which can fully depict the physical logistics scene from multiple dimensions.

[0044] The digital twin layer achieves bidirectional real-time data synchronization and precise mapping between physical and virtual spaces through a real-time data synchronization protocol, thereby obtaining a virtual model. Specific synchronization mechanisms include: Under normal operating conditions, a timed trigger update method is adopted to acquire logistics equipment operating parameters and vehicle operating conditions from physical operating data, and delivery node progress and warehouse inventory data from business process data according to a preset synchronization cycle. The acquired logistics equipment operating parameters, vehicle operating conditions, delivery node progress, and warehouse inventory data are then synchronized to the virtual model to obtain a virtual model that is consistent with the physical space in real time. In one specific implementation, the synchronization cycle under normal operating conditions is set to 200 milliseconds, fully synchronizing basic data such as equipment operating status, node operation progress, and inventory status to ensure the consistency of the basic state between the virtual model and the physical entity.

[0045] Upon detecting an abnormal event, an event-triggered update method is employed to acquire the abnormal event data and synchronize it to the virtual model in real time. This results in a virtual model that accurately maps the abnormal operating state of the physical space, while simultaneously triggering an emergency response from the algorithm scheduling layer. In one specific implementation, when abnormal events such as vehicle malfunctions, warehouse overload, delivery delays, or equipment operating parameters exceeding limits occur, an event-driven high-priority update is immediately triggered. The abnormal data is reported in real time and synchronized to the virtual model, with a synchronization delay controlled within 100 milliseconds. Simultaneously, an emergency response from the scheduling algorithm is triggered.

[0046] In one specific implementation, the end-to-end data synchronization delay between the physical entity and the virtual model is controlled within 500 milliseconds to meet the millisecond-level synchronization requirements; the synchronization delay when an abnormal event triggers an update is controlled within 100 milliseconds.

[0047] The digital twin layer also performs scenario simulations based on the virtual model, obtaining simulation results which are then synchronized to the algorithm scheduling layer. In one specific implementation, during the simulation, the system, based on the collaborative work of the geometric, behavioral, and physical models within the virtual model, simulates the complete flow of goods from warehousing to last-mile delivery in chronological order. It records key indicators such as resource occupancy, task completion time, and equipment load rate at each logistics node, generating simulation result data. The simulation results specifically include information such as the expected resource requirements, expected task completion time, potential bottlenecks, and risk points for each logistics node within a future time window. These simulation results are synchronized to the algorithm scheduling layer as one of the inputs for strategy optimization in the scheduling algorithm.

[0048] Step S4: The algorithm scheduling layer takes the standard dataset and simulation results as input, and constructs a full-link dynamic scheduling algorithm based on reinforcement learning algorithm. It transforms the logistics full-link scheduling problem into a Markov decision process. With logistics operation efficiency and operating cost as objective functions, it iteratively optimizes the global scheduling strategy and generates executable scheduling instructions covering the entire process of transportation, warehousing and last-mile delivery.

[0049] The end-to-end dynamic scheduling algorithm is based on the reinforcement learning DQN algorithm framework. It transforms the end-to-end logistics scheduling problem into a Markov decision process, using the highest logistics efficiency and lowest operating cost as the reward function. Through real-time data learning from the logistics scenario, it iteratively optimizes the scheduling strategy. The core parameters and feedback optimization mechanism of the DQN algorithm are defined as follows: S41. Construct a state space by normalizing the logistics equipment operating parameters, vehicle conditions, delivery node progress, warehouse inventory data, and cargo physical characteristics information in the standard dataset and using them as the state vector input to the reinforcement learning algorithm.

[0050] In one specific implementation, the real-time status data of the entire logistics chain is normalized to construct the state vector S input to the algorithm, which includes resource status dimension, business status dimension, and environment and abnormal status dimension.

[0051] Resource status dimensions include the real-time location, load, remaining range, and driving status of transport vehicles; warehouse space occupancy, loading and unloading equipment operation status, and real-time inventory quantity; and the on-duty status of last-mile delivery personnel, the number of completed and pending orders, and their current location.

[0052] The business status dimension includes full order information, specifically covering order number, shipping and receiving addresses, delivery time requirements, cargo priority, weight and volume; real-time road conditions, traffic control, and weather data for transportation routes; and the operational progress of each logistics node, including loading and unloading completion rate, sorting progress, and delivery completion progress.

[0053] The environmental and abnormal status dimensions include the types of abnormal events, the scope of impact, and the estimated recovery time for each logistics node; historical scheduling strategy execution deviation data; and load saturation data for core resources such as vehicles, warehouses, and personnel.

[0054] S42. Construct the action space, and discretize all executable scheduling operations in the entire process to serve as the action vectors output by the reinforcement learning algorithm.

[0055] In one specific implementation, the executable scheduling operations across the entire chain are discretized to construct an action vector A output by the algorithm, covering executable instructions for the transportation, warehousing, last-mile delivery, and emergency response stages. The transportation process includes vehicle-order matching and allocation results, optimal transportation route planning results, vehicle formation adjustment schemes, and instructions for adjusting intermediate transfer points.

[0056] The warehousing process includes the results of inbound storage location allocation, the results of outbound picking route optimization, the loading and unloading operation sequence adjustment plan, and the scheduling instructions for warehouse personnel and equipment.

[0057] The last-mile delivery process includes delivery personnel-order dispatch results, delivery route optimization results, delivery priority adjustment instructions, and last-mile network resource allocation plans.

[0058] Emergency response measures include emergency dispatch strategies for abnormal events, temporary cross-node resource allocation instructions, and order timeliness adjustment plans.

[0059] S43. Construct a reward function to quantify the objective function into a weighted combination of logistics operation efficiency and operating costs. Use the reward function to evaluate the merits of the scheduling strategy output by the reinforcement learning algorithm.

[0060] In one specific implementation, the optimization objective of maximizing logistics efficiency and minimizing operating costs is quantitatively broken down to construct a total reward function R, which is calculated using the following formula:

[0061] in Rewards for efficiency As a cost incentive, This is a penalty item.

[0062] Efficiency Rewards The effectiveness of improving the overall operational efficiency is quantified using the reduction rate of average order fulfillment time within a single scheduling cycle and the reduction rate of waiting time at each stage as the core calculation indicators, calculated through the following formula:

[0063] in The average order fulfillment time under the baseline scheduling strategy. This represents the actual fulfillment time under the current scheduling strategy. The total waiting time for each stage under the baseline scheduling strategy. This represents the actual waiting and dwell time under the current strategy. , These are weighting coefficients, with values ​​of 0.6 and 0.4 respectively.

[0064] Cost Incentive The effectiveness of operational cost control is quantified using the overall cost reduction rate of transportation mileage, manpower input, and equipment energy consumption within a single scheduling cycle as the core calculation indicator, calculated using the following formula:

[0065] in The single-cycle comprehensive operating cost under the baseline scheduling strategy, The actual overall operating cost under the current strategy, This is the weighting coefficient, with a value of 0.5.

[0066] Penalty items Negative rewards are set for behaviors that do not meet the optimization goals, such as order timeouts, unresponsive abnormal events, and resource overload, calculated using the following formula:

[0067] in For the number of orders that have exceeded the time limit, This represents the number of unresponsive exceptions. Total duration of core resource overload. , , The penalty coefficients are 2, 5, and 1, respectively.

[0068] S44. Using the DQN algorithm framework, based on the state space, action space and reward function, the Q-value network parameters are iteratively updated to output the globally optimal scheduling policy, and executable scheduling instructions are generated according to the globally optimal scheduling policy.

[0069] In one specific implementation, the algorithm scheduling layer employs an experience replay mechanism for model training. The four-tuple data generated for each scheduling step—current state S, executed action A, reward R, and next state S'—is stored in the experience replay pool. During each training iteration, a batch of samples is randomly selected from the replay pool, and the Q-value network parameters of the DQN network are updated using gradient descent to minimize the mean squared error loss between the target Q-value and the predicted Q-value. Simultaneously, a fixed target network update period is set, updating the target network every 50 scheduling steps to ensure the stability of the training process. For scheduling execution deviation data, the algorithm automatically adjusts the weight coefficients of the reward function, strengthening the learning weights of high-yield scheduling strategies and weakening the impact of low-yield and negative-yield strategies, achieving autonomous learning and optimization of the model and continuously adapting to dynamically changing logistics scenarios.

[0070] In a specific embodiment, the intelligent logistics full-link dynamic scheduling method based on digital twin provided by the present invention further includes step S5: sending executable scheduling instructions to the physical devices and execution terminals corresponding to the physical sensing layer, so that on-site personnel and equipment can execute the executable scheduling instructions.

[0071] The visualization control layer establishes a full-chain visualization control platform for logistics, visually displaying information such as the virtual model of the digital twin layer, the operational status of logistics nodes, real-time equipment data, and the execution status of scheduling instructions. The visualization platform adopts a B / S architecture, supporting access from multiple terminals including computers and mobile devices. It displays full-chain logistics data statistics and risk warning information through a 2D visualization interface, and the digital twin virtual model through a 3D visualization interface, achieving an integrated display of two-dimensional data and three-dimensional scenes. Real-time data push from the visualization platform utilizes Redis caching technology to ensure data timeliness, and uses the WebSocket protocol to achieve real-time data refresh of the front-end interface. The visualization control layer visualizes the executable scheduling instructions generated by the algorithm scheduling layer. After confirmation by management personnel or automatic execution by the system, the executable scheduling instructions are sent to the corresponding physical devices and execution terminals in the physical sensing layer, where on-site personnel and equipment complete the execution of the scheduling instructions. The visualization control layer supports manual intervention in scheduling strategies by management personnel, achieving a combination of manual and intelligent scheduling.

[0072] In a specific embodiment, the intelligent logistics full-link dynamic scheduling method based on digital twin provided by the present invention further includes step S6: after execution, the execution results and feedback data of the executable scheduling instructions are collected in real time through the physical perception layer and transmitted back. After standardization processing by the data fusion layer, the data is synchronized to the digital twin layer to update the virtual model and pushed to the algorithm scheduling layer. The reinforcement learning algorithm parameters are updated through the experience playback mechanism to complete the autonomous optimization of the reinforcement learning algorithm.

[0073] After the execution terminals such as vehicles, warehousing equipment, and delivery personnel in the physical execution layer complete the scheduling instructions, the data acquisition devices in the physical sensing layer collect the instruction execution status in real time, including executed, failed, or partially executed status, as well as feedback data such as actual fulfillment time, actual transportation mileage, actual resource consumption, delay time in each link, and occurrence of abnormal events. Specifically, the instruction execution status is divided into three categories: executed, failed, and partially executed. An executed status indicates that the scheduling instruction has been fully executed and achieved the expected effect; a failed status indicates that the scheduling instruction was not executed or an unrecoverable anomaly occurred during execution; and a partially executed status indicates that part of the scheduling instruction was executed while the rest was not executed or was interrupted due to an anomaly. The above feedback data is uploaded to the data fusion layer via the MQTT protocol. After standardization processing, it is synchronized to the digital twin layer to update the virtual model, and simultaneously pushed to the experience replay pool of the algorithm scheduling layer in real time through a Redis cache queue, completing the end-to-end feedback data transmission. Based on the transmitted feedback data, the algorithm scheduling layer trains the model according to the experience replay mechanism and parameter update method described in step S44, realizing the model's autonomous learning and optimization. The above process forms a complete closed-loop chain of acquisition, fusion, modeling, scheduling, execution, feedback, and optimization, namely, constructing a closed-loop scheduling mechanism of physical acquisition, digital mapping, algorithm scheduling, physical execution, result feedback, and algorithm optimization.

[0074] The system provided by this invention achieves a scheduling command response time of less than 2 seconds, a logistics node operation status prediction accuracy of over 90%, and an overall logistics operation efficiency improvement of over 20%. The software tools involved include ETL data processing tools such as DataStage and Kettle, Redis caching database, MySQL business database, 3D modeling software such as 3ds Max and Blender, reinforcement learning algorithm frameworks such as TensorFlow and PyTorch, front-end visualization development frameworks such as Vue and Three.js, and back-end development frameworks such as Spring Boot.

[0075] In one specific implementation, the reinforcement learning algorithm automatically adjusts the weight coefficients of the reward function based on scheduling execution deviation data. Specifically, when the execution performance of a certain type of scheduling strategy consistently outperforms the baseline level, the algorithm increases the weight coefficient of the reward component corresponding to that type of strategy to strengthen the learning weight of high-yield scheduling strategies; when the execution performance of a certain type of scheduling strategy consistently falls below the baseline level, the algorithm decreases the weight coefficient of its corresponding reward component to weaken the impact of low-yield and negative-yield strategies. Through this mechanism, the algorithm model possesses the ability to continuously adapt to dynamically changing logistics scenarios.

[0076] To meet the application needs of small and medium-sized logistics enterprises, the system provided by this invention features a lightweight and simplified architecture. The digital twin layer, data fusion layer, and algorithm scheduling layer adopt a cloud-native deployment approach, reducing the deployment cost of local servers for enterprises. Simultaneously, it supports modular system decomposition, allowing enterprises to select and deploy warehouse management, transportation scheduling, and last-mile delivery modules as needed based on their business requirements. This significantly lowers the application threshold and maintenance costs of the system, enabling the low-cost, end-to-end digital twin scheduling system to be implemented and applied.

[0077] Based on the complete technical chain of data fusion, twin mapping, algorithm scheduling, instruction execution, and feedback optimization described above, the system provided by this invention also constructs a four-dimensional integrated management system that enables the entire logistics scenario to be visible, manageable, controllable, and predictable. Specifically, a digital twin layer enables full-scenario visibility in logistics; a visual control layer enables full-chain management; an algorithm scheduling layer enables full-stage control; and the predictive capabilities of the algorithm scheduling layer enable predictable operational status of logistics nodes, thereby achieving intelligent management and control throughout the entire chain and lifecycle.

[0078] In specific embodiments, the end-to-end dynamic scheduling algorithm can replace the reinforcement learning DQN framework with the PPO framework. Both algorithms use logistics efficiency and operating costs as core reward functions, and autonomously optimize scheduling strategies through different reinforcement learning algorithms to achieve the same scheduling effect. Regarding deployment, the system can replace cloud-native deployment with a local lightweight deployment. By simplifying system code and model parameters, the configuration requirements of local servers are reduced, meeting the needs of small and medium-sized logistics enterprises requiring cloud-free deployment. In terms of visualization technology, the visualization control layer can replace the B / S architecture with a C / S architecture. Client software enables the visualization of the entire logistics chain data, ensuring data security while achieving the same integrated two-dimensional and three-dimensional display effect.

[0079] Further reference Figure 2 As an implementation of the above method, this invention also proposes a dynamic scheduling system for the entire intelligent logistics chain based on digital twins. This system can be specifically applied to various electronic devices. The system includes a physical sensing layer, a data fusion layer, a digital twin layer, and an algorithm scheduling layer.

[0080] The physical sensing layer is configured to collect physical operation data and business process data of the entire logistics scenario in real time through deployed sensing devices, and upload the physical operation data and business process data to the data fusion layer through communication protocols.

[0081] The data fusion layer is configured to use ETL tools to clean, convert, match, and align the received physical operation data and business process data to form a unified standard dataset, and then synchronize the standard dataset to the digital twin layer and the algorithm scheduling layer respectively.

[0082] The digital twin layer is configured to acquire static modeling data of logistics scenarios. Based on the static modeling data and standard datasets, a multi-level virtual mapping model of the entire logistics scenario is constructed using a hierarchical modeling method. A real-time data synchronization protocol is used to achieve bidirectional real-time data synchronization and accurate mapping between physical and virtual spaces to obtain a virtual model. Simultaneously, scenario simulation is performed based on the virtual model to obtain simulation results, which are then synchronized to the algorithm scheduling layer.

[0083] The algorithm scheduling layer is configured to take standard datasets and simulation results as inputs, and transform the logistics end-to-end scheduling problem into a Markov decision process based on reinforcement learning algorithms. With logistics operation efficiency and operating cost as objective functions, it iteratively optimizes the global scheduling strategy and generates executable scheduling instructions covering all aspects of transportation, warehousing and last-mile delivery.

[0084] In a specific embodiment, the intelligent logistics full-link dynamic scheduling system based on digital twin of the present invention further includes a visualization control layer, which is configured to visualize and distribute executable scheduling instructions to the physical devices and execution terminals corresponding to the physical perception layer.

[0085] The following is for reference. Figure 3 It shows a schematic diagram of the structure of a computer system 300 suitable for implementing terminal devices or servers of the present invention. Figure 3 The terminal device or server shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of the present invention.

[0086] like Figure 3 As shown, the computer system 300 includes a central processing unit (CPU) 301, which can perform various appropriate actions and processes based on programs stored in read-only memory (ROM) 302 or programs loaded from storage section 308 into random access memory (RAM) 303. The RAM 303 also stores various programs and data required for the operation of the computer system 300. The CPU 301, ROM 302, and RAM 303 are interconnected via a bus 304. An input / output (I / O) interface 305 is also connected to the bus 304.

[0087] The following components are connected to I / O interface 305: an input section 306 including a keyboard, mouse, etc.; an output section 307 including a liquid crystal display (LCD) and speakers, etc.; a storage section 308 including a hard disk, etc.; and a communication section 309 including a network interface card such as a LAN card and a modem, etc. The communication section 309 performs communication processing via a network such as the Internet. A drive 310 is also connected to I / O interface 305 as needed. A removable medium 311, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on drive 310 as needed so that computer programs read from it can be installed into storage section 308 as needed.

[0088] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 309, and / or installed from removable medium 311. When the computer program is executed by central processing unit (CPU) 301, it performs the functions defined in the methods of this invention. It should be noted that the computer-readable medium described in this invention can be a computer-readable signal medium or a computer-readable medium or any combination thereof. The computer-readable medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this invention, a computer-readable medium can be any tangible medium containing or storing a program that can be used by or in connection with an instruction execution system, apparatus, or device. In this invention, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can also be any computer-readable medium other than a computer-readable medium that can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on a computer-readable medium can be transmitted using any suitable medium, including but not limited to: wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0089] Computer program code for performing the operations of this invention can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0090] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0091] The above description is merely a preferred embodiment of the present invention and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of the invention is not limited to the specific combination of the above-described technical features, but also includes other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the inventive concept. For example, technical solutions formed by substituting the above-described features with (but not limited to) technical features with similar functions disclosed in this invention.

Claims

1. A dynamic scheduling method for the entire intelligent logistics chain based on digital twins, characterized in that, Includes the following steps: S1. Real-time collection of physical operation data and business process data of the entire logistics scenario through sensing devices deployed in the physical sensing layer, and uploading the physical operation data and business process data to the data fusion layer through communication protocols; S2. The data fusion layer uses ETL tools to clean, convert, match, and align the received physical operation data and business process data to form a unified standard dataset, and then synchronizes the standard dataset to the digital twin layer and the algorithm scheduling layer respectively. S3. The digital twin layer acquires static modeling data of the logistics scenario. Based on the static modeling data and the standard dataset, a multi-level virtual mapping model of the entire logistics scenario is constructed using a hierarchical modeling method. A real-time data synchronization protocol is used to achieve bidirectional real-time data synchronization and accurate mapping between the physical space and the virtual space to obtain a virtual model. Simultaneously, scenario simulation is performed based on the virtual model to obtain simulation results. The simulation results are then synchronized to the algorithm scheduling layer. S4. The algorithm scheduling layer takes the standard dataset and the simulation results as input, and transforms the logistics end-to-end scheduling problem into a Markov decision process based on the reinforcement learning algorithm. With logistics operation efficiency and operating cost as objective functions, it iteratively optimizes the global scheduling strategy and generates executable scheduling instructions covering the entire process of transportation, warehousing and last-mile delivery.

2. The intelligent logistics end-to-end dynamic scheduling method based on digital twins according to claim 1, characterized in that, Following step S4, the following steps are also included: S5. The executable scheduling instruction is sent to the physical devices and execution terminals corresponding to the physical sensing layer, and the on-site personnel and equipment execute the executable scheduling instruction. S6. After execution, the execution results and feedback data of the executable scheduling instructions are collected in real time through the physical perception layer and transmitted back. After standardization processing by the data fusion layer, the data is synchronized to the digital twin layer to update the virtual model. At the same time, it is pushed to the algorithm scheduling layer to update the reinforcement learning algorithm parameters through the experience replay mechanism, thereby completing the autonomous optimization of the reinforcement learning algorithm.

3. The intelligent logistics end-to-end dynamic scheduling method based on digital twins according to claim 1, characterized in that, In step S1, the physical operation data and the business process data are uploaded to the data fusion layer via a communication protocol, including the following sub-steps: S11. The IoT sensors and GPS positioning terminals of the physical sensing layer use the MQTT protocol to report the collected physical operation data to the data fusion layer. S12. The data fusion layer uses the OPC UA protocol to interface with the WMS warehouse management system and the TMS transportation management system to obtain the business process data and upload it to the data fusion layer.

4. The intelligent logistics end-to-end dynamic scheduling method based on digital twins according to claim 1, characterized in that, In step S2, the data fusion layer uses ETL tools to clean, convert formats, perform correlation matching, and timeline alignment on the received physical operation data and business process data to form a unified standard dataset, including the following sub-steps: S21. The data fusion layer cleans the received physical operation data and business process data, removes missing fields and abnormal data, and obtains cleaned data; wherein, the physical operation data includes logistics equipment operating parameters and vehicle operating conditions, and the business process data includes delivery node progress, warehouse inventory data and cargo physical characteristic information; S22. The cleaned data is converted into a unified data format to obtain unified format data; S23. Perform association matching on the unified format data: Using the globally unique order number as the primary key, associate the inbound and outbound documents and warehouse inventory data of the WMS warehouse management system, the transportation waybill data of the TMS transportation management system, the dispatch data of last-mile delivery, and the transportation trajectory data of the GPS positioning terminal through the primary key. Using the unique device ID as the secondary key, bind the real-time collected data of IoT sensors, GPS positioning terminals, warehouse barcode scanning devices, and vehicle ECUs with the corresponding transportation vehicles, warehouse equipment, and delivery terminals through the secondary key. Then, complete the association matching with the business process data through the primary key bound to the device to obtain the data that has completed the association matching. S24. Using the collection timestamp of the GPS positioning terminal as the global reference time axis, the data that has been associated and matched is time-aligned on the global reference time axis to obtain the time-aligned data and form a unified standard dataset.

5. The intelligent logistics end-to-end dynamic scheduling method based on digital twins according to claim 1, characterized in that, In step S3, the digital twin layer acquires static modeling data of the logistics scenario. Based on the static modeling data and the standard dataset, a multi-level virtual mapping model of the entire logistics scenario is constructed using a hierarchical modeling method, including the following sub-steps: S31. The digital twin layer acquires static modeling data of the logistics scenario, the static modeling data including: spatial structure information, equipment shape information, operation process information and interaction relationship information; S32. Construct a geometric model based on the spatial structure information and equipment shape information in the static modeling data; S33. Based on the work process information and interaction relationship information in the static modeling data, construct a behavior model; S34. Construct a physical model based on the logistics equipment operating parameters and cargo physical characteristics information in the standard dataset; S35. Superimpose the geometric model, the behavioral model, and the physical model to obtain the multi-level virtual mapping model of the entire logistics scenario.

6. The intelligent logistics end-to-end dynamic scheduling method based on digital twins according to claim 1, characterized in that, In step S3, a real-time bidirectional data synchronization and precise mapping between the physical and virtual spaces is achieved through a real-time data synchronization protocol to obtain a virtual model, including: Under normal operating conditions, a timed trigger update method is adopted to acquire the logistics equipment operating parameters and vehicle operating conditions in the physical operating data, and the delivery node progress and warehouse inventory data in the business process data according to a preset synchronization cycle. The acquired logistics equipment operating parameters, vehicle operating conditions, delivery node progress and warehouse inventory data are then synchronized to the virtual model to obtain a virtual model that is consistent with the physical space in real time. When an abnormal event is detected, an event-triggered update method is used to obtain abnormal event data and synchronize the abnormal event data to the virtual model in real time to obtain the virtual model that accurately maps the abnormal operating state of the physical space, while triggering the emergency response of the algorithm scheduling layer.

7. The intelligent logistics end-to-end dynamic scheduling method based on digital twins according to claim 1, characterized in that, In step S4, the algorithm scheduling layer takes the standard dataset and the simulation results as input, and transforms the logistics end-to-end scheduling problem into a Markov decision process based on the reinforcement learning algorithm. Using logistics operation efficiency and operating cost as objective functions, iteratively optimizes the global scheduling strategy to generate executable scheduling instructions covering all aspects of transportation, warehousing, and last-mile delivery. This includes the following sub-steps: S41. Construct a state space by normalizing the logistics equipment operating parameters, vehicle operating conditions, delivery node progress, warehouse inventory data, and cargo physical characteristic information in the standard dataset and using them as the state vector input to the reinforcement learning algorithm. S42. Construct an action space, and discretize all executable scheduling operations in the entire process to obtain the action vectors output by the reinforcement learning algorithm. S43. Construct a reward function, quantify the objective function into a weighted combination of logistics operation efficiency and operating cost, and evaluate the merits of the scheduling strategy output by the reinforcement learning algorithm through the reward function; S44. Using the DQN algorithm framework, based on the state space, the action space, and the reward function, the Q-value network parameters are iteratively updated to output a globally optimal scheduling strategy, and the executable scheduling instruction is generated according to the globally optimal scheduling strategy.

8. A dynamic scheduling system for the entire intelligent logistics chain based on digital twins, used to implement the method as described in any one of claims 1 to 7, characterized in that, The system includes: The physical sensing layer is configured to collect physical operation data and business process data of the entire logistics scenario in real time through deployed sensing devices, and upload the physical operation data and business process data to the data fusion layer through a communication protocol. The data fusion layer is configured to use ETL tools to clean, convert, match, and align the received physical operation data and business process data to form a unified standard dataset, and then synchronize the standard dataset to the digital twin layer and the algorithm scheduling layer respectively. The digital twin layer is configured to acquire static modeling data of the logistics scenario. Based on the static modeling data and the standard dataset, a multi-level virtual mapping model of the entire logistics scenario is constructed using a hierarchical modeling method. A real-time data synchronization protocol is used to achieve bidirectional real-time data synchronization and accurate mapping between the physical space and the virtual space to obtain a virtual model. Simultaneously, scenario simulation is performed based on the virtual model to obtain simulation results. The simulation results are then synchronized to the algorithm scheduling layer. The algorithm scheduling layer is configured to take the standard dataset and the simulation results as input, and transform the logistics end-to-end scheduling problem into a Markov decision process based on the reinforcement learning algorithm. With logistics operation efficiency and operating cost as objective functions, iteratively optimizes the global scheduling strategy and generates executable scheduling instructions covering the entire process of transportation, warehousing and last-mile delivery.

9. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and capable of running on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the intelligent logistics end-to-end dynamic scheduling method based on digital twins as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the intelligent logistics end-to-end dynamic scheduling method based on digital twins as described in any one of claims 1 to 7.

Images