A warehouse and distribution logistics integrated platform based on reinforcement learning dynamic scheduling

Abstract

The present application relates to the technical field of artificial intelligence logistics scheduling, in particular to a warehouse-distribution logistics integrated platform based on dynamic scheduling of reinforcement learning, which comprises a time allowance analysis module, a time sequence change recognition and corresponding table generation module, a urgency sorting module, a conflict task arrangement module, a scheduling performance monitoring module, an execution instruction generation module, and an execution task sequence obtained by associating the turning point to avoid platform conflicts. The present application constructs an order allowance structured index through time sequence changes and spatial mapping, optimizes the sorting by combining scheduling and loading collaborative verification, uses conflict recognition execution boundaries, dynamically reorganizes tasks according to offset turning points, realizes the synchronization and coordination of order organization and resource occupation, enhances the coherence of task arrangement and the adaptability of execution sequence, reduces repeated adjustment of work, and improves the continuous operation ability of warehouse-distribution collaboration.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence logistics scheduling technology, and in particular to an integrated warehousing and distribution logistics platform based on reinforcement learning dynamic scheduling. Background Technology

[0002] The field of artificial intelligence-based logistics scheduling technology encompasses a logistics resource allocation and operation control system centered on computer decision-making mechanisms. This includes order allocation, transportation route planning, collaborative warehousing operations, inventory status updates, and dynamic adjustments during delivery execution. The core of this technology lies in using state-aware data as a foundation to create a unified model of factors such as vehicle location, warehouse capacity, and time constraints at order demand transportation nodes. This forms a multi-node, multi-constraint scheduling system, which, combined with continuous time-series data, enables continuous updates to scheduling strategies, thereby constructing an integrated operational framework covering the entire process of warehousing, sorting, transportation, and delivery.

[0003] Among them, the warehousing and distribution logistics integrated platform based on reinforcement learning dynamic scheduling refers to a platform that uses an interactive decision-making process to uniformly schedule warehousing and distribution links through state input action selection and feedback updates. It is mainly constructed around specific elements such as order arrival time sequence, warehouse inventory status, current location of transport vehicles, delivery task priority, and time window constraints. By setting state variables including inventory quantity, vehicle load, path node order, and order completion progress, scheduling actions are defined as order allocation to designated warehouses, vehicle path node selection, loading order adjustment, and delivery order arrangement. Feedback evaluation rules are constructed based on task completion status and time consumption, thereby forming a continuously updated scheduling strategy in the continuous decision-making process, realizing the collaborative operation of warehousing and transportation links in the same decision-making process.

[0004] Existing technologies employ unified state modeling and continuous feedback updates for scheduling decisions. The decision results primarily reflect changes in the overall operational status, lacking fine-grained characterization of the relative temporal trends between orders. When order fulfillment pressure exhibits phased differences, these differences are difficult to translate into clear ranking criteria. Furthermore, the local correlation characteristics formed between different operational resources are difficult to retain in subsequent scheduling stages. For example, some orders may have similar spatial affiliations but different temporal status changes, and may still be included in similar scheduling positions. This leads to frequent order correction requirements during execution, causing fluctuations in task organization relationships and affecting the continuity and stability of subsequent scheduling arrangements. Summary of the Invention

[0005] The purpose of this invention is to address the shortcomings of existing technologies by proposing an integrated warehousing and distribution logistics platform based on reinforcement learning dynamic scheduling.

[0006] To achieve the above objectives, the present invention adopts the following technical solution: an integrated warehousing and distribution logistics platform based on reinforcement learning dynamic scheduling, comprising:

[0007] The time margin analysis module retrieves operation and delivery connection data by order number, filters orders by picking status, binds the aisle platform affiliation and scheduling time into records, identifies changes in time sequence, merges segments in the same direction into a status identifier sequence, associates order aisle platform affiliation, and obtains the order time margin correspondence table.

[0008] The urgency ranking module, based on the order time margin correspondence table, reads the status identifier arrangement position, generates an initial priority sequence, calls the vehicle loading record, queues them according to lane affiliation, compares the loading order and scheduling order of adjacent orders, adjusts them if they are inconsistent, and summarizes and retains the platform affiliation according to the initial sequence to obtain the order urgency ranking sequence.

[0009] The conflict task scheduling module reads adjacent order lanes and platforms according to the order urgency sorting sequence, determines conflicts in the same lane or platform, marks the conflict location and divides the execution segments, retains boundary nodes to be scheduled, and outputs candidate actions in the order of segments to obtain a set of executable delivery tasks.

[0010] The scheduling performance monitoring module, based on the set of executable delivery tasks, reads fragment orders and vehicle information, registers the order and dispatch results upon completion, backfills to form an execution record, compares the offset direction of adjacent fragments and marks the turning point, establishes a preceding association record, and obtains a scheduling performance change sequence.

[0011] The execution instruction generation module locates turning points based on the scheduling performance change sequence, reads the unfinished order lane platform, matches the segment list, writes the segments into the execution sequence, and continues the unmatched orders according to the order urgency sorting sequence. Based on the vehicle position and the conflict order of moving the occupied platform, the warehouse and distribution scheduling execution task sequence is obtained.

[0012] As a further aspect of the present invention, the order time margin correspondence table includes order time margin level, order fulfillment risk level, order timeliness margin range, and order scheduling concern category; the order urgency sorting sequence includes order priority sorting result, order urgency level classification, order sorting position information, and order scheduling order information; the executable delivery task set includes delivery task execution segments, delivery task execution order, delivery task resource allocation result, and delivery task collaboration relationship; the scheduling performance change sequence includes scheduling performance level, scheduling trend category, scheduling abnormal status information, and scheduling change characteristic information; and the warehouse and distribution scheduling execution task sequence includes task execution order, task distribution list, task resource allocation result, and task execution status identifier.

[0013] The working principle and advantages of this invention are as follows:

[0014] In this invention, a status identifier sequence is constructed by changing the temporal relationship between adjacent orders, and a mapping relationship is established between the characteristics of the changing segments and the spatial attribution information of the orders. This allows for a structured expression of the differences in order time margins. Combined with the collaborative verification of scheduling order and loading order, a sorting result that takes into account both timeliness and operational connection characteristics is formed. Execution segment boundaries are formed by identifying conflict locations, and task reorganization criteria are generated based on the execution offset state transition relationship. This enables subsequent execution sequences to have targeted adjustment capabilities, achieving synchronous coordination between order organization relationships and resource occupancy relationships. This enhances the coherence of task arrangement and the adaptability of execution sequences, reduces the phenomenon of repeated order adjustments during execution, and improves the stability and continuous operation capability of the warehousing and distribution collaboration process. Attached Figure Description

[0015] Figure 1 This is a platform flowchart of the present invention;

[0016] Figure 2 This is a flowchart illustrating the acquisition process of the time margin analysis module of the present invention.

[0017] Figure 3 This is a flowchart illustrating the urgency ranking module of the present invention.

[0018] Figure 4 This is a flowchart illustrating the acquisition process of the conflict task orchestration module of the present invention.

[0019] Figure 5 This is a flowchart illustrating the acquisition process of the scheduling performance monitoring module of the present invention;

[0020] Figure 6 This is a flowchart illustrating the process of obtaining the instruction generation module of the present invention. Detailed Implementation

[0021] The following detailed explanation illustrates the specific implementation methods:

[0022] The basic implementation examples are as follows: Figure 1 As shown: A warehousing and distribution logistics integrated platform based on reinforcement learning dynamic scheduling, the platform includes:

[0023] The time margin analysis module retrieves warehouse operation status data and delivery connection data using the order number as an index. It filters orders participating in scheduling based on order picking progress and vehicle arrival status. The filtered orders are bound to lane assignment, platform assignment, and scheduling time to form an order time sequence record. It identifies the first time sequence relationship between the order's expected delivery time and the scheduling time, and the second time sequence relationship between the order's fulfillment deadline and the order's expected delivery time. It compares the changing direction of the first and second time sequence relationships in adjacent order time sequence records along the scheduling time sequence. Records with continuous changes in the same direction are grouped into the same change segment. A status identifier sequence is generated according to the change segment order. The status identifier sequence is associated with the order number, lane assignment, and platform assignment to obtain the order time margin correspondence table.

[0024] The order time margin correspondence table includes the order time margin level, order fulfillment risk level, order timeliness margin range, and order scheduling concern category.

[0025] Please see Figure 2 Specifically, the time margin analysis module includes:

[0026] The order filtering submodule retrieves warehouse operation status data and delivery connection data using the order number as an index. It reads the order picking progress, vehicle arrival status, and current scheduling time. Order numbers whose picking progress is in a schedulable state and whose vehicle arrival status is registered are written into the reinforcement learning state representation. A set of orders participating in scheduling is formed according to the order number, and a scheduling admission set is generated. The details are as follows:

[0027] The system retrieves warehouse operation status data and delivery coordination data using the order number as an index. Specifically, it uses an Enterprise Application Programming Interface (API) to retrieve data from the distributed relational database of the Warehouse Management System (WMS) via SQL statements, obtaining information including the order number. Current picking operation progress percentage (Calculated based on the real-time feedback from the in-warehouse programmable logic controller (PLC) and SCADA system regarding the on / off status of the picking points, i.e., the proportion of currently completed lit-up workstations to the total lit-up workstations for that order), along with the geographical coordinates (latitude and longitude) and entry registration code of the vehicle. By jointly collecting data using UHF-RFID barrier antennas deployed in the logistics park and vehicle-mounted Beidou / GPS dual-mode terminals (by using the vehicle's unique VIN code in the system database to associate the RFID-read entry code with the coordinate data uploaded by Beidou / GPS via a foreign key), the current scheduling time of the system clock is synchronously obtained. By reading the picking operation progress percentage The floating-point value is used to set the job schedulable threshold. It is 0.85, if The value is If the picking progress is within the specified range, it is determined that the picking progress is in a schedulable state; otherwise, if... The value is Orders within the specified interval are deemed unschedulable. Unschedulable orders are placed in a delayed waiting queue, and their status is re-polled at the next system clock cycle (e.g., after 5 minutes), while simultaneously retrieving the vehicle entry registration code. The logic value for completing vehicle registration upon arrival is set to 1, and the logic value for incomplete registration is set to 0. This will satisfy the following conditions. and The order number is extracted and used as the input feature vector for the reinforcement learning state representation based on the DQN (Deep Q-Network) architecture. For example, its picking progress The value is 0.92, vehicle registration code. If the value is 1, the judgment condition is met, then the [condition] will be [set]. Mapped to the first dimension of a multidimensional tensor, items that have timed out and do not meet the conditions will be pushed into the delay queue. Orders with a value of 0.45 are removed and transferred to the anomaly monitoring dashboard. Then, all eligible order numbers are sorted in ascending order according to the numerical part after removing the letter prefix, and sequentially filled into a predefined fixed-length scheduling candidate matrix. When a position in the matrix is ​​empty, a placeholder value of -1 is assigned. This is achieved by iterating through the matrix. The matrix is ​​then processed, placeholder values ​​are removed, and the remaining order numbers are... Non-repetitive summation is performed to form a set of orders participating in scheduling, and a scheduling admission set is generated.

[0028] The time-series relation submodule, based on the scheduling admission set, calls the order number, roadway assignment, platform assignment, scheduling time, estimated delivery time, and fulfillment deadline time. It binds the order number with the roadway assignment, platform assignment, and scheduling time to form an order time-series record, identifies the order-series status of the estimated delivery time relative to the scheduling time, and the order-series status of the fulfillment deadline time relative to the estimated delivery time, and obtains the margin relation sequence; specifically as follows:

[0029] Based on the scheduling admission set, call the order number. The corresponding physical aisle numbers inside the warehouse Delivery docking platform number , preset system scheduling clock cycle time The Transportation Management System (TMS) uses route planning algorithms to provide estimated delivery times. Contractual deadlines with ERP systems The time data is uniformly converted into absolute minutes (in min) starting from midnight of the current day. For example, 10:30:00 is converted into 630 minutes. This is done by analyzing individual orders. of minus Perform the subtraction operation to obtain the first time-series absolute difference. ,like It is then identified as a "post-state" and the first state flag is set. If the value is assigned to 1, It will be identified as the "previous state" and will Assigning a value of 0 identifies the order's expected delivery time relative to the scheduling time, and then... minus Perform the second subtraction operation to obtain the second time-series absolute difference. ,like It is then identified as a "safe state" and the second state flag is set. If the value is assigned to 1, It will be identified as "overdue" and will The value is assigned to 0 to identify the order of the fulfillment deadline relative to the expected delivery time. The order status assignments are then processed according to... The calculation process uses a combination formula, which encapsulates the quintuple form. The two binary status bits are combined into a single decimal status identifier (output field {0, 1, 10, 11}, dimensionless). Indicates order The combined time series feature values ​​are used to perform the above iterative operation on all orders in the admission set to obtain the margin relation sequence.

[0030] The segment mapping submodule calls the margin relation sequence to read the time sequence records of adjacent orders in the scheduling time order. It compares the direction of change of the states of adjacent orders in the reinforcement learning state transition, groups the continuous change records with the same direction of change into the same change segment, generates a state identifier sequence according to the order of change segments, and associates the state identifier sequence with the order number, lane assignment, and platform assignment to obtain the order time margin mapping table; as detailed below:

[0031] Call the margin relation sequence to read the first time-series absolute difference in the time-series records of adjacent orders in the order of scheduling time. absolute difference with the second time series and status code Calculate the rate of change of the first time-series difference between adjacent orders. (Unit: minutes) and the rate of change of the difference between the second and third time series Set direction determination variable ,like and but This is denoted as "increasing in the same direction". and but This is denoted as "decreasing in the same direction". If the two rates of change are one positive and one negative or contain zero, then... This is denoted as "anomaly in urgency." In engineering physics, "anomaly in urgency" represents an increase in the transit time of subsequent orders while the allowed delivery buffer time is shortened. This signifies a sudden, abrupt change in the system's urgency. (Comparing adjacent order pairs...) and The value is determined by five consecutive adjacent order pairs. If all five orders are "increasing in the same direction", then the order pairs covered by these five orders will be included. to Divided into change sections And assign a hexadecimal status identifier code, for example The corresponding code is 0x01. If subsequent orders are correct... and of The change transforms into "anomaly in urgency," and due to the abrupt change in urgency, it is immediately truncated at this point, initiating a new phase of change. The code 0x02 is assigned. If no abnormal fluctuations occur subsequently, the sequence is forcibly truncated to the maximum segment size (e.g., 10 orders). The order number is matched with the corresponding lane number by performing a linear scan of the sequence from beginning to end. Platform number Associating them according to their respective change segment codes, forming, for example... The structured records are formatted and ultimately all structured records are aggregated and stored in a Redis in-memory key-value pair hash mapping table to obtain the order time margin mapping table.

[0032] The urgency ranking module, based on the association in the order time margin correspondence table, reads the position of the status identifier of each order in the status identifier sequence, generates an initial priority sequence of orders according to the position, calls the vehicle loading sequence record, assigns the orders in the initial priority sequence to the corresponding lane queue according to lane affiliation, and determines the loading order relationship of adjacent orders within the same lane queue according to the vehicle loading sequence record. It compares the loading order relationship with the scheduling order relationship in the initial priority sequence. When the loading order relationship of adjacent orders is inconsistent with the scheduling order relationship, the order of adjacent orders is adjusted until the order order in each lane queue simultaneously satisfies the scheduling order relationship and the loading order relationship. The adjusted lane queues are summarized according to the order of appearance in the initial priority sequence, and the platform affiliation of the order is retained to obtain the order urgency ranking sequence.

[0033] The order urgency sorting sequence includes the order priority sorting result, the order urgency level classification, the order sorting position information, and the order scheduling order information.

[0034] Please see Figure 3 Specifically, the urgency ranking module includes:

[0035] The position reading submodule, based on the association relationship in the order time margin correspondence table, reads the order number, status identifier sequence, and status identifier arrangement position. It writes the order number's corresponding arrangement position into the reinforcement learning scheduling state, compares the order arrangement positions, arranges the order numbers from front to back according to their arrangement positions, and records the corresponding roadway and platform affiliation for each order, generating an initial order priority sequence; specifically as follows:

[0036] Based on the relationships in the order time margin table, the encapsulated structured records are extracted from the Redis cache register, including the order number. Status identifier code 0x01, tunnel number and platform number The key-value pair data, including the data itself, is processed by reading the starting index of the changed segment on the global concatenation sequence timeline (by summing the total number of orders in all segments before the current changed segment as the global order offset, and adding the segment index), to determine the global physical location index of all orders. , will obtain Numerical mapping to observation vectors in the state space of a reinforcement learning DQN model The first feature dimension retrieves the array of permutation positions of the orders to be processed. Perform an ascending order reordering calculation on the values ​​in the array. Then determine the order Prior to orders in terms of time logic Let's illustrate with a practical example, if The position index is 1. If the position index is 3, then... Place it at the beginning of the sequence, followed by the corresponding roadway attribution data item. Platform attribution data item The data is reassembled and associated according to the order number, and then stored in contiguous memory blocks in ascending order of position index. When a position index conflict occurs (i.e.,...),... When reading the first time-series absolute difference, Perform a second comparison, and Those with lower values ​​are ranked higher to ensure priority for urgent orders. If they are still equal, they are sorted according to the order creation timestamp. The generated sequence is stored as a fixed-length structure array containing the order number, lane code, and platform code, thus generating the initial priority sequence of orders.

[0037] The lane correction submodule calls the initial order priority sequence and vehicle loading order record, reads the order number, lane affiliation, and vehicle loading registration order, assigns the orders to the corresponding lane queue according to lane affiliation, and determines whether the loading order relationship of adjacent orders in the same lane queue is consistent with the scheduling order relationship. Inconsistent adjacent orders are added to the reinforcement learning action candidate list to obtain the lane coordination queue; the details are as follows:

[0038] The system retrieves the initial order priority sequence and vehicle loading order records, and reads each order from the delivery task database. These orders are pre-calculated using a 3D packing algorithm based on volume constraints (length, width, height) and maximum load capacity limits. Corresponding physical stacking order index inside the vehicle (An integer sequence generated according to the inside-out stacking principle, with smaller numbers placed further inside), based on orders , , All belong to alleyways For example, extract all order subsets for that lane from the initial priority sequence, according to lane number. As a filtering condition, it is classified into an independent roadway queue vector. And maintain their relative order in the initial priority sequence, read the logical position in the vehicle loading registration order, and set... (This indicates that the order needs to be loaded by the second person.) (Indicates that the first item inside the carriage is loaded), and at the same time, read the scheduling order in the initial priority sequence. The sequence number and The sequence number For adjacent orders, perform logical judgments and calculate the loading scheduling coordination coefficient formula. (Dimensionless coefficient, its engineering significance lies in determining whether the loading and anti-pressure sequence matches the dispatching and outbound sequence), where , The index of the adjacent order position in the queue, if the calculation result This indicates a discrepancy between the loading order and the scheduling order (i.e., orders that leave the warehouse first are instead loaded into the deepest part of the carriage, causing overturning and accumulation on the platform). In this case, the corresponding orders are... As a conflict characteristic value;

[0039] That Value If the order is in an inconsistent state, the feature identifier of the conflicting order pair will be written into the action space candidate set of the reinforcement learning model. Perform a traversal and comparison on all lane queues; if... If there is no conflict, the original order is maintained. For inconsistent order pairs, their relative offset in the roadway queue and the physical coordinates of the roadway to which they belong are recorded to obtain the roadway coordination queue.

[0040] The sequence aggregation submodule, based on the roadway coordination queue, reads adjacent orders and reinforcement learning action candidates within each roadway queue. It adjusts the order of adjacent orders whose loading and scheduling relationships are inconsistent, retains the platform affiliation of each order, and aggregates the roadway queues according to the order of appearance in the initial priority sequence to obtain the order urgency ranking sequence; specifically as follows:

[0041] Based on the tunnel coordination queue, read each The vector stores adjacent order number pairs and their corresponding action candidate sets. Conflicting indexes marked as inconsistent are identified as such. The order pair will be swapped, meaning the order at the end of the queue will be swapped. Priority orders that are in the lead time During the storage unit address exchange, the platform affiliation data corresponding to the order is temporarily stored by declaring intermediate pointer variables. and To ensure that the mapping relationship of physical platforms moves synchronously with the order number, a consistency judgment threshold for order adjustment is set. The value is set to 0.95, and a maximum iteration threshold is set to prevent system deadlock. The loading scheduling coordination coefficient is recalculated repeatedly. If the proportion of conflict-free orders in the adjusted sequence is higher than this threshold, the result is considered valid. Or the number of iterations reaches Then, the iteration is forcibly stopped, and the original timestamps of the orders appearing in the initial priority sequence are used as a reference. The order of completion for each lane queue is then adjusted. , Rearrange and concatenate according to the smallest index value of its first order in the original sequence, for example The first order index is 1. If the first order index is 4, then output the first order. All members then continue output During the aggregation process, the members reread and retained the identified platform affiliation number for each order node. All adjusted lane sub-queues are connected into a singly linked list containing global priority levels. By traversing the linked list and writing its node information into the final scheduling and sorting stack according to the first-in-first-out principle, the order urgency sorting sequence is obtained.

[0042] The conflict task orchestration module reads the lane and platform affiliations of adjacent orders based on the order arrangement order in the order urgency sorting sequence. It determines whether adjacent orders meet at least one of the conflict conditions of having the same lane affiliation or the same platform affiliation. If either conflict condition is met, the position between the adjacent orders is marked as a conflict position. Using the conflict position as a segment blocking point, the order urgency sorting sequence is continuously scanned. Consecutive orders that are not blocked by conflict positions and maintain their original arrangement order are grouped into the same execution segment. Orders located on both sides of the conflict position are kept as nodes to be orchestrated at the corresponding segment boundary. The module generates scheduling action candidates according to the order of appearance of the execution segment in the order urgency sorting sequence. It prioritizes outputting scheduling action candidates that do not involve conflict positions. For scheduling action candidates that involve conflict positions, they are output sequentially according to their original arrangement order, keeping the order of orders within each execution segment unchanged, forming an executable action sequence, and obtaining an executable delivery task set.

[0043] The set of executable delivery tasks includes delivery task execution segments, delivery task execution order, delivery task resource allocation results, and delivery task collaboration relationships.

[0044] Please see Figure 4 Specifically, the conflict task orchestration module includes:

[0045] The collision identification submodule, based on the order order sequence in the order urgency ranking sequence, reads the adjacent order numbers, lane assignments, and platform assignments. It writes the states where adjacent orders share the same lane assignment and the same platform assignment into the reinforcement learning state item. It then determines whether adjacent orders meet any conflict condition, records the source of the conflict condition and the original order sequence number, marks the positions between the conflicting orders as segment blocking points, and generates a list of blocking nodes. Specifically, as follows:

[0046] Based on the order order in the urgency sorting sequence, sequentially read adjacent order numbers from the memory stack. and Lane attribution field and and platform affiliation field and Determine by performing a logical equality operation. Is it equal to At the same time, perform logical equality checks. Is it equal to ,like Then, the lane belonging to the same state position is written into the first state position of the reinforcement learning state vector. Dimension and assign a value of 1, if Then assign a value of 0, if Then, the platform belonging to the same state bit is written into the first position of this vector. Dimension and assign a value of 1, if The value is then assigned to 0. Taking actual data as an example, the first order is read. Its alleyway is Platform Read the second order Its alleyway is Platform ,because and The system determines that the order simultaneously meets both the conditions of lane congestion conflict and platform queuing conflict. It extracts the original permutation numbers 1 and 2 in the urgent sequence, registers the conflict source code "11" at sequence index position 1 (the first 1 represents lane conflict, and the second 1 represents platform conflict), and marks index position 1 as the blocking point of the physical segment. Then continue reading With the third order , obtain , If the collision condition in the alleyway is met, the source code "10" is registered and the blocking point is marked. Set the coefficients of the conflict determination matrix. (Dimensionless), calculated using formulas The numerical value, where the ternary operator determines whether the strings are completely equal (returns 1 if equal, 0 if not). In engineering terms, when two properties are exactly the same... Representing a serious conflict, an identical... Representing a partial conflict, when The value is in the range When a conflict condition is identified, it is considered a hit condition when the two attributes are completely different. Determined to be conflict-free, all The adjacent position index coordinates, conflict type codes, and involved order numbers are uniformly written into an independently maintained list of blocking nodes.

[0047] The segment orchestration submodule calls the blocking node list and the order urgency sorting sequence, continuously scans the order numbers along the order arrangement order, and groups consecutive orders that are not cut off by segment blocking points and maintain their original arrangement order into the same execution segment. Orders on both sides of the conflict position are kept as nodes to be orchestrated at the corresponding segment boundary, and scheduling action candidates are generated according to the order of appearance of the execution segments in the order urgency sorting sequence; as detailed below:

[0048] Invoke the list of blocking nodes and the order urgency sorting sequence, and set a scan pointer pointing to the first address of the sequence. From the order sequence Start by incrementing the pointer offset one by one, read the current order number and its marking status in the list of blocked nodes, and when the pointer... An obstacle was detected when moving to index 1. This will trigger the preceding order at the blocking point. Execute a hard truncation instruction on the memory bus and encapsulate it into an independent execution fragment. ,because There is a risk of resource conflict on both sides. and Each node to be arranged is marked in the fragment. The pointer continues to move to index 2 to detect the following boundary and the starting boundary of the subsequent segment. ,Will Encapsulated as an execution fragment When the pointer moves to index 3 and subsequent positions, as shown below... , , If no blocking marker is detected, this segment of consecutive orders will maintain its original order. Included in execution fragment During the scanning process, the quantity of orders contained in each segment is read. ,like If the segment capacity exceeds the preset threshold of 5 (determined by the maximum number of pallets that the AGV can carry at one time), then forced segmentation logic is executed, that is, a virtual blocking point is forcibly inserted at index 5, and the remaining orders are pushed into the next segment. In The intervals retain the existing natural logical segmentation results, and are arranged according to the physical addresses of each execution segment in the original urgent sorting sequence, in the following order: , , Assign priority labels to scheduling actions Extract the node number to be scheduled and its corresponding lane and platform physical attributes stored at the boundary of each segment. Package the segment identifier, the order list within the segment, and the boundary conflict node information into JSON format and combine them into a feature tensor. Then, send it into the experience replay buffer of the DQN reinforcement learning model and generate scheduling action candidates according to the order in which the execution segments appear in the order urgency sorting sequence.

[0049] The action output submodule, based on the scheduled action candidate, reads the execution segment, the node to be arranged, the lane assignment, and the platform assignment. It determines whether the scheduling action candidate is associated with conflicting locations, prioritizing the output of scheduling action candidates with unassociated conflicting locations. For scheduling action candidates with associated conflicting locations, it outputs them sequentially according to their original arrangement, maintaining the order within the execution segment, thus forming an executable action sequence and obtaining a set of executable delivery tasks. Specifically:

[0050] Based on the scheduling action candidates, execution fragments are read from the scheduling pool. The content, the list of nodes to be arranged, and the affiliation of each associated alleyway. Platform ownership By retrieving the conflict codes from the blocking node list, it is determined whether the orders involved in each scheduling action candidate are associated with conflict locations. For segments without blocking markers... It was determined to be a clean action candidate without any associated conflict positions. Since the estimated delivery time slack for all orders within this segment is greater than 30 minutes (confirmed to have no risk of overdue delivery), the system scheduling kernel executed the priority output instruction, placing its internal orders... Write the executable action sequence into the front address space in the original relative order, and then handle any conflicting locations. and Read the original sequence numbers 1 and 2 of the records, and output them sequentially in ascending order of sequence number. In and In Arranged in order After the sequence is generated, a sequence index comparison operation is used to confirm that the order order within each segment has not been shifted or disordered during the writing process to the buffer. The final sequence integrity checksum formula is then calculated. (That is, calculate the algebraic sum of all extracted pure numeric IDs), compare it with the benchmark checksum calculated by the urgent sorting sequence. If the difference is 0, the sequence is considered successfully reassembled. If the difference is not 0, it means that the data was lost or misaligned during bus transmission or reassembly. At this time, the abnormal rollback mechanism is triggered, the currently executable action sequence is cleared and a re-encapsulation request is sent. After successful verification, the reassembled order number, the corresponding aisle picking path instruction (sent down to the WCS layer), and the platform delivery docking instruction are encapsulated into a unified standard delivery task data packet. The execution status bit of the task packet is set to "ready". The complete linked list of all logistics-related attributes is written into the system execution register to form an executable action sequence, resulting in a set of executable delivery tasks.

[0051] The scheduling performance monitoring module, based on the executable action sequence in the executable delivery task set, reads the order and vehicle information corresponding to each execution segment. After the scheduling of each execution segment is completed, it registers the completion result of the corresponding order and the vehicle departure result, and backfills the order completion registration order and the vehicle departure registration order to the corresponding execution segment to form a continuous execution record. Based on the execution segment order in the executable action sequence, it compares the offset direction of the order completion registration order and the vehicle departure registration order in adjacent execution segments relative to the execution segment order along the scheduling time sequence. When the offset direction changes from a forward state to a backward state, the corresponding execution segment is marked as a state inflection point. The execution segment corresponding to the state inflection point is associated with the adjacent preceding execution segment to form a corresponding record between the execution segment, registration result and execution offset state change, thus obtaining the scheduling performance change sequence.

[0052] The scheduling performance change sequence includes scheduling performance level, scheduling trend category, scheduling abnormal state information, and scheduling change characteristic information.

[0053] Please see Figure 5 Specifically, the scheduling performance monitoring module includes:

[0054] The backfill registration submodule, based on the executable action sequence in the executable delivery task set, reads the order and vehicle information corresponding to each execution segment. After each execution segment completes scheduling, it registers the order completion result and vehicle departure result, backfills the order completion registration order and vehicle departure registration order to the corresponding execution segment, arranges the registration entries according to the scheduling time, and writes the execution segment number into the reinforcement learning status item to generate a continuous execution record; as detailed below:

[0055] Based on the executable action sequence in the set of executable delivery tasks, retrieve the encapsulated execution fragment. (Include ), (Include )as well as (Include ) and its associated delivery vehicles Real-time operational parameters are fed back at the Warehouse Execution Layer (WES) through the AGV control system and the PLC sensors at the picking station. Within the clip Picking was completed at 14:00. Completed at 14:05 Completed at 14:10 and corresponding vehicle When the vehicle departs at 14:15 via the electronic fence of the site, the absolute timestamp values ​​of each task completed are extracted;

[0056] Record the order completion registration order as follows: The vehicle departure registration order is recorded as follows Fill these floating-point time data back into The data carrier was then read. The overall completion timestamp of the segment is 14:25 and The overall completion timestamp of the segment is 14:20. All registered entries are sorted in ascending order according to the system clock sequence. The completion time of each segment registration relative to the initial scheduling time is calculated. Offset time formula (Unit: minutes) For example, its offset time is Minutes are used to map the unique identifier of each segment to the discrete variable space of the reinforcement learning state vector. (The total dimension of this vector is set to a fixed size, such as 10 dimensions, corresponding to the maximum number of concurrent segments.) If Actual precedes Upon completion, the first dimension of the state vector is set to 3 and the second dimension to 2. The completion time of each segment is encrypted using the RSA asymmetric encryption algorithm (the system server holds the private key for decryption, and the data collection end uses the public key for encryption) and written to a continuous system log address (to prevent data tampering). This generates a structured text containing timestamps, order sets, vehicle status, and execution priority, thus generating a continuous execution record.

[0057] The offset comparison submodule calls the continuous execution record, reads the execution fragment order, order completion registration order, vehicle departure registration order and scheduling time order, uses the execution fragment order in the executable action sequence as the fragment order benchmark, compares the forward and backward states of the adjacent execution fragment registration order relative to the fragment order benchmark along the scheduling time order, writes the offset direction into the reinforcement learning reward feedback term, and obtains the direction flow sequence.

[0058] The process of comparing the scheduling time sequence with the forward and backward states of adjacent execution segment registration order relative to the segment order reference is as follows:

[0059] The execution segment number in the execution segment sequence is compared with the order completion registration position in the order completion registration sequence and the vehicle departure registration position in the vehicle departure registration sequence. When the order completion registration position or the vehicle departure registration position is before the position corresponding to the execution segment number, it is marked as a forward state. When the order completion registration position or the vehicle departure registration position is after the position corresponding to the execution segment number, it is marked as a backward state.

[0060] The process of writing the offset direction into the reinforcement learning reward feedback item is as follows:

[0061] Read the forward and backward states corresponding to adjacent execution segments in the order of scheduling time. Mark records where the offset states of adjacent execution segments remain consistent as continuation in the same direction, and mark records where the offset states of adjacent execution segments change as direction changes. Write the direction change records into the reinforcement learning reward feedback item; as follows:

[0062] Invoke the continuous execution record and read the preset execution segment baseline order vector. Their corresponding logical position indices are respectively Extract the actual completed registration location at the fragment level from the backfill records. Let's illustrate with a practical example, if If a fragment is the second to complete registration out of all fragments, then its... By performing position coordinate subtraction operation (Dimensionless integer) yields the position offset, for Fragment calculation ,because And the result is in the negative integer range. Within this segment, the execution efficiency offset state is marked as "forward state" (representing premature completion) and assigned the value 1 (similar calculations are performed). of Treated as scheduled execution and assigned a logical value of 0), if another segment The actual registration position is 3, then ,because And the result is located in the positive integer range. Within this segment, mark it as "shifted state" (representing delay) and assign it the value of logic 2. If This is then considered as executing according to plan and assigning a logical value of 0. These state bits are combined into a state transition vector in chronological order, and the reinforcement learning reward calculation coefficient is introduced. (The default system learning rate scaling constant is 0.1, which is dimensionless).

[0063] Reward function determined by logic Perform the operation, when adjacent bits Set the decision factor at time It is determined to be a continuation in the same direction, when Set the decision factor at time The system is judged to have changed direction (the physical meaning of this setting is to encourage the system to maintain a continuous and stable execution cycle, and to severely punish frequent oscillations in the state direction), and the value is changed. The corresponding segment index number is written into the negative feedback channel of the reinforcement learning reward function as a penalty term parameter, thereby updating the hidden layer neural network weight distribution of the current DQN scheduling strategy through the backpropagation algorithm to obtain the directional flow sequence.

[0064] The transition association submodule reads adjacent execution segments, forward state, backward state and execution segment order according to the direction flow sequence, determines whether the offset direction changes from forward state to backward state, marks the hit execution segment as a state transition point, calls the execution segment corresponding to the state transition point and the adjacent preceding execution segment to establish an association relationship, registers the execution segment, registers the result, updates the mark and execution offset state changes through reinforcement learning, and obtains the scheduling performance change sequence.

[0065] The process of determining whether the offset direction has changed from a forward movement state to a backward movement state is as follows:

[0066] The offset state of adjacent execution segments is detected item by item along the directional flow sequence. When there is a continuous relationship where the previous execution segment is in a forward state and the next execution segment is in a backward state, the next execution segment is marked as a state inflection point. The execution segment corresponding to the state inflection point is then associated with the adjacent execution segments before that state inflection point according to the execution segment order. Specifically, as follows:

[0067] Based on the directional flow sequence, read adjacent execution segment pairs. and its associated offset status bits and Retrieve logical assignment records from memory to determine if a specific evolution sequence from a forward state to a backward state exists. This is demonstrated using actual data. In the forward-moving state ( And immediately following Switch to back-moving state ( If the combination matches the condition indicating a shift from a leading to a lagging system performance transition, then a logical matching operation is performed to confirm that the combination has met the condition. The segment's identifier and physical address are stored in the state transition point register and marked as... If no transition condition is met, skip this step and maintain the current scheduling queue; if a transition condition is met, call... Establish a doubly linked list relationship with its preceding data object to read data. Domestic orders Picking completion rate, vehicles Based on the system registration results such as punctuality rate, the formula for calculating the absolute value of the change in the execution position offset between adjacent segments is used. ,by of and of For example, calculate the absolute difference This value is greater than the preset fluctuation tolerance threshold. This triggers a forced update of the label bits in the reinforcement learning model parameters. Set it to 1 if the difference is small, otherwise set it to 0. The dynamic process description of the segment number, actual registration time, update mark and offset direction is encapsulated into a dynamic performance record in JSON format. By performing a cyclic scan on all segments, all vulnerable nodes that conform to the transformation logic are extracted, and the difference in work efficiency (number of completed orders / hour) before and after the transition is stored in the corresponding dimension of the performance evaluation matrix to obtain the scheduling performance change sequence.

[0068] The execution instruction generation module locates the execution segments marked as state inflection points and their segment order in the executable action sequence based on the corresponding records in the scheduling performance change sequence. It reads the order number, lane assignment, and platform assignment of the currently incomplete orders, matches the current incomplete orders with the execution segment order list associated with the state inflection point, writes the successfully matched incomplete orders into the execution sequence according to the corresponding segment order, and writes the unmatched incomplete orders into the execution sequence according to the original order in the order urgency sorting sequence. It reads the current vehicle position and platform occupancy registration record, and determines whether the order platform assignment corresponds to an occupied platform along the execution sequence. If it corresponds to an occupied platform, it moves the order to the position of the first different platform assignment in the subsequent order, while keeping the relative order of the remaining orders unchanged. It writes the adjusted execution sequence back to the state mapping relationship to obtain the warehouse and distribution scheduling execution task sequence.

[0069] The task sequence for warehouse and distribution scheduling includes the task execution order, task distribution list, task resource allocation results, and task execution status identifier.

[0070] Please see Figure 6 Specifically, the instruction generation module includes:

[0071] The transition location submodule, based on the corresponding records in the scheduling performance change sequence, reads the state transition point marker, execution segment number, executable action sequence, and segment order. It writes the execution segment corresponding to the state transition point into the reinforcement learning state item, determines the position of the execution segment number in the executable action sequence, registers the order list corresponding to the state transition point and the adjacent preceding segment number, and generates a transition segment list; specifically as follows:

[0072] Based on the corresponding records in the scheduling performance change sequence, the execution segment numbers marked as state inflection points are read from the system's core storage array. Its baseline segment sequence index 2 on the logical scheduling timeline calls the encapsulated executable action sequence. The sequence is linearly scanned using pointers to retrieve the desired information. The corresponding order list includes the order number. Simultaneously, extract the adjacent preceding execution segment numbers in the time evolution sequence. and its included order subset ,Will The fragment identifiers and transition state logical features are written into the reinforcement learning state tensor as state independent variables. The fifth dimension space is used to read the offset time data of this fragment during actual execution. Define a judgment range for scheduling fluctuation levels. The physical engineering significance of this range is based on logistics industry operating standards: If In minute value The range is determined to be normal execution fluctuation within the regular loading and unloading buffer time. The interval is considered a moderate offset (requiring local scheduling adjustments). If it exceeds the 60-minute limit of the departure window, it is considered a significant state transition (requiring a global recalculation). In the current calculation example... The actual offset time reached 235 minutes due to physical roadway blockage, a value far exceeding the lower limit for determining a significant turning point. The turning point segment was then extracted. Physical tunnel number to which it belongs With pre-assigned platform number Static properties, will Order list and preceding fragments The order list is subjected to a union operation, which generates a complete set of related orders. and preceding fragment identifier Encapsulate each order into a structured data record of fixed length, registering the job priority weight score, the entry registration timestamp of the delivery vehicle, and the current logical arrangement position index. Write this structured record into a dedicated scheduling transition descriptor register according to the descending order of scheduling time to generate a list of transition segments.

[0073] The order writing submodule calls the transition segment list, reads the current incomplete order number, roadway affiliation, platform affiliation, and the execution segment order list associated with the status transition point, and matches the current incomplete orders with the associated order lists item by item. Successfully matched orders are written into the execution sequence in segment order, while unmatched orders are written into the execution sequence in their original order according to the order urgency sorting sequence, resulting in a list of orders to be executed; details are as follows:

[0074] The system retrieves the order numbers currently in an incomplete state, such as "picked" or "pending shipment," by calling the WMS database real-time status query API. , and their respective lane affiliation codes Platform ownership code Associate currently incomplete orders with inflection points in the execution fragment order list. Perform a membership matching check by comparing the hexadecimal code stream of the order number digit by digit to determine if there are any duplicates. If the match is true, read the logical offset of the order in the original execution segment and write the order number back to its original position in the new dynamic execution sequence. The initial addressing segment is for deliveries that were returned by the system and marked as incomplete due to abnormal delivery. And newly added urgent orders The original permutation numbers 2 and 6 are read from the order urgency sorting sequence, and then sorted according to the ascending order of the numbers. and Continue writing execution sequence The remaining free storage space is used to calculate the overall execution priority index of the order during the write process using the following formula:

[0075] (Dimensionless score), where the roadway congestion coefficient is set based on sensor feedback. Time urgency coefficient based on customer service level ,Order System assessment of original urgency (Customer rating preset by the ERP system), absolute value of in-stock waiting time Minutes (obtained by subtracting the order creation timestamp from the current clock), calculated using algebraic addition and division. ,like The value is in The scope is then determined to be at the regular pending level; if it is in If an order is not processed, it will be automatically promoted to the priority level (queue-jumping). If the priority level is exceeded, an audible and visual alarm will be triggered and the order will be transferred to the manual scheduling queue to prevent program deadlock. All unfinished orders will be physically stored in the form of a linear linked list reconstructed according to priority, resulting in a list of orders to be executed.

[0076] The platform avoidance submodule, based on the list of orders to be executed, reads the current vehicle location, platform occupancy registration record, order number, and platform ownership. It then determines whether the order's platform ownership corresponds to an already occupied platform along the execution sequence. Orders hitting occupied platforms are moved to the first subsequent order with a different platform ownership, while maintaining the relative order of the remaining orders. The adjusted execution sequence is written back to the reinforcement learning state mapping relationship to obtain the warehouse distribution scheduling execution task sequence; specifically as follows:

[0077] Based on the list of pending orders, delivery vehicles are retrieved via a radio frequency identification (RFID) gateway. The current physical latitude and longitude coordinates within the station are retrieved, and the occupancy registration record logical bitmap uploaded by the millimeter-wave radar and infrared beam sensors deployed on the platform berths is read simultaneously. (An array of 0s and 1s, where a bit is set to 1 when the sensor detects a target vehicle for 5 consecutive seconds, and 0 otherwise.) Extract orders to be executed. Platform attribution field By accessing the bitmap The third index address is used to read the stored logic level signal. If the signal is 1, it is determined that the delivery platform corresponding to the order is currently occupied by another vehicle that is loading. If it is 0, the order is released normally. This applies to orders that meet the condition of an occupied platform. At the scheduling algorithm layer, a position shift addressing operation is performed, and subsequent orders are read sequentially from the list of orders to be executed in ascending order of address. Platform attribution field Through the and Execute an inequality check; if the result is true, then... The storage pointer is shifted and inserted into a non-conflicting location. The subsequent logic unit, and for An average platform loading time (e.g., 15 minutes) monitored in real-time by a system timer is added as an estimated delay window. Within this window, the system will continuously poll the platform's status bit. If the platform is not released by the end of the window, a secondary addressing instruction to re-enter the scheduling queue is triggered. Simultaneously, other orders are kept in the queue. and The relative storage offset remains unchanged to maintain the optimized topology order of the system, and the step size parameter for platform avoidance adjustment is set. (Unit is the number of order spans), if The value is in the integer range If it is determined to be a low-latency adjustment, it can be executed directly. If the interval is determined to be of medium delay and a warning needs to be marked, after adjustment, the probability of path conflict in the adjusted sequence is calculated using the following formula. ;

[0078] like Then the sequence fine-tuning and reshaping is deemed qualified. If the local avoidance strategy fails, the system will trigger an exception fallback branch, rejecting excess orders and temporarily storing them in a virtual buffer. After the adjustment is approved, the order number, path code, and platform allocation instructions that have been finally repositioned will be synchronously written back to the reinforcement learning state-action mapping table and submitted to the database to obtain the final warehouse and distribution scheduling execution task sequence.

[0079] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A warehouse and distribution logistics integrated platform based on reinforcement learning dynamic scheduling, characterized in that, The platform includes: The time margin analysis module retrieves operation and delivery connection data by order number, filters orders by picking status, binds the lane platform affiliation and scheduling time into records, identifies changes in time sequence, merges segments in the same direction into a status identifier sequence, associates the order lane platform affiliation, and obtains the order time margin correspondence table. The urgency ranking module, based on the order time margin correspondence table, reads the status identifier arrangement position, generates an initial priority sequence, calls the vehicle loading record, queues them according to lane affiliation, compares the loading order and scheduling order of adjacent orders, adjusts them if they are inconsistent, and summarizes and retains the platform affiliation according to the initial sequence to obtain the order urgency ranking sequence. The conflict task scheduling module reads adjacent order lanes and platforms according to the order urgency sorting sequence, determines conflicts in the same lane or platform, marks the conflict location and divides the execution segments, retains boundary nodes to be scheduled, and outputs candidate actions in the order of segments to obtain a set of executable delivery tasks. The scheduling performance monitoring module, based on the set of executable delivery tasks, reads fragment order and vehicle information, registers the order and dispatch results upon completion, backfills to form an execution record, compares the offset direction of adjacent fragments and marks the turning point, establishes a preceding association record, and obtains a scheduling performance change sequence.

2. The integrated warehousing and distribution logistics platform based on reinforcement learning dynamic scheduling according to claim 1, characterized in that: The order time margin correspondence table includes order time margin level, order fulfillment risk level, order timeliness margin range, and order scheduling concern category. The order urgency sorting sequence includes order priority sorting result, order urgency level classification, order sorting position information, and order scheduling order information. The executable delivery task set includes delivery task execution segments, delivery task execution order, delivery task resource configuration result, and delivery task collaboration relationship. The scheduling performance change sequence includes scheduling performance level, scheduling trend category, scheduling abnormal state information, and scheduling change characteristic information.

3. The integrated warehousing and distribution logistics platform based on reinforcement learning dynamic scheduling according to claim 1, characterized in that: The time margin analysis module includes: The order filtering submodule retrieves warehouse operation status data and delivery connection data using the order number as an index. It reads the order picking progress, vehicle arrival status and current scheduling time, writes the order numbers of orders with picking progress in a schedulable state and vehicle arrival status registration completed into the reinforcement learning state representation, forms a set of orders to participate in scheduling according to the order number, and generates a scheduling admission set. The timing relationship submodule, based on the scheduling admission set, calls the order number, roadway ownership, platform ownership, scheduling time, estimated delivery time, and fulfillment deadline time, binds the order number with the roadway ownership, platform ownership, and scheduling time as an order timing record, identifies the order status of the estimated delivery time relative to the scheduling time and the order status of the fulfillment deadline time relative to the estimated delivery time, and obtains the margin relationship sequence. The segment mapping submodule calls the margin relationship sequence, reads the time sequence records of adjacent orders in the scheduling time order, compares the change direction of the state of adjacent orders in the reinforcement learning state transition, groups the continuous change records with the same change direction into the same change segment, generates a state identifier sequence according to the change segment order, and associates the state identifier sequence with the order number, lane assignment and platform assignment to obtain the order time margin correspondence table.

4. The integrated warehousing and distribution logistics platform based on reinforcement learning dynamic scheduling according to claim 1, characterized in that: The urgency ranking module includes: The position reading submodule reads the order number, status identifier sequence and status identifier arrangement position based on the association relationship in the order time margin correspondence table, writes the order number corresponding to the arrangement position into the reinforcement learning scheduling state, compares the order arrangement positions of different orders, arranges the order numbers from front to back according to the arrangement position, records the corresponding lane and platform affiliation of the order, and generates the initial priority sequence of orders. The lane correction submodule calls the initial priority sequence of orders and the vehicle loading order record, reads the order number, lane affiliation and vehicle loading registration order, assigns the orders to the corresponding lane queue according to lane affiliation, determines whether the loading order relationship and scheduling order relationship of adjacent orders in the same lane queue are consistent, writes inconsistent adjacent orders into reinforcement learning action candidates, and obtains the lane coordination queue. The sequence aggregation submodule reads adjacent orders and reinforcement learning action candidates in each lane queue according to the lane coordination queue, adjusts the order of adjacent orders whose loading order relationship is inconsistent with the scheduling order relationship, retains the platform affiliation of each order, and aggregates each lane queue according to the order of order appearance in the initial priority sequence to obtain the order urgency sorting sequence.

5. The integrated warehousing and distribution logistics platform based on reinforcement learning dynamic scheduling according to claim 1, characterized in that: The conflict task orchestration module includes: The collision identification submodule reads the adjacent order number, lane affiliation and platform affiliation according to the order arrangement order in the order urgency sorting sequence, writes the same lane affiliation and the same platform affiliation status into the reinforcement learning status item, determines whether adjacent orders hit any conflict condition, registers the source of the hit condition and the original order arrangement number, marks the position between the hit orders as segment blocking points, and generates a blocking node list. The segment arrangement submodule calls the blocking node list and the order urgency sorting sequence, continuously scans the order numbers along the order arrangement order, and groups the consecutive orders that are not cut off by the segment blocking point and maintain the original arrangement order into the same execution segment. The orders on both sides of the conflict position are kept as nodes to be arranged at the corresponding segment boundary, and scheduling action candidates are generated according to the order of appearance of the execution segment in the order urgency sorting sequence. The action output submodule, based on the scheduled action, reads the execution segment, the node to be arranged, the lane affiliation and the platform affiliation, determines whether the scheduling action candidate is associated with a conflicting position, prioritizes outputting scheduling action candidates for unassociated conflicting positions, and outputs scheduling action candidates for associated conflicting positions in the original order, keeping the order of orders within the execution segment unchanged, forming an executable action sequence, and obtaining an executable delivery task set.

6. The integrated warehousing and distribution logistics platform based on reinforcement learning dynamic scheduling according to claim 1, characterized in that: The scheduling performance monitoring module includes: The backfill registration submodule reads the order and vehicle information corresponding to each execution segment based on the sequence of executable actions in the set of executable delivery tasks. After each execution segment is scheduled, it registers the order completion result and vehicle departure result, backfills the order completion registration order and vehicle departure registration order to the corresponding execution segment, arranges the registration entries according to the scheduling time, and writes the execution segment number into the reinforcement learning status item to generate a continuous execution record. The offset comparison submodule calls the continuous execution record, reads the execution fragment order, order completion registration order, vehicle departure registration order and scheduling time order, uses the execution fragment order in the executable action sequence as the fragment order benchmark, compares the forward and backward states of the adjacent execution fragment registration order relative to the fragment order benchmark along the scheduling time order, writes the offset direction into the reinforcement learning reward feedback term, and obtains the direction flow sequence. The transition association submodule reads adjacent execution segments, forward state, backward state, and execution segment order according to the direction flow sequence, determines whether the offset direction changes from forward state to backward state, marks the hit execution segment as a state transition point, calls the execution segment corresponding to the state transition point and the adjacent preceding execution segment to establish an association relationship, registers the execution segment, registration result, reinforcement learning update mark, and execution offset state change, and obtains the scheduling performance change sequence.

7. The integrated warehousing and distribution logistics platform based on reinforcement learning dynamic scheduling according to claim 6, characterized in that: The process of comparing the forward and backward states of the scheduling time sequence registration order of adjacent execution segments relative to the segment order reference is as follows: The execution segment number in the execution segment sequence is compared with the order completion registration position in the order completion registration sequence and the vehicle departure registration position in the vehicle departure registration sequence. When the order completion registration position or the vehicle departure registration position is before the position corresponding to the execution segment number, it is marked as a forward state. When the order completion registration position or the vehicle departure registration position is after the position corresponding to the execution segment number, it is marked as a backward state. The process of writing the offset direction into the reinforcement learning reward feedback item is as follows: Read the forward and backward states corresponding to adjacent execution segments in the order of scheduling time. Mark the records where the offset states of adjacent execution segments remain consistent as continuation in the same direction, and mark the records where the offset states of adjacent execution segments change as direction change. Write the direction change records into the reinforcement learning reward feedback item.

8. The integrated warehousing and distribution logistics platform based on reinforcement learning dynamic scheduling according to claim 6, characterized in that: The process of determining whether the offset direction has changed from a forward shift to a backward shift is as follows: The offset state of adjacent execution segments is detected item by item along the directional flow sequence. When there is a continuous relationship where the previous execution segment is in a forward state and the next execution segment is in a backward state, the next execution segment is marked as a state inflection point. The execution segment corresponding to the state inflection point is associated with the adjacent execution segments before the state inflection point according to the execution segment order.

9. The integrated warehousing and distribution logistics platform based on reinforcement learning dynamic scheduling according to claim 1, characterized in that: The platform also includes: The execution instruction generation module locates turning points based on the scheduling performance change sequence, reads the unfinished order lane platform, matches the segment list, writes the segments into the execution sequence, and continues the unmatched orders according to the order urgency sorting sequence. Based on the vehicle position and the platform occupancy conflict order, the warehouse and distribution scheduling execution task sequence is obtained. The warehouse and distribution scheduling execution task sequence includes the task execution order, task distribution list, task resource allocation results, and task execution status identifier.

10. The integrated warehousing and distribution logistics platform based on reinforcement learning dynamic scheduling according to claim 9, characterized in that: The execution instruction generation module includes: The transition location submodule reads the state transition point marker, execution segment number, executable action sequence and segment order according to the corresponding record in the scheduling performance change sequence, writes the execution segment corresponding to the state transition point into the reinforcement learning state item, determines the corresponding position of the execution segment number in the executable action sequence, registers the order list corresponding to the state transition point and the adjacent preceding segment number, and generates a transition segment list. The order writing submodule calls the transition segment list, reads the current incomplete order number, roadway affiliation, platform affiliation and status transition point associated execution segment order list, matches the current incomplete orders with the associated order list item by item, writes the successfully matched orders into the execution sequence in segment order, and writes the unmatched orders into the execution sequence in the original order of the order urgency sorting sequence, to obtain the order list to be executed; The platform avoidance submodule, based on the list of orders to be executed, reads the current location of the vehicle, the platform occupancy registration record, the order number and the platform ownership. It determines whether the platform ownership of the order corresponds to an already occupied platform along the execution sequence, moves the order that hits an already occupied platform to the first different platform ownership position in the subsequent order, keeps the relative order of the remaining orders unchanged, and writes the adjusted execution sequence back to the reinforcement learning state mapping relationship to obtain the warehouse and distribution scheduling execution task sequence.

Images