Logistics scheduling planning method and system based on graph neural network and reinforcement learning

Abstract

The present application relates to the technical field of intelligent logistics scheduling, and particularly relates to a logistics scheduling planning method and system based on a graph neural network and reinforcement learning, comprising the following steps: S1: constructing a dynamic graph structure of a logistics network; S2: performing embedding learning on the dynamic graph structure through a graph attention network to extract a multi-dimensional feature vector of each node; S3: inputting the multi-dimensional feature vector into a multi-agent reinforcement learning framework to generate an initial vehicle path planning scheme; S4: dynamically correcting a road section traffic state in the initial vehicle path planning scheme; S5: iteratively updating the vehicle path planning scheme through local reinforcement learning; and S6: outputting a final cooperatively optimized cargo transportation trajectory and vehicle driving path. The present application can realize dynamic modeling of a logistics network structure and multi-agent path cooperative optimization, and has self-adaptive adjustment capability for real-time traffic and environmental changes, thereby improving overall scheduling efficiency.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent logistics scheduling, and in particular to a logistics scheduling planning method and system based on a graph neural network and reinforcement learning. BACKGROUND

[0002] With the rapid expansion of e-commerce and supply chain business scale, the number of nodes and path selection complexity of distribution centers and distribution networks have significantly increased. The cargo flow pattern between distribution nodes has obvious spatiotemporal dynamic changes, and traditional route planning based on static graph models cannot accurately reflect the correlation strength between nodes. Meanwhile, when multiple delivery vehicles are cooperatively scheduled in a large-scale network, there is a mutual restraining relationship between transportation cost, time consumption, and empty rate. Most existing logistics scheduling systems rely on predefined rules or empirical models to generate paths, lack the ability to extract high-dimensional features of network structure and vehicle operating state, and are difficult to adapt to sudden scheduling demands caused by real-time traffic congestion and weather changes.

[0003] Existing methods cannot deeply model the structural relationship between distribution nodes when dealing with large-scale, dynamically updated logistics networks, resulting in incomplete representation of node features. At the same time, a single agent or centralized optimization strategy cannot simultaneously optimize global transportation cost, delivery timeliness, and vehicle fleet load balancing, often resulting in extended transportation time or increased vehicle empty running rate. In addition, the adjustment of road segments in response to real-time traffic and weather risks relies heavily on manual rules, which is inefficient and lacks accuracy. Therefore, there is an urgent need for a logistics scheduling planning method and system based on a graph neural network and reinforcement learning to address the above problems. SUMMARY

[0004] Based on the above purpose, the present application provides a logistics scheduling planning method and system based on a graph neural network and reinforcement learning.

[0005] The logistics scheduling planning method based on a graph neural network and reinforcement learning comprises the following steps:

[0006] S1: Taking the latitude and longitude coordinates of distribution nodes as node features, and the cargo flow, historical transportation efficiency, and distance between distribution nodes as edge features, a dynamic graph structure of the logistics network is constructed;

[0007] S2: The dynamic graph structure is embedded and learned by a graph attention network to extract a multi-dimensional feature vector of each node and calculate the correlation weight between nodes;

[0008] S3: The multi-dimensional feature vector is input into a multi-agent reinforcement learning framework, each vehicle is defined as an independent agent, the global transportation cost, time cost, and empty rate are defined as joint optimization objectives, and an initial vehicle path planning scheme is generated;

[0009] S4: Real-time access to electronic map traffic state data and weather forecast information to dynamically correct the traffic state of the initial vehicle path planning scheme, and generate adaptive path adjustment instructions;

[0010] S5: Based on the adaptive path adjustment instructions, the multi-dimensional feature vector of the affected distribution node is recalculated, and the vehicle path planning scheme is iteratively updated through local reinforcement learning;

[0011] S6: Output the final collaborative optimized goods transportation trajectory and vehicle driving path, and synchronously update the node features and edge features of the dynamic graph structure, and complete the closed loop.

[0012] Optionally, the S1 specifically comprises:

[0013] S11: Collect the geographic coordinate information of all distribution nodes, and use the WGS-84 coordinate system to standardize the position of each distribution node, to obtain the position vector group in the node set;

[0014] S12: Statistically record the historical goods flow records between the distribution nodes, extract the average goods flow between each pair of nodes per unit time, and construct the flow feature matrix of the edge;

[0015] S13: Calculate the historical transportation efficiency between the distribution nodes, the transportation efficiency being the ratio of the average actual transportation time of unit goods in a specified time window to the theoretical shortest transportation time, denoted as the transportation efficiency matrix;

[0016] S14: Based on the latitude and longitude information between the distribution nodes, calculate the spherical distance between each pair of nodes using geodetic surveying method, and construct the distance matrix;

[0017] S15: Take the position vector group as the node feature, and jointly encode the flow feature matrix, the transportation efficiency matrix and the distance matrix as the edge feature, use the graph data structure to represent the logistics network, and form a dynamic graph structure supporting time update.

[0018] Optionally, the S2 specifically comprises:

[0019] S21: Input the logistics network dynamic graph structure constructed by S1 into the graph attention network, and take the original features of each distribution node as the node feature vector of the input data, and uniformly organize it into a node feature matrix;

[0020] S22: Linearly map the node features through the graph attention network, and convert the original feature dimension to the feature representation in the new high-dimensional embedding space through the set mapping weight matrix;

[0021] S23: For any one node in the graph, retrieve its adjacent node set in turn, and use the attention mechanism to combine the feature vectors between the node and each of its neighbor nodes; the combined feature vector will be inputted to calculate the relevance score with the set activation function and attention parameters, to obtain the importance coefficient of the node pair adjacent nodes;

[0022] S24: Normalize the importance coefficients of all adjacent nodes of the node to ensure that the sum of the attention weights of all adjacent nodes is a constant;

[0023] S25: After obtaining the attention weights between nodes, the mapping feature vectors of adjacent nodes are weighted and summed, and an output transformation is performed in combination with a nonlinear activation function, to finally obtain the multi-dimensional feature representation of each distribution node in the embedding space.

[0024] Optionally, the S3 specifically comprises:

[0025] S31: Integrate the multi-dimensional feature vectors of each distribution node output by S2 and the current task state into an environment state, and input it into a multi-agent reinforcement learning environment;

[0026] S32: In the multi-agent reinforcement learning environment, each transport vehicle is an independent agent, and each agent is configured with a local observation interface to read the current position node features, adjacent edge features, and vehicle load state;

[0027] S33: Take the total transportation cost, total time cost, and vehicle fleet empty rate as three optimization indicators to establish a joint reward function;

[0028] S34: Each agent selects the next target node from the action set according to the policy network and generates a vehicle access sequence, and all vehicle sequences are summarized to form an initial vehicle path planning scheme.

[0029] Optionally, the S33 specifically comprises:

[0030] S331: After each round of agent path decision is completed, the total transportation cost, total time cost, and total empty mileage of all vehicles in the current scheduling period are calculated; wherein the total transportation cost is the product sum of the actual running distance of all vehicles and the fuel consumption coefficient; the total time cost is the cumulative time consumption of all orders from the distribution starting point to the delivery end point; and the total empty rate is the average value of the ratio of the total empty mileage to the total mileage of all vehicles;

[0031] S332: Set the weight coefficients of the three indicators to represent the optimization tendency of the transportation cost, time cost, and empty rate, respectively, and construct a joint reward function by weighted combination, which is expressed as:

[0032] wherein, represents a joint reward value; represents a total transportation cost; represents a total time cost; represents a vehicle fleet empty rate; are positive real weight coefficients of transportation cost, time cost and empty rate, respectively;

[0033] S334: feeding back the joint reward value to all agents as a common reward signal of the action results of this round.

[0034] Optionally, the S34 specifically comprises:

[0035] S341: determining an action set for each agent, the action set including all optional actions from the current position to all adjacent distribution nodes and back to the starting node;

[0036] S342: inputting the current environment state of the agent into its policy network, and outputting a decision score of each action by the policy network;

[0037] S343: selecting the action with the highest score in the action set as the next target node according to the decision score, and recording the selected action into the access sequence of the agent;

[0038] S344: after selecting the action, updating the local state of the agent, including changing the current position, updating the cargo state and marking the task completion, and feeding back the updated local state to the environment;

[0039] S345: repeating S342 to S344 until all scheduled vehicles complete their delivery tasks, and forming an initial path planning scheme for each vehicle.

[0040] Optionally, the S4 specifically comprises:

[0041] S41: acquiring real-time traffic state data of each road segment through the accessed electronic map service interface, including the current passing speed and congestion level of the road segment, and acquiring the precipitation intensity, wind speed and road icing risk information in the future period through the weather data interface;

[0042] S42: mapping the traffic state data and weather forecast information to each road segment in the initial path planning scheme, and matching and identifying the passing performance and safety risk of each segmented path;

[0043] S43: for each segmented path, calculating a passing state score according to its passing performance and safety risk indicators , and comparing with a preset passing threshold value, when the passing state score is less than the preset passing threshold value ​When the time is reached, the corresponding segment is marked as a segment to be adjusted;

[0044] S44: For all segments marked as to be adjusted, retrieve a set of alternative paths, and select the optimal alternative route according to the shortest time increment and safety risk increment;

[0045] S45: Based on the alternative route selected in S44 and the latest road condition status of the affected distribution node, generate adaptive path adjustment instructions, including affected segment identification, alternative route sequence and recommended adjustment time.

[0046] Optionally, the S5 specifically includes:

[0047] S51: According to the adaptive path adjustment instructions generated in step S4, identify the affected path segment and the corresponding distribution node, and mark the feature vectors of these nodes in the graph neural network structure as to be updated;

[0048] S52: Extract the local subgraph of the affected node, including its adjacent nodes and connecting edges, and input the subgraph structure into the graph attention network for embedding learning to generate updated multi-dimensional feature vectors;

[0049] S53: Input the updated node features into the reinforcement learning environment, and only update the local scheduling of the affected path-related agents, while keeping the vehicle scheduling strategy unchanged for other unaffected paths;

[0050] S54: During the local reinforcement learning process, recalculate the action set of the affected agents, call the original policy network for policy correction, and use the adjusted environment feedback for policy iteration;

[0051] S55: After iterative convergence, output the updated vehicle path access sequence to replace the corresponding part of the original initial scheme, thereby completing the local update of the overall path planning scheme.

[0052] Optionally, the S6 specifically includes:

[0053] S61: Receive the driving trajectory data and cargo delivery trajectory data of all transport vehicles during the execution of the final path planning, and arrange the arrival time, departure time and loading state of each node in chronological order;

[0054] S62: Based on the trajectory data after execution, re-statistics the actual cargo flow and average residence time of each distribution node as new features of the node; At the same time, according to the actual driving distance and time consumption between nodes, recalculate the real-time transportation efficiency and segment reliability of each edge as new features of the edge;

[0055] S63: write the node new features and edge new features obtained in S62 into the dynamic graph structure, update the node feature matrix and edge feature matrix, and mark the time stamp of this update in the graph data storage;

[0056] S64: take the updated dynamic graph structure as the input of the next round of scheduling and embedding learning.

[0057] The logistics scheduling planning system based on the graph neural network and the reinforcement learning is used for realizing the logistics scheduling planning method based on the graph neural network and the reinforcement learning, and comprises the following modules:

[0058] The network construction module is used for collecting the geographic coordinates, the historical cargo flow, the transportation efficiency and the distance data between nodes of the distribution nodes, and constructing a dynamic graph structure of the logistics network supporting time update based on the collected data;

[0059] The embedding learning module is used for receiving the dynamic graph structure provided by the network construction module, performing node feature mapping and neighbor feature aggregation on the dynamic graph structure based on the graph attention network, so as to extract the multi-dimensional feature vectors of the distribution nodes and the attention weights between the nodes;

[0060] The reinforcement learning decision module is used for receiving the multi-dimensional feature vectors provided by the embedding learning module, modeling each transportation vehicle as an independent agent, constructing a joint reward function based on the global transportation cost, the global time cost and the vehicle fleet empty rate, and generating an initial vehicle path planning scheme by using a multi-agent reinforcement learning framework;

[0061] The real-time data acquisition module is used for acquiring the current road section passing speed, the congestion level and the future weather risk information from an electronic map service and a weather service;

[0062] The dynamic correction module is used for scoring and threshold comparison of the passing state of each road section in the initial vehicle path planning scheme based on the traffic state and weather risk information provided by the real-time data acquisition module, identifying the road sections needing adjustment and retrieving the alternative paths, and generating adaptive path adjustment instructions;

[0063] The path update module is used for updating the affected vehicle paths in the local subgraph range by reinforcement learning iteration according to the adaptive path adjustment instructions provided by the dynamic correction module and the node features provided by the embedding learning module, and feeding back the updated complete path to the network construction module to update the dynamic graph structure synchronously, so as to form a closed loop.

[0064] The logistics scheduling planning system based on the graph neural network and the reinforcement learning has the following beneficial effects:

[0065] The application can accurately extract the space-time correlation features between nodes by constructing a dynamic graph network based on a distribution node graph structure and introducing a graph attention network for embedding learning, and realize high-dimensional modeling of the topology structure and running state of the logistics network; by combining a multi-agent reinforcement learning method, each transport vehicle is modeled as an independent agent, and the global transportation cost, time cost and vehicle fleet empty rate are taken as the joint optimization objectives, so that the total system cost and empty running ratio can be effectively reduced while maintaining the task completion efficiency, and multi-objective collaborative optimization of the scheduling scheme is realized.

[0066] The application can realize rapid response to sudden environmental changes by intelligently identifying and scoring the road sections with traffic obstacles in the path scheme by accessing real-time traffic states and weather risk information, and dynamically generating path adjustment instructions combined with time and risk increment; at the same time, a local reinforcement learning strategy is used to efficiently update the affected path, and the execution result is fed back to the dynamic graph structure to form a closed-loop path optimization mechanism, thereby improving the adaptive ability and operation stability of the scheduling system. BRIEF DESCRIPTION OF DRAWINGS

[0067] In order to more clearly illustrate the technical solutions in the application or prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are only a part of the application, and other drawings can also be obtained by those skilled in the art without creative labor.

[0068] Figure 1 The figure is a logistics scheduling planning method of the embodiment of the application.

[0069] Figure 2 The figure is a logistics scheduling planning system of the embodiment of the application. DETAILED DESCRIPTION

[0070] The application will be described in detail below in combination with the drawings and specific embodiments. It should be noted that in order to make the embodiments more detailed, the following embodiments are the best and preferred embodiments, and other alternative ways can also be used by those skilled in the art to implement some known technologies; and the drawings are only used to more specifically describe the embodiments, and are not intended to specifically limit the application.

[0071] It is noted that references in the specification to "one embodiment," "an embodiment," "an example embodiment," "some embodiments," etc. indicate that the embodiment described can include a particular feature, structure, or characteristic, but every embodiment can not necessarily include the particular feature, structure, or characteristic. Moreover, appearances of the phrases "in one embodiment" or "in an example embodiment" in various places in the specification do not necessarily all refer to the same group of embodiments, nor are they necessarily mutually exclusive.

[0072] Generally, the terminology can be understood at least in part from usage in context. For example, the term "one or more" as used herein, depending at least in part upon context, can be used to describe any feature, structure, or characteristic in a singular sense or can be used to describe combinations of features, structures or characteristics in a plural sense. Additionally, the term "based on" can be understood as not necessarily requiring explicitly stated

[0073] As shown in Figure 1 The logistics scheduling planning method based on graph neural network and reinforcement learning includes the following steps:

[0074] S1: Taking the latitude and longitude coordinates of the distribution nodes as node features, and the cargo flow, historical transportation efficiency and distance between the distribution nodes as edge features, a dynamic graph structure of the logistics network is constructed;

[0075] S2: The dynamic graph structure is embedded and learned by a graph attention network (GAT) to extract a multi-dimensional feature vector of each node and calculate the correlation weight between nodes;

[0076] S3: The multi-dimensional feature vector is input into a multi-agent reinforcement learning framework, each vehicle is defined as an independent agent, the global transportation cost, time cost and empty rate are defined as joint optimization objectives, and an initial vehicle path planning scheme is generated;

[0077] S4: Real-time traffic state data and weather forecast information are obtained by accessing an electronic map to dynamically correct the road section passing state in the initial vehicle path planning scheme, and adaptive path adjustment instructions are generated;

[0078] S5: Based on the adaptive path adjustment instructions, the multi-dimensional feature vector of the affected distribution node is recalculated, and the vehicle path planning scheme is iteratively updated by local reinforcement learning;

[0079] S6: The final collaborative optimized cargo transportation trajectory and vehicle driving path are output, and the node features and edge features of the dynamic graph structure are updated synchronously to complete the closed loop.

[0080] S1 specifically includes:

[0081] S11: Collect the geographic coordinate information of all distribution nodes, use the WGS-84 coordinate system to standardize the position of each distribution node, and obtain the position vector group in the node set;

[0082] S12: Statistics the historical goods flow records between distribution nodes, extract the average goods flow per unit time between each pair of nodes, and construct the flow feature matrix of the edge; The calculation formula of the average goods flow is as follows:

[0083] , wherein, represents the average goods flow per unit time from the distribution node to the node , represents the goods transportation volume in the th record, represents the transportation times in the statistical period;

[0084] S13: Calculate the historical transportation efficiency between distribution nodes, the transportation efficiency is the ratio of the average actual transportation time of unit goods in the specified time window and the theoretical shortest transportation time, denoted as transportation efficiency matrix, the calculation formula of transportation efficiency is as follows: , wherein, represents the transportation efficiency from node to node , represents the actual average transportation time, represents the corresponding shortest path theoretical time consumption;

[0085] S14: Based on the latitude and longitude information between distribution nodes, the spherical distance between each pair of nodes is calculated by geodetic method, and the distance matrix is constructed, wherein the spherical distance calculation formula is:

[0086] , wherein, represents the spherical distance from node to node , unit: km; is the average radius of the earth, taking the value of ; are the latitudes (radian system) of nodes , are the corresponding longitudes (radian system);

[0087] S15: The position vector group is taken as the node feature, the traffic feature matrix, the transportation efficiency matrix and the distance matrix are jointly coded as the edge feature, the logistics network is represented by a graph data structure, a dynamic graph structure supporting time updating is formed, and is taken as the basis for input of the subsequent graph neural network; the dynamic graph structure constructed through the above steps not only fuses multi-dimensional actual business data and geographical spatial relationship, but also maintains physical consistency and dimensionality uniformity of the input features through formalized calculation, so that the graph neural network can obtain higher precision and stronger generalization capability of structure expression in subsequent embedding learning, significantly improving the modeling basis and scalability of logistics scheduling.

[0088] S2 specifically comprises:

[0089] S21: inputting the logistics network dynamic graph structure constructed in S1 into a graph attention network, taking the original feature of each distribution node as the node feature vector of the input data, and uniformly organizing the node feature vector into a node feature matrix for subsequent embedding learning process;

[0090] S22: linearly mapping the node features through the graph attention network, converting the original feature dimension into a feature representation in a new high-dimensional embedding space through a set of mapping weight matrices, the purpose of the mapping operation is to enhance the expression ability of the original node features, so that they can more effectively participate in the interaction and aggregation of neighbor information in the graph structure;

[0091] S23: for any node in the graph, the adjacent node set is retrieved in sequence, and the attention mechanism is used to combine the feature vectors between the node and each neighbor node; the combined feature vector is taken as input, and the relevance score is calculated through the set activation function and attention parameters to obtain the importance coefficient of the node to the adjacent node;

[0092] S24: normalizing the importance coefficients of all adjacent nodes of the node to ensure that the sum of the attention weights of all adjacent nodes is a constant, thereby realizing the correlation weight distribution between nodes;

[0093] S25: after obtaining the attention weight between nodes, the mapping feature vectors of the adjacent nodes are weighted and summed, and the output is transformed by combining a nonlinear activation function, and finally the multi-dimensional feature representation of each distribution node in the embedding space is obtained; through the above node embedding learning process based on the graph attention network, adaptive modeling of the relationship strength between distribution nodes can be realized, so that each node can extract high-precision features according to its context structure in the dynamic graph, thereby improving the state perception ability and path optimization effect of the subsequent scheduling strategy.

[0094] The steps of embedding learning of the dynamic graph structure are as follows:

[0095] First, the dynamic graph structure constructed by S1 is input into the graph attention network as the node feature matrix as input, where, denotes the total number of nodes in the graph, denotes the initial feature dimension of each node;

[0096] Next, the node features are dimensionally mapped by linear transformation to obtain the mapped feature matrix , and the mapping relationship is: , where, is the trainable linear mapping weight matrix in the graph attention network, is the mapped node feature dimension, denotes matrix multiplication;

[0097] Subsequently, for each node , the mapped feature vectors of its neighbor nodes , are calculated using the attention mechanism to calculate the associated weight , and the calculation formula is:

[0098] , where, denotes the attention weight of node to its neighbor node ; is the trainable attention vector; denotes the concatenation operation of the feature vectors of node and node ; is an activation function with a negative slope value of 0.2; denotes the normalization operation on all neighbor nodes of node ;

[0099] Finally, the neighbor node features of node are weighted and aggregated according to the attention weight to obtain the final embedding feature vector , and the calculation formula is: , where, denotes a nonlinear activation function, preferably a ReLU function; is the final multi-dimensional feature representation of node ; all vector dimensions are kept as to ensure dimensional consistency.

[0100] S3 specifically includes:

[0101] S31: Integrating the multi-dimensional feature vectors of each node of S2 output and the current task state into an environment state, and inputting the multi-agent reinforcement learning environment;

[0102] S32: In the multi-agent reinforcement learning environment, each transport vehicle is an independent agent, and each agent is configured with a local observation interface to read the current position node features, adjacent edge features, and vehicle load status;

[0103] S33: A joint reward function is established for all agent action results with total transportation cost, total time cost, and vehicle fleet empty rate as three optimization indicators;

[0104] S34: Each agent selects the next target node from the action set according to the policy network and generates a vehicle access sequence. All vehicle sequences are summarized to form an initial vehicle path planning scheme. Through the above multi-agent reinforcement learning process, the vehicle path planning can quickly converge to an executable scheme under the three-dimensional targets of cost, time, and empty rate, achieving the coordinated improvement of global scheduling efficiency and resource utilization.

[0105] S33 specifically includes:

[0106] S331: After each round of agent path decision, the total transportation cost, total time cost, and total empty mileage of all vehicles in the scheduling period are calculated. The total transportation cost is the product sum of the actual running distance of all vehicles and the fuel consumption coefficient. The total time cost is the cumulative time consumption of all orders from the distribution starting point to the delivery end point. The total empty rate is the average value of the ratio of the total empty mileage to the total mileage of all vehicles;

[0107] S332: Set the weight coefficients of the three indicators to represent the optimization tendency of transportation cost, time cost, and empty rate, respectively. The joint reward function is constructed by weighted combination, and its expression is:

[0108] , wherein, represents the joint reward value (the goal is to maximize this value, so the negative sign in front means cost minimization); represents the total transportation cost, with the unit of currency; represents the total time cost, with the unit of hours; represents the vehicle fleet empty rate, which is a dimensionless ratio; are positive real weight coefficients of transportation cost, time cost, and empty rate, respectively, satisfying the normalization condition;

[0109] S334: The joint reward value The synchronization feedback is given to all agents as a common reward signal of the action results of this round; the above steps can simultaneously optimize the delivery time efficiency and vehicle fleet load efficiency on the premise of minimizing the cost by fusing the transportation cost, time cost and empty rate into the same reward function and unified feedback, so as to realize the global collaborative optimization of multi-agent path planning.

[0110] S34 specifically comprises:

[0111] S341: determining the action set of each agent, the action set including all adjacent distribution nodes reachable from the current position and the optional action of returning to the starting node;

[0112] S342: the agent inputs the current environment state into the policy network, and the policy network outputs the decision score of each action, which is used to reflect the long-term cumulative reward that can be obtained after the corresponding action is executed;

[0113] S343: the agent selects the action with the highest score in the action set as the next target node according to the decision score, and records the selected action in the access sequence of the agent;

[0114] S344: after selecting the action, the agent updates the local state, including the change of the current position, the update of the cargo state and the completion of the task, and feeds back the updated local state to the environment;

[0115] S345: repeating S342 to S344 until all scheduled vehicles complete their delivery tasks, forming an initial path planning scheme for each vehicle; the above steps can ensure that each vehicle makes the optimal path decision under the driving of the global optimization target, significantly improving the executability and scheduling efficiency of path planning.

[0116] S4 specifically comprises:

[0117] S41: acquiring the traffic state data of each road section in real time through the accessed electronic map service interface, including the current traffic speed and congestion level of the road section, and acquiring the precipitation intensity, wind speed and road icing risk information in the future period through the weather data interface;

[0118] S42: mapping the traffic state data and weather forecast information to each road section in the initial path planning scheme, and matching and identifying the traffic performance and safety risk of each segmented path;

[0119] S43: for each segmented path, calculating the traffic state score according to the traffic performance and safety risk indicators , and comparing with the preset traffic threshold value, when the traffic state score is less than the preset traffic threshold value The corresponding segment is marked as a segment to be adjusted; the traffic state score is calculated by the following formula:

[0120] , wherein, represents the current average traffic speed of the segment; represents the theoretical free-flow speed of the segment; represents the current congestion level value; represents the maximum scale of the congestion level; represents the road icing risk index; represents the maximum scale of the icing risk index; are the speed weight, congestion weight, and risk weight coefficients, respectively;

[0121] S44: For all segments marked as segments to be adjusted, a set of alternative paths is retrieved, and the optimal alternative route is selected according to the shortest time increment and the safety risk increment;

[0122] Specifically, the time increment and the risk increment are calculated by the following formulas, respectively:

[0123]

[0124]

[0125] A comprehensive comparison is made: , wherein, represents the comprehensive increment index of the alternative path relative to the original path, and when The smaller the value is, the lower the cost of the alternative route in terms of time required to complete the distribution and safety risk is; are the predicted time and risk indicators of the original path, respectively; are the predicted time and risk indicators of the alternative path, respectively; are the weight coefficients of the time increment and the risk increment, respectively; and the alternative path with the minimum is selected as the optimal alternative route;

[0126] S45: Based on the alternative route selected in S44 and the latest road condition state of the affected distribution node, adaptive path adjustment instructions are generated, including the identification of the affected segment, the sequence of the alternative route, and the recommended adjustment time; the above steps can accurately identify the segment to be adjusted and replace the route by comprehensively considering the time and risk increments, realize high-precision decision-making of the path adjustment instructions, and significantly improve the response capability of the logistics scheduling scheme to real-time traffic and weather changes.

[0127] S5 specifically includes:

[0128] ​​S51: According to the adaptive path adjustment instruction generated in step S4, the affected path segments and the corresponding distribution nodes are identified, and the feature vectors of these nodes in the graph neural network structure are marked as needing to be updated;

[0129] S52: Extract the local subgraph of the affected node, including its adjacent nodes and connecting edges, and input the subgraph structure into the graph attention network for embedding learning to generate updated multi-dimensional feature vectors;

[0130] Specifically, for each node in the subgraph , the updated embedding vector is calculated as follows: , wherein represents the node , the neighbor set in the local subgraph, is the attention weight, is the input feature vector of the neighbor node , and is a nonlinear activation function;

[0131] S53: Input the updated node features into the reinforcement learning environment, and only update the local scheduling of the affected path-related agents, while keeping the vehicle scheduling strategy unchanged for other unaffected paths;

[0132] S54: In the local reinforcement learning process, recalculate the action set of the affected agents, call the original policy network for policy correction, and use the adjusted environmental feedback for policy iteration;

[0133] S55: After iterative convergence, output the updated vehicle path access sequence to replace the corresponding part of the original initial scheme, thereby completing the local update of the overall path planning scheme. The above steps can realize fast and efficient local path update without global recalculation, ensuring the response speed of the scheduling system and the continuous stability of the overall path planning under complex environmental changes.

[0134] S6 specifically includes:

[0135] S61: Receive the driving trajectory data and cargo delivery trajectory data of all transport vehicles during the execution of the final path planning, and arrange the arrival time, departure time and loading state of each node in chronological order;

[0136] S62: Based on the post-execution trajectory data, re-statistics the actual cargo flow and average residence time of each distribution node as new features of the node; at the same time, according to the actual driving distance and time consumption between nodes, recalculate the real-time transportation efficiency and road segment reliability of each edge as new features of the edge;

[0137] Specifically, the actual cargo flow is calculated by the following formula: wherein, is the delivery amount of the node for the first time, is the delivery frequency in the statistical period;

[0138] The average residence time is calculated by the following formula: wherein, and are the arrival and departure times of the node for the first time,

[0139] The real-time transportation efficiency is calculated by the following formula: wherein, is the actual driving distance, is the corresponding driving time consumption;

[0140] The reliability of the road section is calculated by the following formula: wherein, is the number of times of completing driving within the predetermined time window, is the total driving frequency;

[0141] S63: The node new features and edge new features obtained in S62 are written into the dynamic graph structure, the node feature matrix and the edge feature matrix are updated, and the time stamp of this update is marked in the graph data storage;

[0142] S64: The updated dynamic graph structure is taken as the input of the next round of scheduling and embedding learning, and the closed loop of initial construction-path planning-dynamic correction-local update-synchronous feedback is completed, so that the model execution and the graph structure state are continuously iterated and synchronized. Through the above closed loop feature updating process with quantitative calculation, the dynamic graph structure can continuously reflect the latest running state, so as to further improve the prediction accuracy and execution stability of the scheduling system based on the most accurate network features in the next round of scheduling.

[0143] As shown in Figure 2 , the logistics scheduling planning system based on the graph neural network and the reinforcement learning is used to implement the logistics scheduling planning method based on the graph neural network and the reinforcement learning, and includes the following modules:

[0144] The network construction module is used to collect the geographic coordinates, historical cargo flow, transportation efficiency and node distance data of the distribution node, and construct a logistics network dynamic graph structure supporting time update based on the collected data.​​

[0145] Embedding learning module: for receiving the dynamic graph structure provided by the network construction module, mapping the node features and aggregating the neighbor features of the dynamic graph structure based on the graph attention network, to extract the multi-dimensional feature vectors of each sub-node and the attention weight between nodes;

[0146] Reinforcement learning decision module: for receiving the multi-dimensional feature vectors provided by the embedding learning module, modeling each transport vehicle as an independent agent, constructing a joint reward function based on global transportation cost, global time cost and vehicle fleet empty rate, and generating an initial vehicle path planning scheme using a multi-agent reinforcement learning framework;

[0147] Real-time data acquisition module: for acquiring current road segment passing speed, congestion level and future weather risk information from electronic map services and weather services;

[0148] Dynamic correction module: based on the traffic state and weather risk information provided by the real-time data acquisition module, scoring and threshold comparing the passing state of each road segment in the initial vehicle path planning scheme, identifying the adjustment road segment and retrieving the alternative path, and generating adaptive path adjustment instructions;

[0149] Path update module: for updating the affected vehicle path in the local subgraph range according to the adaptive path adjustment instructions provided by the dynamic correction module and the node features provided by the embedding learning module, and feeding back the updated complete path to the network construction module to update the dynamic graph structure synchronously, forming a closed loop.

[0150] The present application covers any alternative, modification, equivalent method and scheme made on the essence and scope of the present application. In order to make the public have a thorough understanding of the present application, specific details are described in the following preferred embodiments of the present application, and the present application can also be fully understood without the description of these details. In addition, in order to avoid unnecessary confusion to the essence of the present application, well-known methods, processes, procedures, elements and circuits, etc. are not described in detail.

[0151] The above is only the preferred embodiment of the present application, and it should be pointed out that for ordinary skilled in the art, without departing from the principle of the present application, a number of improvements and refinements can be made, and these improvements and refinements should be regarded as the protection scope of the present application.

Claims

1. A logistics scheduling planning method based on a graph neural network and reinforcement learning, characterized in that, The method comprises the following steps: S1: taking the latitude and longitude coordinates of the distribution nodes as node features, and taking the cargo flow, historical transportation efficiency and distance between the distribution nodes as edge features, a dynamic graph structure of the logistics network is constructed; S2: embedding learning is performed on the dynamic graph structure through a graph attention network, a multi-dimensional feature vector of each node is extracted, and a correlation weight between the nodes is calculated; S3: the multi-dimensional feature vector is input into a multi-agent reinforcement learning framework, each vehicle is defined as an independent agent, the global transportation cost, time cost and empty rate are taken as joint optimization objectives, and an initial vehicle path planning scheme is generated; S3 specifically comprises: S31: integrating the multi-dimensional feature vectors of the distribution nodes output by S2 and the current task state into an environment state, and inputting the environment state into a multi-agent reinforcement learning environment; S32: in the multi-agent reinforcement learning environment, each transportation vehicle is an independent agent, and each agent is configured with a local observation interface, so that it can read the current position node feature, adjacent edge feature and vehicle loading state; S33: a joint reward function is established by taking the total transportation cost, total time cost and vehicle fleet empty rate as three optimization indicators; S34: each agent selects a next target node from an action set according to a policy network and generates a vehicle access sequence, all vehicle sequences are summarized to form an initial vehicle path planning scheme; S4: real-time traffic state data and weather forecast information are obtained from an electronic map to dynamically correct the road section passing state in the initial vehicle path planning scheme, and adaptive path adjustment instructions are generated; S5: based on the adaptive path adjustment instructions, the multi-dimensional feature vectors of the affected distribution nodes are recalculated, and the vehicle path planning scheme is iteratively updated through local reinforcement learning; S6: output the final collaborative optimized cargo transportation trajectory and vehicle driving path, and synchronously update the node features and edge features of the dynamic graph structure to complete the closed loop.

2. The logistics scheduling planning method based on a graph neural network and reinforcement learning according to claim 1, characterized in that, S1 specifically comprises: S11: collecting geographic coordinate information of all distribution nodes, uniformly standardizing the positions of the distribution nodes by using a WGS-84 coordinate system, and obtaining a position vector group in the node set; S12: statistical records of historical cargo flow between the distribution nodes are obtained, the average cargo flow between each pair of nodes per unit time is extracted, and a flow feature matrix of the edges is constructed; S13: the historical transportation efficiency between the distribution nodes is calculated, the transportation efficiency is the ratio of the average actual transportation time of unit cargo in a specified time window to the theoretical shortest transportation time, and is denoted as a transportation efficiency matrix; S14: based on the latitude and longitude information between the distribution nodes, the spherical distance between each pair of nodes is calculated by using geodetic surveying method, and a distance matrix is constructed; S15: the position vector group is taken as the node features, the flow feature matrix, the transportation efficiency matrix and the distance matrix are jointly coded as the edge features, the logistics network is represented by using a graph data structure, and a dynamic graph structure supporting time updating is formed.

3. The logistics scheduling planning method based on a graph neural network and reinforcement learning according to claim 1, characterized in that, S2 specifically comprises: S21: inputting the dynamic graph structure of the logistics network constructed in S1 into a graph attention network, and taking the original features of each distribution node as the node feature vectors of the input data, and uniformly organizing the node feature vectors into a node feature matrix; S22: Linear mapping processing is performed on the node features by the graph attention network, and the original feature dimension is converted into a feature representation in a new high-dimensional embedding space through a set of mapping weight matrices; S23: For any node in the graph, the adjacent node set is retrieved in turn, and the attention mechanism is used to combine the feature vectors between the node and each of its neighbor nodes; the combined feature vector is input into a set activation function and relatedness scoring calculation is performed with attention parameters to obtain the importance coefficient of the node pair adjacent nodes; S24: The importance coefficients of all adjacent nodes of the node are normalized to ensure that the sum of the attention weights of all adjacent nodes is a constant; S25: After obtaining the attention weights between nodes, the mapping feature vectors of adjacent nodes are weighted and summed, and an output transformation is performed in combination with a nonlinear activation function, and finally the multi-dimensional feature representation of each distribution node in the embedding space is obtained.

4. The logistics scheduling planning method based on a graph neural network and reinforcement learning according to claim 1, characterized in that, The S33 specifically includes: S331: After each round of path decision of the agent, the total transportation cost, total time cost and total empty mileage of all vehicles in the scheduling period are counted; wherein the total transportation cost is the product sum of the actual running distance of all vehicles and the fuel consumption coefficient; the total time cost is the cumulative time cost of all orders from the distribution starting point to the delivery end point; and the total empty rate is the average value of the ratio of the empty mileage of all vehicles to the total mileage; S332: The weight coefficients of the three indicators are set to represent the optimization tendency of the transportation cost, time cost and empty rate, respectively, and a joint reward function is constructed by weighted combination, and its expression is: wherein, denotes the joint reward value; denotes the total transportation cost; denotes the total time cost; denotes the fleet empty rate; are positive real weight coefficients for transportation cost, time cost and empty rate, respectively. S333: the joint reward value is sent to all agents as a common reward signal of the result of this round of actions. Synchronization feedback to all agents as a common reward signal of the result of this round of actions.

5. The logistics scheduling planning method based on a graph neural network and reinforcement learning according to claim 4, characterized in that, The S34 specifically includes: S341: An action set is determined for each agent, and the action set includes all adjacent distribution nodes that can be reached from the current position and the optional action of returning to the starting node; S342: The agent inputs the current environment state into its policy network, and the policy network outputs the decision score of each action; S343: The agent selects the action with the highest score in the action set as the next target node according to the decision score, and records the selected action in the access sequence of the agent; S344: After selecting the action, the agent updates its local state, including changing the current position, updating the cargo state and marking the task completion, and feeds back the updated local state to the environment; S345: Repeat S342 to S344 until all scheduled vehicles complete their distribution tasks, and an initial path planning scheme for each vehicle is formed.

6. The logistics scheduling planning method based on a graph neural network and reinforcement learning according to claim 1, characterized in that, The S4 specifically includes: S41: Real-time traffic state data of each road section is obtained through the accessed electronic map service interface, including the current traffic speed and congestion level of the road section, and the precipitation intensity, wind speed and road icing risk information in the future period are obtained through the weather data interface; S42: Map the traffic state data and weather forecast information to each road section in the initial path planning scheme, and match and identify the traffic performance and safety risk of each segmented path; S43: For each segmented path, calculate a passing state score according to its passing performance and safety risk indicators and compare with a preset passing threshold value, when the passing state score is less than the preset passing threshold value mark the corresponding segment as a segment to be adjusted S44: For all segments marked for adjustment, retrieve the alternative path set, and select the optimal alternative route according to the shortest time increment and safety risk increment; S45: Based on the selected alternative route and the latest traffic status of the affected distribution node in S44, generate adaptive path adjustment instructions, including affected road segment identification, alternative route sequence and recommended adjustment time.

7. The logistics scheduling planning method based on a graph neural network and reinforcement learning according to claim 1, characterized in that, The S5 specifically includes: S51: According to the adaptive path adjustment instructions generated in step S4, identify the affected path segment and the corresponding distribution node, and mark the feature vectors of these nodes in the graph neural network structure as needing to be updated; S52: Extract the local subgraph of the affected node, including its adjacent nodes and connecting edges, and re-input the subgraph structure into the graph attention network for embedding learning to generate updated multi-dimensional feature vectors; S53: Input the updated node features into the reinforcement learning environment, and only update the local scheduling of the affected path agent, while keeping the vehicle scheduling strategy unchanged for other unaffected paths; S54: In the local reinforcement learning process, recalculate the action set of the affected agent, call the original policy network for policy correction, and use the adjusted environment feedback for policy iteration; S55: After iterative convergence, output the updated vehicle path access sequence to replace the corresponding part of the original initial scheme, thereby completing the local update of the overall path planning scheme. 8.The logistics scheduling planning method based on graph neural network and reinforcement learning according to claim 1, characterized in that, The S6 specifically includes: S61: Receive all transportation vehicle driving trajectory data and cargo delivery trajectory data in the final path planning execution process, and arrange the arrival time, departure time and loading state of each node in chronological order; S62: Based on the post-execution trajectory data, re-statistics the actual cargo flow and average residence time of each distribution node as new node features; At the same time, according to the actual driving distance and time consumption between nodes, recalculate the real-time transportation efficiency and road segment reliability of each edge as new edge features; S63: Write the node and edge features obtained in S62 into the dynamic graph structure, update the node feature matrix and edge feature matrix, and mark the time stamp of this update in the graph data storage; S64: Update the dynamic graph structure as the input of the next round of scheduling and embedding learning.

9. The logistics scheduling planning system based on graph neural network and reinforcement learning, for implementing the logistics scheduling planning method based on graph neural network and reinforcement learning according to any one of claims 1-8, characterized in that, Including the following modules: Network construction module: used to collect the geographic coordinates, historical cargo flow, transportation efficiency and node distance data of the distribution node, and construct a dynamic graph structure of the logistics network supporting time update based on the collected data; Embedding learning module: used to receive the dynamic graph structure provided by the network construction module, map the node features and aggregate the neighbor features of the dynamic graph structure based on the graph attention network, to extract the multi-dimensional feature vectors of each distribution node and the attention weights between nodes; Reinforcement learning decision module: used to receive the multi-dimensional feature vectors provided by the embedding learning module, model each transportation vehicle as an independent agent, construct a joint reward function based on global transportation cost, global time cost and vehicle fleet empty rate, and use the multi-agent reinforcement learning framework to generate an initial vehicle path planning scheme; Real-time data acquisition module: used to obtain current road segment passing speed, congestion level and future weather risk information from electronic map services and weather services; The dynamic correction module scores and compares thresholds for the traffic states of each road segment in the initial vehicle path planning scheme based on the traffic state and weather risk information provided by the real-time data acquisition module, identifies road segments that need to be adjusted and retrieves alternative paths, and generates adaptive path adjustment instructions. The path updating module is used to update the affected vehicle path in the local subgraph range through reinforcement learning iteration according to the adaptive path adjustment instructions provided by the dynamic correction module and the node features provided by the embedded learning module, and feed back the updated complete path to the network construction module to update the dynamic graph structure synchronously, forming a closed loop.

Images