Heterogeneous multi-agent collaborative emergency material allocation method, system, device and medium

Abstract

The application relates to a heterogeneous multi-agent collaborative emergency material allocation method, system, device and medium, and belongs to the technical field of emergency logistics and intelligent scheduling. The method comprises the following steps: collecting state information and real-time disaster information of an emergency logistics network, and dynamically constructing a heterogeneous directed graph with type identification and relationship identification; a plurality of local observations are generated for each node, node structure embedding is obtained by using a heterogeneous graph network with different mapping and message functions for different node / edge types, the local observations of active nodes are spliced with the structure embedding, and then the spliced result is input into a type-specific Actor network, a mixed action containing discrete destination selection and continuous shipping proportion is output, actual allocation actions are generated and updated by means of demand perception action masks, downstream residual demand and transportation constraints. By using the method, invalid, late and fragmented allocation can be effectively inhibited, and the timeliness, robustness and global collaborative efficiency of emergency material allocation in a dynamic disaster environment can be improved.

Description

Technical Field

[0001] This application relates to the field of emergency logistics and intelligent dispatching technology, and in particular to a method, system, equipment and medium for the coordinated dispatch of emergency supplies by heterogeneous multi-agent systems. Background Technology

[0002] Emergency supplies allocation is a core component of disaster response and relief operations. Its goal is to efficiently and promptly deliver various relief supplies from supply sources (such as central warehouses and external suppliers) through a multi-level warehousing network to the demand nodes (such as distribution stations and temporary shelters) in disaster-stricken areas. However, real-world disaster environments, such as floods, earthquakes, and typhoons, are typically characterized by high dynamism and uncertainty: on the one hand, the demand in disaster-stricken areas may surge or change rapidly in a short period; on the other hand, transportation networks may be dynamically restructured due to road disruptions and bridge damage, and some warehousing nodes or temporary supply points may become active or inactive at any time due to the deployment of relief forces. Developing rapid, robust, and globally coordinated supplies allocation strategies under these dynamic, constrained, and information-impaired conditions presents a significant technical challenge in the field of emergency management.

[0003] Existing technologies have the following main shortcomings. First, static optimization models (such as linear programming and network flow models) require complete supply and demand and topology information and solve the allocation scheme in one go. They cannot adapt to dynamic changes such as sudden increases in demand and road interruptions during disasters, and they are difficult to effectively utilize information on goods in transit, which can easily lead to duplicate planning or resource waste. Second, although heuristic scheduling rules (such as inventory thresholds and distance-first allocation) are simple to calculate, the rules are fixed and cannot be dynamically adjusted according to the scenario. When multiple upstream nodes make decisions independently according to the rules, it can easily cause downstream congestion, duplicate shipments, or ineffective allocations, resulting in poor global coordination. Third, single-agent reinforcement learning compresses the entire logistics network into a single action space, faces the curse of dimensionality, and has difficulty expressing the role differences of nodes such as suppliers, warehouses, and distribution stations, resulting in weak policy generalization ability. Fourth, while multi-agent reinforcement learning can model nodes separately, existing methods mostly use homogeneous graph networks, ignoring the heterogeneous semantics of node types (supply, storage, temporary, demand) and edge relationships (hierarchical allocation, alternative pathways). Furthermore, the action space often separates or discretizes destination selection from shipment volume, which is disconnected from the need in real-world allocation operations to continuously determine the proportion of multiple materials shipped. In addition, independent decision-making by each agent easily leads to local optima, lacking collaborative awareness of downstream residual demand, en route arrivals, and congestion risks. Summary of the Invention

[0004] Based on this, it is necessary to provide a heterogeneous multi-agent collaborative emergency material allocation method, system, equipment, and medium that can realize unified expression of dynamic heterogeneous logistics networks, multi-node collaborative hybrid action allocation, and demand perception to suppress invalid and delayed allocation.

[0005] A heterogeneous multi-agent collaborative emergency supplies allocation method, the method comprising: The emergency logistics network collects real-time status information of each node and edge, and also collects real-time disaster information. Based on the state information, a heterogeneous directed graph is dynamically constructed to represent the emergency logistics network. In the heterogeneous directed graph, a type identifier is marked for each node, a relationship identifier is marked for each edge, and the state of each node and edge is updated according to the real-time disaster information. For each node in the heterogeneous directed graph, local observation data is generated, which includes the corresponding node's inventory, unmet demand, forecast of the arrival of goods in transit, throughput and occupancy, and risk of delay. The heterogeneous directed graph and the local observations of each node are input into the heterogeneous graph network. Corresponding input mappings are used for different node types in the heterogeneous directed graph, and corresponding message passing functions are used for different edge types. The structural embedding of each node is obtained through multiple rounds of message aggregation. After splicing the local observations of the active nodes with allocation capabilities in the heterogeneous directed graph with the corresponding structural embeddings, the data is input into the Actor network corresponding to the node type of the active nodes to obtain hybrid actions. The hybrid actions include: discrete destination selection and continuous multi-material shipment ratio; wherein, the destination selection is to select a target node from the downstream nodes or to choose to wait. Based on the hybrid action, the effective destination is determined from the downstream nodes of the current node in the heterogeneous directed graph through the demand-aware action mask, and the shipment ratio is converted into the actual shipment volume according to the downstream residual demand, transport edge capacity and arrival time limit, and the transfer action is generated. The aforementioned allocation action is executed, and the inventory, in-transit materials, and demand fulfillment of each node are updated to achieve collaborative emergency material allocation among heterogeneous multi-agent systems.

[0006] In one embodiment, when constructing the heterogeneous directed graph: The node types include at least two of the following: supply nodes, warehousing nodes, delivery demand nodes, and temporary nodes; The edge types include at least one or more of the following: hierarchical allocation edge, alternative route edge, and disaster impact edge.

[0007] In one embodiment, determining the valid destination using a demand-aware action mask includes: Based on the state information of the active node and its corresponding downstream node, downstream nodes that meet the demand-aware action mask are determined as valid destinations, wherein the demand-aware action mask is: If the active node is active and has shippable inventory, and the active node's throughput is not saturated, and the downstream node is active and its throughput is not saturated, and the downstream node or its downstream has residual demand, then the corresponding downstream node is determined as a valid destination. And, the waiting action is judged as a valid destination.

[0008] In one embodiment, when calculating the downstream residual demand: The residual demand of a delivery node is obtained by subtracting the demand already met and the goods in transit that are expected to arrive on time from the total demand of the delivery node. Along the downstream reachable path of the heterogeneous directed graph, the residual demand of each delivery demand node is propagated or aggregated upstream; The residual demand is obtained by deducting the existing inventory at each node along the propagation or aggregation path and eliminating the portion that cannot be delivered on time based on the transportation time limit.

[0009] In one embodiment, during the hybrid action: The continuous multi-material shipment ratio is obtained by sampling the Beta distribution output by the Actor network; The discrete destination selection is obtained by sampling the classification distribution output by the Actor network.

[0010] In one embodiment, the method further includes a training phase: Each active node in the heterogeneous directed graph is treated as an agent, and a reinforcement learning framework with centralized training and distributed execution is used to train the heterogeneous graph network and each Actor network. During training, at each decision moment, each agent generates and executes hybrid actions through its respective Actor network based on the current local observations and the structural embedding output by the heterogeneous graph network, thereby obtaining a reward and entering the next state. The local observations, global states, actions, rewards, and next states of each agent are stored as experience data in the experience pool, wherein the global state is generated based on the local observations of each node and the real-time disaster information. The global state is received using a centralized Critic network, the global value is estimated, and the advantage function is calculated. The pruning target is optimized using a multi-agent proximal strategy. Based on the empirical data, the strategy loss and value loss are calculated, and the heterogeneous graph network, the Actor networks of each agent, and the centralized Critic network are jointly updated.

[0011] In one embodiment, the training phase further includes: adopting a course learning mechanism to dynamically adjust the complexity of the environmental scenario according to the training process, gradually increasing the intensity of demand surges, the frequency of road interruptions, the probability of temporary node activation, and the complexity of event combinations, so that the training transitions from a low-disturbance scenario to a high-disturbance scenario.

[0012] This application also proposes a heterogeneous multi-agent collaborative emergency supplies allocation system, the system comprising: The data acquisition module is used to collect real-time status information of each node and edge based on the emergency logistics network, and at the same time collect real-time disaster information. The dynamic graph construction module is used to dynamically construct a heterogeneous directed graph to represent the emergency logistics network based on the state information. In the heterogeneous directed graph, a type identifier is marked for each node, a relationship identifier is marked for each edge, and the state of each node and edge is updated according to the real-time disaster information. The state coding module is used to generate local observation data for each node in the heterogeneous directed graph. The local observation data includes the corresponding node's inventory, unmet demand, forecast of the arrival of goods in transit, throughput occupancy, and risk of delay. The heterogeneous graph network module is used to input the heterogeneous directed graph and the local observations of each node into the heterogeneous graph network. It adopts corresponding input mapping for different node types in the heterogeneous directed graph and corresponding message passing functions for different edge types. The structural embedding of each node is obtained through multiple rounds of message aggregation. The strategy decision module is used to concatenate the local observations of the active nodes with allocation capabilities in the heterogeneous directed graph with the corresponding structural embeddings, and then input them into the Actor network corresponding to the node type of the active node to obtain hybrid actions. The hybrid actions include: discrete destination selection and continuous multi-material shipment ratio; wherein, the destination selection is to select a target node from the downstream nodes or choose to wait. The allocation action generation module is used to determine the effective destination from the downstream nodes of the current node in the heterogeneous directed graph based on the hybrid action, through the demand-aware action mask, and convert the shipment ratio into the actual shipment volume according to the downstream residual demand, transport edge capacity and arrival time limit, and generate the allocation action. The allocation execution module is used to execute the allocation action and update the inventory, in-transit materials and demand fulfillment of each node to realize the collaborative allocation of emergency materials by heterogeneous multi-agent systems.

[0013] A computer device includes a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described heterogeneous multi-agent collaborative emergency material allocation method.

[0014] A computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps in the above-described heterogeneous multi-agent collaborative emergency material allocation method.

[0015] The aforementioned heterogeneous multi-agent collaborative emergency material allocation method, system, equipment, and medium collects real-time state information of each node and edge based on the emergency logistics network, while simultaneously collecting real-time disaster information. Based on the state information, a heterogeneous directed graph is dynamically constructed to represent the emergency logistics network. Each node in the heterogeneous directed graph is labeled with a type identifier, and each edge is labeled with a relationship identifier. The state of each node and edge is updated according to real-time disaster information, generating local observation data for each node in the heterogeneous directed graph. This local observation data includes the corresponding node's inventory, unmet demand, predicted arrival of materials in transit, throughput, and late arrival risk. The heterogeneous directed graph and the local observations of each node are input into the heterogeneous graph network. Corresponding input mappings are used for different node types in the heterogeneous directed graph, and corresponding message passing functions are used for different edge types. Multi-round message aggregation yields the structural embedding of each node. Local observations of active nodes with allocation capabilities in the heterogeneous directed graph are concatenated with their corresponding structural embeddings and input into the Actor network corresponding to the node type of the active node. This results in hybrid actions, including discrete destination selection and continuous multi-material shipment ratios. Destination selection involves choosing a target node or waiting from downstream nodes. Based on these hybrid actions, effective destinations are determined from the current node's downstream nodes in the heterogeneous directed graph using a demand-aware action mask. The shipment ratio is converted into actual shipment volume based on downstream residual demand, transport edge capacity, and arrival time limits, generating and executing allocation actions. This updates the inventory, in-transit materials, and demand fulfillment of each node, enabling collaborative emergency material allocation among heterogeneous multi-agent systems.

[0016] Beneficial effects: By collecting real-time state information and disaster information of nodes and edges in the emergency logistics network, a heterogeneous directed graph with labeled type and relationship identifiers is dynamically constructed and continuously updated. This graph accurately expresses the diverse relationships among different roles such as suppliers, warehouses, distribution stations, and temporary nodes, as well as hierarchical allocation, alternative pathways, and disaster impacts, overcoming the semantic deficiencies of traditional isomorphic graphs. For each node, local observations are generated, including inventory, unmet demand, in-transit arrival predictions, throughput, and late arrival risks, enabling decision-making to simultaneously perceive the current state and future arrivals of materials, effectively avoiding duplicate shipments and resource waste. Utilizing a heterogeneous graph network with differentiated mappings and message passing functions for different node and edge types, the heterogeneous semantics of the logistics network are fully preserved. Through multi-round aggregation, a structural embedding containing upstream and downstream topological information is obtained, providing... Collaborative decision-making provides high-quality state representation; by embedding local observations and structures into a concatenated Actor network corresponding to the node type, the output simultaneously includes a hybrid action of discrete destination selection (including waiting actions) and continuous multi-material shipment ratios, achieving joint optimization of "where to send" and "how much to ship," which aligns with actual allocation business needs; furthermore, by filtering effective destinations through demand-aware action masks, and converting the ratios into actual shipment volumes based on downstream residual demand, transportation edge capacity, and arrival time limits, it can dynamically suppress invalid, duplicate, over-limit, and late allocations, significantly reducing congestion and fragmented transportation; after executing the allocation, the inventory, materials in transit, and demand fulfillment are updated, forming a closed-loop feedback, thereby improving the timeliness, robustness, and global collaborative efficiency of emergency material allocation in dynamic disaster environments. Attached Figure Description

[0017] Figure 1 This is a flowchart illustrating a heterogeneous multi-agent collaborative emergency material allocation method in one embodiment; Figure 2 This is a schematic diagram illustrating the evolution of course learning and training difficulty in one embodiment; Figure 3 This is a schematic diagram of the overall process and framework of the method in one embodiment; Figure 4 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0019] To address the problems in existing technologies, such as the inability to effectively express the heterogeneous semantic relationships between suppliers, multi-level warehouses, distribution stations, and temporary nodes in emergency logistics networks, the difficulty in achieving global multi-agent collaboration under dynamic disturbances like demand surges, road disruptions, and the start-up and shutdown of temporary nodes, and the separation of destination selection from shipment volume in allocation actions, leading to frequent invalid and delayed allocations due to a lack of demand perception capabilities, this application addresses these issues. Figure 1 As shown, a heterogeneous multi-agent collaborative emergency material allocation method is provided, which specifically includes the following steps: Step S100: Collect the status information of each node and edge in real time based on the emergency logistics network, and collect real-time disaster information at the same time.

[0020] Step S110: Based on the state information, a heterogeneous directed graph is dynamically constructed to represent the emergency logistics network. In the heterogeneous directed graph, a type identifier is marked for each node, a relationship identifier is marked for each edge, and the state of each node and edge is updated according to real-time disaster information.

[0021] Step S120: Generate local observation data for each node in the heterogeneous directed graph. The local observation data includes the corresponding node's inventory, unmet demand, forecast of the arrival of goods in transit, throughput occupancy, and risk of delay.

[0022] Step S130: Input the heterogeneous directed graph and the local observations of each node into the heterogeneous graph network. Use the corresponding input mapping for different node types in the heterogeneous directed graph and the corresponding message passing function for different edge types. Obtain the structural embedding of each node through multiple rounds of message aggregation.

[0023] Step S140: After splicing the local observations of the active nodes with allocation capabilities in the heterogeneous directed graph with the corresponding structural embeddings, the data is input into the Actor network corresponding to the node type of the active nodes to obtain hybrid actions. The hybrid actions include: discrete destination selection and continuous multi-material shipment ratio; wherein, destination selection is to select a target node from the downstream nodes or choose to wait.

[0024] Step S150: Based on hybrid actions, the effective destination is determined from the downstream nodes of the current node in the heterogeneous directed graph through the demand-aware action mask, and the shipment ratio is converted into the actual shipment volume according to the downstream residual demand, transportation edge capacity and arrival time limit, and the transfer action is generated.

[0025] Step S160: Execute the allocation action and update the inventory, in-transit materials and demand fulfillment of each node to achieve collaborative emergency material allocation among heterogeneous multi-agent systems.

[0026] In step S100, the emergency logistics network in the prior art is typically abstracted as a material circulation system consisting of supply nodes, warehousing nodes, distribution demand nodes, and transportation edges connecting these nodes. Its core function is to efficiently distribute relief supplies from the source to the disaster area when a disaster occurs. In this application, based on the emergency logistics network concept in the prior art, at each decision-making moment, the status information of each node and edge is collected in real time from the emergency command platform, warehousing management system, road traffic monitoring system, vehicle management system, and on-site reporting system, while real-time disaster information is also collected.

[0027] Specifically, node information includes node identifier, type (supply node, warehousing node, delivery demand node, temporary node), activation status, inventory, throughput capacity, occupied throughput, and role code; edge information includes directed transportation connections, transportation mode, transportation distance or estimated transportation time, transportation capacity, road conditions, alternative route identification, and disaster impact level; demand information includes the material demand list and task deadlines for each delivery demand node; in-transit material information includes dispatched but not yet arrived transfer records and their estimated arrival time and transportation status; disaster event information includes demand surges, road interruptions, transportation mode switching, node failures, and temporary node activation. The system timestamps data from different sources to form a network snapshot at the current decision-making moment, marks or fills in missing values, and normalizes or standardizes the data for use in subsequent steps.

[0028] In step S110, considering that traditional emergency logistics scheduling methods typically abstract the network as a homogeneous graph (i.e., all nodes and edges are considered to be of the same type) or a static graph, they cannot effectively express the functional differences of different roles such as suppliers, multi-level warehouses, distribution stations, and temporary nodes, nor can they reflect the semantic constraints of multi-edge relationships such as hierarchical allocation, alternative routes, and disaster impacts. This results in the strategy being insensitive to changes in network structure and limited collaborative decision-making capabilities. Therefore, this application constructs a dynamic heterogeneous directed graph as the basic data structure for subsequent graph neural networks and agent decision-making.

[0029] In this embodiment, a heterogeneous directed graph is dynamically constructed to represent the current emergency logistics network based on the state information collected in step S100. In this heterogeneous directed graph, each node is labeled with a type identifier (e.g., supply node, warehousing node, delivery demand node, temporary node), and each directed edge is labeled with a relationship identifier (e.g., hierarchical allocation edge, alternative route edge, disaster impact edge). Simultaneously, based on real-time disaster information, the activation status of each node and the availability and transportation attributes of each edge are updated (e.g., marking an edge as unavailable when a road is interrupted, activating the corresponding node when a temporary node is enabled). Through this dynamic graph modeling that explicitly distinguishes node types and edge relationships, the subsequent heterogeneous graph network can employ different message passing functions for different types, enabling each agent to accurately perceive the role differences between upstream and downstream nodes and the real-time availability of routes during decision-making, providing an expressive state representation for multi-agent collaborative allocation.

[0030] In this embodiment, when constructing the heterogeneous directed graph: the node types include at least two of the following: supply nodes, storage nodes, distribution demand nodes, and temporary nodes. The edge types include at least one or more of the following: hierarchical allocation edges, alternative pathway edges, and disaster impact edges.

[0031] Specifically, at every moment The emergency logistics network is represented as a heterogeneous directed graph. .in, Represents a set of nodes. This represents a set of directed edges that change over time. Nodes can include supply source nodes, primary or secondary storage nodes, distribution demand nodes, temporary supply nodes, temporary storage nodes, and temporary delivery nodes. Edges can represent allocation relationships from upstream to downstream, relationships between different transportation modes, alternative pathways, or availability relationships after a disaster.

[0032] Specifically, unlike isomorphic graph modeling, heterogeneous directed graphs assign a node type to each node. And assign relation types to edges. or edge attribute Node types represent the roles of nodes in the emergency logistics chain, while edge relationships represent the semantics and constraints of transportation connections. In this way, the model can distinguish between different allocation meanings such as "suppliers replenishing warehouses," "transferring goods between warehouses," "warehouses delivering goods to distribution stations," and "temporary nodes joining the network."

[0033] Furthermore, heterogeneous directed graphs are dynamic graphs. Partial updates can be performed when an event occurs without rebuilding the entire system. For example, when a road is interrupted, the corresponding edge can be marked as unavailable or its transport mode can be switched to an alternative route; when a temporary warehouse is enabled, the activation status of the corresponding node can be set to valid and the relevant edge can be connected; when a new delivery request is reported, the delivery node demand vector can be updated or a temporary delivery node can be added.

[0034] In this embodiment, the dynamic graph modeling method described above has two important functions: first, it provides an interpretable state transition basis for the reinforcement learning environment, enabling inventory, in-transit, arrival, and demand fulfillment to occur along the graph edges; second, it provides structural input for heterogeneous graph networks, enabling the strategy to identify which nodes are upstream supply bottlenecks, which nodes are downstream demand hotspots, and which pathways have congestion or delay risks from the topological relationships.

[0035] After the heterogeneous directed graph is constructed, to enable each node to make reasonable allocation decisions based on its own and its surrounding environment's state, and to provide information-rich state input for subsequent policy networks, this method further constructs local observation data for each node. In traditional methods, nodes typically rely solely on their own inventory and direct downstream demand for decisions, ignoring the arrival prediction of goods in transit, node throughput constraints, and the risk of goods being late. This can easily lead to duplicate shipments, increased congestion, or allocation failures. Therefore, in step S120, a type of local observation data containing multi-dimensional forward-looking information is designed. This allows each node to not only grasp its current inventory and demand gap during subsequent decisions, but also predict the quantity of goods arriving in the future, the current node's throughput pressure, the global or local congestion level, the total amount of goods in transit pointing to that node and their lateness risk, as well as the node's own activation state, type encoding, and role encoding. Through this rich observation information, subsequent allocation decisions can effectively determine key issues such as "whether immediate shipment is needed," "how much to ship," and "whether the target node still has a real demand," thereby suppressing ineffective allocations and resource waste.

[0036] Specifically, for any node At any moment Constructing local observation data Local observation data may include: the node's current inventory. This indicates the quantity of materials available for shipment or used to meet demand at that node; demand gap. This is used to represent the unmet material demand at delivery demand nodes; for non-demand nodes, it can be set to an empty or zero vector; future arrival feature. Used to indicate goods in transit that are expected to arrive at that node at some future time; throughput and occupancy. This indicates the loading / unloading or processing capacity that the node has used within the current decision-making cycle; congestion characteristics. Used to indicate the pressure of goods in transit across the entire network or in a specific area; "in transit" refers to the quantity. This indicates the amount of supplies being transported to that node; the risk of late arrival. This is used to indicate supplies that are not expected to arrive within the mission timeframe; node activation identifier. Represents a node At the present moment Whether it is in a state that can participate in scheduling; node type encoding Indicates the hierarchical type of a node in the supply chain / emergency logistics network; node role code. This indicates the functional location of a node within the direction of material flow. Local observation data. It can be represented as: ; In the above formula, local observation data can be normalized, standardized, or have missing values ​​imputed based on the actual system. Its core is not to limit feature dimensions, but to incorporate factors such as "current availability," "current demand," "future arrival," "processing capacity congestion," "potential delay," and "node semantic role" into the decision-making process. In this way, when selecting the destination and shipment ratio, the agent not only knows its inventory levels but also whether there is still real demand in the target direction and whether there are already goods en route that are about to arrive.

[0037] In step S130, after obtaining the local observation data of each node, in order to enable each node to perceive the upstream and downstream topology and the semantic differences between different nodes and edge relationships, this application employs a heterogeneous graph network to perform representation learning on the heterogeneous directed graph. Traditional homogeneous graph networks treat all nodes and edges as the same type, using a unified message passing function. This makes it impossible to distinguish the different functions of supply nodes, storage nodes, distribution demand nodes, temporary nodes, etc., and it is also difficult to express the different meanings of edge relationships such as hierarchical allocation, alternative pathways, and disaster impact. Therefore, this application designs a heterogeneous multi-agent graph network, the core of which is to adopt differentiated mapping and message passing mechanisms for different node types and edge types, thereby preserving the heterogeneous semantics in the emergency logistics network.

[0038] Specifically, firstly, a separate input projection function is set for different types of nodes. For nodes If its type is Then the original observations can be... Mapped to the initial hidden representation : ; The reason for using different projection functions for different node types is that the same feature fields of different nodes have different semantics. For example, the demand gap of a delivery node is core target information, while the demand gap of a warehousing node may be empty; the inventory of a supply node represents external supply capacity, while the inventory of a warehouse node represents transit capacity. Therefore, using a uniform projection weakens semantic differences, while type-related projection can preserve role characteristics.

[0039] Then, construct the relation-related message function for each edge relation type r. For edges from the source node Point to target node And the relation type is The directed edge, in the first... In layer message passing, the generated messages are: ; In the above formula, This represents the attributes of the edges (such as transport distance, capacity, availability, and disaster impact level). Different edge relationships use different message functions. For example, hierarchical transfer edges transmit inventory and throughput pressure, alternative route edges transmit alternative capacity and risk, and disaster impact edges transmit interruption probability and delay information.

[0040] For each target node Get the total message of this node. Aggregation methods can include summation, mean, max pooling, or attention-based weighted aggregation. In this embodiment, summation aggregation with edge weights is preferred. ; In the above formula, AGG can be a summation, mean, attention-weighted, or edge-weighted aggregation. Then, an update function related to the node type is used. Hidden representation of the current node and aggregated messages The nodes are then merged, and residual connections and normalization operations are added to obtain the node representations for the next layer: ; In the above formula, is a non-linear activation function (such as ReLU), and Norm represents normalization. This is a residual mapping (either an identity mapping or a linear mapping). The above message passing process can be repeated multiple times (e.g., ...). or This allows information to propagate along the graph structure for multiple hops, with each node gradually acquiring structural information within its multi-hop neighborhood.

[0041] After After layer message passing, each node is obtained. structural embedding This embedding not only includes the node's own local observation information but also integrates the topological semantics of different types of nodes and edge relationships upstream and downstream. For example, the structural embedding of a warehouse node reflects the supply capacity of upstream supply nodes, the residual demand of downstream delivery demand nodes, and the availability and congestion level of each transportation edge. This structural embedding will be fed into the subsequent policy network, enabling each active node to indirectly perceive the key information required for global coordination during decentralized execution.

[0042] In this embodiment, the parameters of the heterogeneous graph network (including the adjustable weights in each type of input projection, each relation message function, and each type of update function) are jointly updated during the training phase along with the subsequent Actor and Critic networks through multi-agent proximal policy optimization.

[0043] In this embodiment, the heterogeneous graph network is not limited to the message passing structure described in the above embodiments. Relational graph convolutional network (R-GCN), heterogeneous graph attention network (HAN), message passing neural network (MPNN), graph Transformer, or any combination thereof can also be used to achieve message aggregation between nodes. The message aggregation method is not limited to summation. Mean, max pooling, attention mechanism, or weighted aggregation based on transport capacity can also be used.

[0044] After obtaining the structural embedding of each node through a heterogeneous graph network, specific allocation actions need to be generated for each active node with allocation capabilities (such as supply nodes, storage nodes, and temporary supply nodes). Traditional methods typically handle "where to send" and "how much to send" separately, for example, selecting the target node first and then determining the shipment quantity, or only outputting discrete shipment options. This is difficult to adapt to the actual situation in emergency material allocation where continuous and precise determination of the shipment ratio of various materials is required. To address this, in step S140, a hybrid action decision mechanism based on a type-specific Actor network is proposed, enabling each active node to simultaneously output discrete destination selections and continuous multi-material shipment ratios, thereby directly corresponding to the information required by the actual allocation order.

[0045] Specifically, for each active node with allocation capability in a heterogeneous directed graph The local observations generated in step S120 Embedded with the structure obtained in step S130 The vectors are concatenated to form the decision feature vector of this node: ; Then Input and node type The corresponding Actor network. Different node types (such as supply nodes, storage nodes at different levels, and temporary supply nodes) can have their own Actor networks, or they can share the underlying encoding layer and then be processed by type-specific output headers. In this embodiment, an independent Actor network is set up for each type of active node to learn the decision-making patterns of different roles.

[0046] Specifically, each Actor network contains two output heads: one for discrete destination selection and the other for the proportion of continuous multi-material shipments. Hybrid Actions Selected by discrete destination and the proportion of continuous material shipments Composition. The discrete part answers "to which downstream node to send or whether to wait," while the continuous part answers "what proportion of various materials are sent, and their mixed actions." The probability is defined as: ; ; In the above formula, The classification probability chosen for discrete destinations This represents the probability density function of the continuous shipment ratio. The motivation for merging destination selection and shipment ratio into a hybrid action is that actual allocation orders naturally contain three types of information: destination, material category, and quantity. If only the destination is learned, post-processing is still needed to determine the quantity; if only the quantity is learned, the issues of route selection and node coordination cannot be resolved. This invention uses a hybrid action to ensure that the strategy output directly corresponds to the actual allocation execution, reducing the disconnect between strategy and business rules.

[0047] Specifically, the output head for discrete destination selection outputs a classification distribution, and the Actor network outputs a classification distribution whose categories correspond to all valid downstream nodes of the current node in the heterogeneous directed graph, plus an additional "wait" action. The Actor network first outputs the raw logits of each candidate destination, then uses a demand-aware action mask (described in detail in step S150) to suppress the logits of invalid destinations (e.g., setting them to a minimum), preventing invalid candidate destinations from being sampled. Finally, the probability distribution of valid destinations is obtained through the Softmax function. During training, specific destination selections are obtained by sampling from this distribution; during deployment, a distribution mean or a greedy strategy can be used to obtain deterministic destinations.

[0048] Specifically, for the proportion of continuous shipments of multiple materials, this applies to each category of materials. The Actor network outputs parameters of a probability distribution defined on a bounded interval. Preferably, a Beta distribution is used, defined on... An interval is naturally suitable for expressing the "shipment ratio". The Actor network outputs two shape parameters of the Beta distribution. and The shipment ratio is obtained by sampling from this Beta distribution: ;

[0049] For outputting the shipment ratios of various materials, the distributions of each material's ratio are independent. During training, the actual shipment ratios are sampled from the corresponding Beta distribution; during deployment, the mean of the distribution can be used as the deterministic shipment ratio.

[0050] In other embodiments, the probability distribution of the continuous shipment ratio is not limited to the Beta distribution; it can also be a truncated normal distribution, a Logistic-Normal distribution, a Dirichlet distribution, or other distributions defined on a bounded interval or a simplex. When it is necessary to ensure that the sum of the shipment ratios of various materials is 1 (total quantity conservation), normalization mapping or constrained projection methods can be used for processing.

[0051] Furthermore, the hybrid actions output by the active node include: the selected target node (or waiting) and the shipment ratio of various materials. When the destination is selected as "waiting," the shipment ratio is ignored and no actual allocation occurs. The logarithmic probability of the entire hybrid action is the sum of the logarithmic probabilities of the discrete actions and the logarithmic probability densities of the continuous ratios of each material, expressed as: ; The logarithmic probability in the above formula is used to calculate the policy loss in subsequent multi-agent proximal policy optimization.

[0052] Through the aforementioned hybrid action design, each active node can simultaneously determine "who to send to" and "how much to send." Furthermore, the continuity of the shipment ratio and the joint output of multiple materials allow the strategy to finely control the allocation intensity of various materials, avoiding coarse-grained decision-making errors caused by discretization. In addition, the introduction of a "wait" action allows nodes to choose not to act when there is no effective allocation opportunity, thereby avoiding resource waste caused by forced shipments.

[0053] In step S150, the Actor network outputs hybrid actions, including discrete destination selection and continuous multi-material shipment ratios. However, these hybrid actions only represent the decision-making intent and do not yet consider various constraints in the actual logistics system, such as node throughput capacity, transportation edge capacity, material arrival time limits, and downstream residual demand. Directly executing allocations according to the output ratios of the strategy may lead to ineffective transportation, node congestion, material delays, or resource waste. Therefore, a demand-aware allocation execution mechanism is proposed. Before the hybrid actions are formally transformed into actual allocation actions, effectiveness screening, demand-aware correction, and transportation constraint pruning are performed. This mechanism mainly includes three parts: demand-aware action masking, downstream residual demand estimation, and transportation constraint execution.

[0054] In this embodiment, a demand-aware action mask is dynamically generated based on the real-time status of the current active node and its downstream nodes. This is used to filter valid destinations in mixed actions. Specifically, a downstream node is considered a valid destination if it simultaneously meets the following conditions: the active node is active and has shippable inventory; the active node's throughput is not saturated; the downstream node is active and its throughput is not saturated; and there is residual demand (i.e., unmet demand that is not covered by on-time supplies already en route) within the downstream node or its downstream reachable range. Furthermore, waiting actions are always retained as valid candidates to avoid forced shipments when there are no valid allocation opportunities. If the destination selection output in step S140 is determined to be invalid through masking, the system will automatically correct the action to "waiting" or resample valid destinations based on the classification distribution. Through this masking mechanism, the policy network will not explore or execute obviously invalid actions during training, thereby accelerating convergence and reducing invalid allocations.

[0055] Specifically, through demand-aware action masks A valid destination is obtained, which is represented as: ; In the above formula, This is represented as a probability distribution of discrete actions (destination selection). This indicates a specific selected destination or a waiting action. This represents the action mask (identifying whether each candidate is valid). This represents the scores for each category in the original output of the Actor network. This represents the Softmax function with a mask.

[0056] To avoid continuing shipments to demand points that have already been met or are about to be met, it is necessary to accurately estimate the residual demand at each downstream node. The calculation of downstream residual demand involves: first, for each delivery node, subtracting the actually met demand from its total demand, and then subtracting the amount of goods in transit that are expected to arrive on time within the task deadline. This yields the residual demand for that delivery node, as shown in the following process: ; In the above formula, Indicates delivery demand nodes At any moment Residual demand, Indicates delivery demand nodes At any moment Total demand (cumulative or current stage demand). This indicates the quantity of demand that has actually been delivered and satisfied (demand satisfied). This represents the total amount of goods in transit that are expected to arrive at the delivery request node within the task timeframe (i.e., goods in transit that are expected to arrive within a certain time window in the future and will not be late). Ensure that residual demand is non-negative (if it is negative after deduction, take 0, indicating that the demand has been fully met or over-met).

[0057] Then, along the downstream reachable paths of the heterogeneous directed graph (i.e., starting from the delivery demand node and propagating in either the reverse or forward direction), the residual demand of each delivery demand node is propagated upstream or aggregated. For example, the downstream residual demand of a warehousing node can be understood as the sum of the residual demand of all its downstream reachable delivery nodes, taking into account the deduction of this residual demand by the warehousing node's own inventory. Finally, the residual demand is further deducted based on the existing inventory of each node on the propagation or aggregation path, and the portion that cannot be delivered on time is eliminated based on the transportation time limit (i.e., if a material cannot arrive within the time limit via a certain path, then this portion of the demand is not included in the effective residual demand of that path). After the above calculations, the estimated downstream residual demand of each node that could potentially be a destination is obtained, and this estimate is used for subsequent shipment volume pruning.

[0058] Specifically, the active node is calculated using the following formula. To the target node Shipment of goods Shippable base: ; In the above formula, Indicates active node China Materials Current inventory, Represents the target node For supplies The downstream residual demand (i.e., the effective demand obtained after the above propagation, deduction, and time-limit elimination) is considered. By taking the minimum of the two, it can be ensured that the actual shipment volume does not exceed the available inventory at the source node, nor does it exceed the actual demand gap at the downstream node, thereby avoiding over-shipment and ineffective transportation. This available shipment base will serve as the basis for subsequently converting the continuous shipment ratio into the actual shipment volume.

[0059] Based on this, the actual shipment volume is comprehensively trimmed or scaled according to the continuous shipment ratio output by the strategy, the aforementioned available shipment base, inventory balance, source node throughput capacity, target node throughput capacity, transportation edge capacity, and target demand surplus. Specifically, the initial shipment volume is first obtained by multiplying the available shipment base by the continuous shipment ratio. Then, it is successively constrained by the remaining available throughput capacity of the source node, the remaining acceptable throughput capacity of the target node, the maximum allowable capacity of the transportation edge, and the residual demand surplus of the target node or its downstream. If the target node is a delivery demand node, it needs to be further trimmed according to the current unmet demand of that node to avoid over-shipment and waste. After the above multi-level constraint processing, the final actual shipment volume is obtained.

[0060] Subsequently, based on the quality or volume of the materials, the attributes of the selected transport edge (such as distance and road type), the risk level of the route, the transport mode (road, rail, air, etc.), the existence of alternative routes, the current disaster status, and the actual capacity of the transport vehicles, the actual transport mode, estimated arrival time, and transport cost for this allocation are determined. If the estimated arrival time exceeds the arrival time limit stipulated in the emergency mission, the allocation record is marked as "late risk." This mark will be encoded in the state observation at subsequent decision times, and a penalty term will be introduced into the reward calculation, enabling the policy network to learn to proactively avoid allocation decisions that may lead to lateness.

[0061] In step S160, after generating the executable actual allocation action in step S150, the allocation action is sent to the corresponding execution unit (such as a warehouse management system, transportation scheduling system, or on-site command terminal). During execution, based on the source node, target node, material category, actual shipment volume, and transportation mode specified in the allocation action, operations such as material outbound, loading, transportation, and arrival confirmation are completed. Simultaneously, the status data of each node in the emergency logistics network is updated: the quantity of shipped materials is deducted from the inventory of the source node; the information of the materials shipped this time, including the estimated arrival time and transportation status, is added to the in-transit material record of the target node; when the materials actually arrive and are delivered, the inventory of the target node is further updated, and the quantity of delivered materials is deducted from its unmet demand.

[0062] Furthermore, if a late arrival risk is marked in step S150, the risk information is recorded in the corresponding allocation log and used as the basis for subsequent reward calculation and status coding. Through the above status update, the system completes one allocation decision and execution cycle in a closed loop and enters the next decision moment, thereby continuously realizing dynamic emergency material allocation under heterogeneous multi-agent collaboration.

[0063] In this embodiment, the method further includes a training phase: each active node in the heterogeneous directed graph is treated as an agent, and the heterogeneous graph network and each Actor network are trained using a reinforcement learning framework with centralized training and distributed execution. During training, at each decision time, each agent generates and executes mixed actions through its respective Actor network based on the current local observations and the structural embedding output by the heterogeneous graph network, obtains a reward, and enters the next state. The local observations, global state, actions, rewards, and next state of each agent are stored as experience data in an experience pool. The global state is generated based on the local observations of each node and real-time disaster information. A centralized Critic network is used to receive the global state, estimate the global value, and calculate the advantage function. A pruning target optimized by multi-agent proximal policies is adopted, and the policy loss and value loss are calculated based on the experience data. The heterogeneous graph network, the Actor networks of each agent, and the centralized Critic network are jointly updated.

[0064] Specifically, each active node with allocation capabilities in the heterogeneous directed graph is treated as an independent agent. During training, a paradigm of centralized training and decentralized execution is adopted: during training, access to global information is allowed to learn cooperative strategies; during execution, each agent relies only on its own local observations and structural embeddings to make independent decisions, in order to adapt to the environment of limited communication at disaster sites.

[0065] Specifically, in each training epoch, steps S100 to S160 are repeated, and the states, actions, rewards, and next states of all agents at each decision time are recorded. This includes local observation data for each agent. and its structural embedding Together they constitute its decision-making characteristics Global state It is composed of the aggregation of local observations from each node and the splicing of real-time disaster event features, and is used for centralized value estimation.

[0066] During training, a batch of empirical data is randomly sampled from the experience pool. A centralized Critic network is used to receive the global state. Output state value estimation ,express: ; In the above formula, This represents the forward mapping of the Critic network. The advantage function for each agent's action is calculated based on temporal difference or generalized advantage estimation (GAE). .

[0067] In this embodiment, the policy network is updated using the pruning objective of Multi-Agent Proximal Policy Optimization (MAPPO). For each agent, the joint probability ratio of the old and new policies is first calculated. To avoid an excessively large joint action space for multiple agents, logarithmic probability aggregation is used. Specifically, for each intelligent agent Calculate the probability ratio between the old and new strategies: ; In the above formula, represents the summation (or average) over all agents. As the current strategy, This represents the old strategy at the time of sampling. This ratio reflects the relative change in the joint strategy.

[0068] Specifically, the pruned strategy loss is expressed as: ;

[0069] In this embodiment, the reward function design does not simply provide a single reward at the end of the task based on whether the requirement has been met. Instead, it breaks down the phased goals of the emergency allocation process into multiple learnable signals. Its core objectives include: improving the final requirement completion rate, encouraging earlier completion, encouraging timely arrival of materials, reducing transportation costs, reducing ineffective actions, avoiding congestion en route, reducing over-delivery and fragmented transportation, and reducing delayed allocation.

[0070] Specifically, the incremental demand completion is used to encourage the actual fulfillment of delivery needs at each step; the incremental demand expected to be completed on time is used to encourage allocations that have been dispatched and are expected to arrive on time, providing a forward-looking incentive even if the materials have not yet actually arrived; the single-step allocation progress is used to encourage strategies to proactively promote the flow of materials under reasonable constraints; the transportation cost item is used to avoid blindly taking long-distance or high-cost routes; and the unmet demand item is used to create continuous pressure on delays in meeting demand.

[0071] Specifically, the invalid action penalty is used to suppress the behavior of selecting unavailable edges, nodes with no inventory, targets with full throughput, or targets with no downstream demand; the congestion penalty is used to limit the transportation resource occupation caused by too many materials in transit across the network; the wasteful allocation penalty is used to avoid shipping to points with already met demand; the late allocation penalty is used to make the strategy focus on task deadlines; and the fragmentation penalty is used to reduce the management costs and transportation efficiency decline caused by too many small batch shipments.

[0072] Furthermore, the reward function is calculated comprehensively based on the allocation execution results, and includes at least one or more of the following: incremental demand fulfillment (the actual amount of demand met), incremental demand expected to be fulfilled on time (the forward incentive brought by materials that have been dispatched and are expected to arrive on time), transportation costs (negative rewards), unmet demand (continuous penalties), invalid actions (such as selecting unavailable edges or nodes whose demand has already been met), late allocation (expected to exceed the task deadline), and fragmented shipments (penalties for excessively small batch allocations). The weighted sum of these rewards forms the final immediate reward. , represented as: ; Through the combined effect of multiple rewards, the agent learns that "limited resources should be sent to downstream nodes that still have real demand at the right time and through the right channels," rather than just pursuing short-term shipment volume or the nearest delivery point.

[0073] In this embodiment, the global state at each decision moment is also included. Local observations and actions of each agent, and rewards The global state at the next moment This data is stored in the experience pool as an example.

[0074] In this embodiment, the training objective of multi-agent proximal policy optimization during training can also be combined with behavior cloning, expert rule pre-training, risk-sensitive reinforcement learning, safety-constrained reinforcement learning, or model predictive control to further improve policy security and sample efficiency. The difficulty adjustment method for course learning is not limited to a fixed stage; it can also be dynamically adjusted using performance triggering, automatic course generation, or adversarial scenario generation.

[0075] To further improve the model's robustness and generalization ability in real-world high-disturbance disaster environments, a course learning mechanism was introduced during the training phase. For example... Figure 2 As shown, if the agent is directly exposed to scenarios of high demand fluctuations, multiple road interruptions, frequent addition of temporary nodes, and strictly limited transportation capacity in the early stages of training, it may have difficulty distinguishing which actions truly lead to demand fulfillment, resulting in long waiting times, random shipments, or over-reliance on a single local rule. Therefore, this application divides the training scenario into several difficulty stages. In the basic stage, network disturbances are minimal, demand changes are relatively stable, and the strategy primarily learns supply-demand matching, inventory flow, and basic path selection. In the transition stage, demand fluctuations and local path unavailability events are gradually added, prompting the strategy to learn detours, advance allocation, and multi-node collaboration. In the reinforcement stage, temporary nodes, combined events, and stronger transportation constraints are introduced, prompting the strategy to learn topology reconstruction and congestion control. In the target stage, the scenario complexity approaches that of actual emergency tasks, and the strategy undergoes robust optimization under high disturbance and high uncertainty conditions.

[0076] Specifically, the course learning does not require fixed stage ratios or parameter values. The course function can be dynamically adjusted based on training progress, strategy performance, completion rate thresholds, convergence speed, or manually set emergency drill levels. The course function can be influenced by various factors such as the probability of demand surges, the intensity of demand surges, the probability of road disruptions, the probability of temporary node activation, the degree of transportation capacity tightening, edge risks, the number of event combinations, reward / penalty weights, or scenario scale. Its form can be expressed as: difficulty(e) = Curriculum(training progress, policy performance, scene level) Through the above-mentioned learning mechanism, the strategy first masters the basic logistics rules in a stable environment, and then gradually adapts to complex disturbances, thereby effectively improving the efficiency of training samples, reducing strategy oscillations, and significantly enhancing the generalization ability to unseen disaster scenarios during deployment.

[0077] In this embodiment, dynamic events may include demand surges, road disruptions, node failures, transportation mode switching, activation of temporary supply nodes, activation of temporary warehousing nodes, and the addition of temporary delivery demand nodes. After an event occurs, there is no need to redesign the policy network; instead, by updating the dynamic graph state, the policy can make new decisions in real time within the same model framework.

[0078] Specifically, when a demand surge occurs, the demand vector of the delivery node is updated, and the residual demand propagation mechanism automatically reflects the new demand in the demand estimates of its upstream reachable nodes, thereby increasing the probability of the relevant path being selected. When a road or transport edge is interrupted, the action mask will block the unavailable edge, and the heterogeneous graph network will also perceive the path restriction through changes in edge relationships; if there are alternative routes or alternative transport modes, the allocation execution module can select alternative paths based on transport capacity and arrival time. When a temporary supply node or temporary storage node is activated, the node changes from an inactive state to an active state and is injected with the corresponding initial inventory or throughput capacity; when a temporary delivery node is added, the system adds or activates a demand node and updates the relevant edges. Since the heterogeneous graph network of this application performs local message passing based on node and edge relationships, it can better adapt to the dynamic changes of the node set without having to retrain a dedicated model for each fixed topology.

[0079] In this embodiment, a rolling time-domain approach is adopted during actual deployment. At the beginning of each decision cycle, this method obtains the latest data from the emergency command platform, warehouse management system, road traffic system, vehicle management system, and on-site reporting system, and dynamically constructs the heterogeneous directed graph for the current moment according to steps S100 to S110; then, steps S120 to S160 are executed, the policy network outputs candidate allocation actions for each active node, and the allocation execution module verifies the feasibility of the actions and generates a specific allocation plan.

[0080] Specifically, the output can include: source node, target node, material category, suggested shipment volume, suggested transportation mode, estimated departure time, estimated arrival time, estimated cost, lateness risk, target demand coverage, impact of en route congestion, and alternative route suggestions. For allocations that are unfeasible or high-risk, the system will generate clear risk warnings and suggest waiting, rerouting, using temporary nodes, or prioritizing other demand points. After execution, information such as actual shipments, arrivals, rejections, losses, changes in road conditions, and demand adjustments will be written back to the system in real time, forming the input for the next decision-making cycle. This closed-loop structure enables the allocation strategy to continuously correct errors in early planning and respond promptly to rapidly changing disaster situations.

[0081] In this embodiment, the allocation execution can also introduce additional conditions such as vehicle grouping, loading compatibility, cold chain requirements, road permits, priority requirements, material shelf life, personnel safety risks, and cross-regional administrative constraints according to the actual needs of different emergency scenarios. The core methodological framework of this method is still applicable.

[0082] like Figure 3 The diagram shown illustrates the overall process and framework of this method. Figure 3 The left side represents the input section, which includes dynamic information such as suppliers, multi-level warehouses, distribution stations, various types of materials, road network status, and emergencies in the emergency scenario. This information enters the state perception and detection prediction module through the response event stream, generating local observations and global states. The middle section uses a heterogeneous graph network to structurally embed heterogeneous nodes such as suppliers, warehouses, and distribution stations, as well as transportation edges, and supports distributed and phased execution. In the strategy decision-making phase, each active node performs multi-objective decisions and outputs mixed actions: discrete actions, such as destination selection (including waiting), and continuous actions, such as the proportion of materials dispatched. Subsequently, the environment performs allocation and generates rewards, entering the reward calculation and strategy optimization module. Reward extraction and strategy optimization employ a centralized training method to update network parameters. This figure comprehensively illustrates the complete closed-loop process of this method, from data acquisition, state awareness, graph network encoding, hybrid action decision-making, allocation execution to centralized training and optimization.

[0083] In the aforementioned heterogeneous multi-agent collaborative emergency material allocation method, firstly, by collecting real-time state information and disaster information of each node and edge in the emergency logistics network, and dynamically constructing a heterogeneous directed graph with type and relationship identifiers, it can accurately express the role differences between suppliers, multi-level warehouses, distribution stations, and temporary nodes, as well as multi-semantic relationships such as hierarchical allocation, alternative routes, and disaster impacts, thereby overcoming the deficiency of existing homogeneous graphs in not being able to distinguish the heterogeneity of nodes and edges. Secondly, it generates local observations for each node, including inventory, unmet demand, predicted arrival of materials in transit, throughput and occupancy, and late arrival risks. It also utilizes a heterogeneous graph network with different input mappings for different node types and different message passing functions for different edge types to perform multi-round message aggregation, enabling each agent to fully perceive the upstream and downstream topology, real-time congestion risks, and the expected timely arrival of materials in transit when making decisions, effectively avoiding duplicate shipments and resource waste caused by missing information. Furthermore, by embedding local observations and structures into a concatenated Actor network corresponding to node types, the output simultaneously includes discrete destination selection (including waiting actions) and a hybrid action of continuous multi-material shipment ratios. This ensures that allocation decisions directly correspond to allocation order information in actual business operations, resolving the decision mismatch problem caused by the separation or discretization of "where to send" and "how much to ship" in traditional methods. Further, by filtering effective destinations through demand-aware action masks and converting shipment ratios into actual shipment volumes based on downstream residual demand, transport edge capacity, and arrival time limits, the system can dynamically suppress invalid, over-limit, late, and fragmented allocations, thereby significantly reducing network congestion and transportation costs. Finally, by executing allocations and updating inventory, in-transit materials, and demand fulfillment in real time, a closed-loop feedback is formed, enabling the entire system to continuously make fast, robust, and globally coordinated allocation decisions under dynamic disturbances such as demand surges, road interruptions, and temporary node starts and stops. In summary, this method does indeed improve the timeliness of emergency material allocation, global coordination efficiency, and generalization ability in dynamic disaster environments, and effectively solves the key technical problems mentioned in the background technology, such as insufficient heterogeneous expression, poor dynamic adaptability, multi-agent conflict, action space fragmentation, and training instability.

[0084] It should be understood that, although Figure 1 The steps in the flowchart are shown sequentially as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order in which these steps are executed, and they can be performed in other orders. Figure 1At least some of the steps in the process may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least some of the sub-steps or stages of other steps.

[0085] In one embodiment, a heterogeneous multi-agent collaborative emergency supplies allocation system is also provided, comprising: The data acquisition module is used to collect real-time status information of each node and edge based on the emergency logistics network, and at the same time collect real-time disaster information. The dynamic graph construction module is used to dynamically construct a heterogeneous directed graph to represent the emergency logistics network based on the state information. In the heterogeneous directed graph, a type identifier is marked for each node, a relationship identifier is marked for each edge, and the state of each node and edge is updated according to the real-time disaster information. The state coding module is used to generate local observation data for each node in the heterogeneous directed graph. The local observation data includes the corresponding node's inventory, unmet demand, forecast of the arrival of goods in transit, throughput occupancy, and risk of delay. The heterogeneous graph network module is used to input the heterogeneous directed graph and the local observations of each node into the heterogeneous graph network. It adopts corresponding input mapping for different node types in the heterogeneous directed graph and corresponding message passing functions for different edge types. The structural embedding of each node is obtained through multiple rounds of message aggregation. The strategy decision module is used to concatenate the local observations of the active nodes with allocation capabilities in the heterogeneous directed graph with the corresponding structural embeddings, and then input them into the Actor network corresponding to the node type of the active node to obtain hybrid actions. The hybrid actions include: discrete destination selection and continuous multi-material shipment ratio; wherein, the destination selection is to select a target node from the downstream nodes or choose to wait. The allocation action generation module is used to determine the effective destination from the downstream nodes of the current node in the heterogeneous directed graph based on the hybrid action, through the demand-aware action mask, and convert the shipment ratio into the actual shipment volume according to the downstream residual demand, transport edge capacity and arrival time limit, and generate the allocation action. The allocation execution module is used to execute the allocation action and update the inventory, in-transit materials and demand fulfillment of each node to realize the collaborative allocation of emergency materials by heterogeneous multi-agent systems.

[0086] Specific limitations regarding the heterogeneous multi-agent collaborative emergency material allocation system can be found in the limitations of the heterogeneous multi-agent collaborative emergency material allocation method described above, and will not be repeated here. Each module in the aforementioned heterogeneous multi-agent collaborative emergency material allocation system can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the corresponding operations of each module.

[0087] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 4 As shown, the computer device includes a processor, memory, network interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The network interface is used to communicate with external terminals via a network connection. When the computer program is executed by the processor, it implements a heterogeneous multi-agent collaborative emergency material allocation method. The display screen can be an LCD screen or an e-ink screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad on the computer device's casing, or an external keyboard, touchpad, or mouse.

[0088] Those skilled in the art will understand that Figure 4 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0089] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps: The emergency logistics network collects real-time status information of each node and edge, and also collects real-time disaster information. Based on the state information, a heterogeneous directed graph is dynamically constructed to represent the emergency logistics network. In the heterogeneous directed graph, a type identifier is marked for each node, a relationship identifier is marked for each edge, and the state of each node and edge is updated according to the real-time disaster information. For each node in the heterogeneous directed graph, local observation data is generated, which includes the corresponding node's inventory, unmet demand, forecast of the arrival of goods in transit, throughput and occupancy, and risk of delay. The heterogeneous directed graph and the local observations of each node are input into the heterogeneous graph network. Corresponding input mappings are used for different node types in the heterogeneous directed graph, and corresponding message passing functions are used for different edge types. The structural embedding of each node is obtained through multiple rounds of message aggregation. After splicing the local observations of the active nodes with allocation capabilities in the heterogeneous directed graph with the corresponding structural embeddings, the data is input into the Actor network corresponding to the node type of the active nodes to obtain hybrid actions. The hybrid actions include: discrete destination selection and continuous multi-material shipment ratio; wherein, the destination selection is to select a target node from the downstream nodes or to choose to wait. Based on the hybrid action, the effective destination is determined from the downstream nodes of the current node in the heterogeneous directed graph through the demand-aware action mask, and the shipment ratio is converted into the actual shipment volume according to the downstream residual demand, transport edge capacity and arrival time limit, and the transfer action is generated. The aforementioned allocation action is executed, and the inventory, in-transit materials, and demand fulfillment of each node are updated to achieve collaborative emergency material allocation among heterogeneous multi-agent systems.

[0090] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, the computer program performing the following steps when executed by a processor: The emergency logistics network collects real-time status information of each node and edge, and also collects real-time disaster information. Based on the state information, a heterogeneous directed graph is dynamically constructed to represent the emergency logistics network. In the heterogeneous directed graph, a type identifier is marked for each node, a relationship identifier is marked for each edge, and the state of each node and edge is updated according to the real-time disaster information. For each node in the heterogeneous directed graph, local observation data is generated, which includes the corresponding node's inventory, unmet demand, forecast of the arrival of goods in transit, throughput and occupancy, and risk of delay. The heterogeneous directed graph and the local observations of each node are input into the heterogeneous graph network. Corresponding input mappings are used for different node types in the heterogeneous directed graph, and corresponding message passing functions are used for different edge types. The structural embedding of each node is obtained through multiple rounds of message aggregation. After splicing the local observations of the active nodes with allocation capabilities in the heterogeneous directed graph with the corresponding structural embeddings, the data is input into the Actor network corresponding to the node type of the active nodes to obtain hybrid actions. The hybrid actions include: discrete destination selection and continuous multi-material shipment ratio; wherein, the destination selection is to select a target node from the downstream nodes or to choose to wait. Based on the hybrid action, the effective destination is determined from the downstream nodes of the current node in the heterogeneous directed graph through the demand-aware action mask, and the shipment ratio is converted into the actual shipment volume according to the downstream residual demand, transport edge capacity and arrival time limit, and the transfer action is generated. The aforementioned allocation action is executed, and the inventory, in-transit materials, and demand fulfillment of each node are updated to achieve collaborative emergency material allocation among heterogeneous multi-agent systems.

[0091] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0092] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0093] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A heterogeneous multi-agent collaborative emergency material allocation method, characterized in that, The method includes: The emergency logistics network collects real-time status information of each node and edge, and also collects real-time disaster information. Based on the state information, a heterogeneous directed graph is dynamically constructed to represent the emergency logistics network. In the heterogeneous directed graph, a type identifier is marked for each node, a relationship identifier is marked for each edge, and the state of each node and edge is updated according to the real-time disaster information. For each node in the heterogeneous directed graph, local observation data is generated, which includes the corresponding node's inventory, unmet demand, forecast of the arrival of goods in transit, throughput and occupancy, and risk of delay. The heterogeneous directed graph and the local observations of each node are input into the heterogeneous graph network. Corresponding input mappings are used for different node types in the heterogeneous directed graph, and corresponding message passing functions are used for different edge types. The structural embedding of each node is obtained through multiple rounds of message aggregation. After splicing the local observations of the active nodes with allocation capabilities in the heterogeneous directed graph with the corresponding structural embeddings, the data is input into the Actor network corresponding to the node type of the active nodes to obtain hybrid actions. The hybrid actions include: discrete destination selection and continuous multi-material shipment ratio; wherein, the destination selection is to select a target node from the downstream nodes or to choose to wait. Based on the hybrid action, the effective destination is determined from the downstream nodes of the current node in the heterogeneous directed graph through the demand-aware action mask, and the shipment ratio is converted into the actual shipment volume according to the downstream residual demand, transport edge capacity and arrival time limit, and the transfer action is generated. The aforementioned allocation action is executed, and the inventory, in-transit materials, and demand fulfillment of each node are updated to achieve collaborative emergency material allocation among heterogeneous multi-agent systems.

2. The heterogeneous multi-agent collaborative emergency material allocation method according to claim 1, characterized in that, When constructing the heterogeneous directed graph: The node types include at least two of the following: supply nodes, warehousing nodes, delivery demand nodes, and temporary nodes. The edge types include at least one or more of the following: hierarchical allocation edge, alternative route edge, and disaster impact edge.

3. The heterogeneous multi-agent collaborative emergency material allocation method according to claim 1, characterized in that, The determination of a valid destination through a demand-aware action mask includes: Based on the state information of the active node and its corresponding downstream node, downstream nodes that meet the demand-aware action mask are determined as valid destinations, wherein the demand-aware action mask is: If the active node is active and has shippable inventory, and the active node's throughput is not saturated, and the downstream node is active and its throughput is not saturated, and the downstream node or its downstream has residual demand, then the corresponding downstream node is determined as a valid destination. And, the waiting action is judged as a valid destination.

4. The heterogeneous multi-agent collaborative emergency material allocation method according to claim 1, characterized in that, When calculating the downstream residual demand: The residual demand of a delivery node is obtained by subtracting the demand already met and the goods in transit that are expected to arrive on time from the total demand of the delivery node. Along the downstream reachable path of the heterogeneous directed graph, the residual demand of each delivery demand node is propagated or aggregated upstream; The residual demand is obtained by deducting the existing inventory at each node along the propagation or aggregation path and eliminating the portion that cannot be delivered on time based on the transportation time limit.

5. The heterogeneous multi-agent collaborative emergency material allocation method according to claim 1, characterized in that, In the mixed action: The continuous multi-material shipment ratio is obtained by sampling the Beta distribution output by the Actor network; The discrete destination selection is obtained by sampling the classification distribution output by the Actor network.

6. The heterogeneous multi-agent collaborative emergency material allocation method according to any one of claims 1-5, characterized in that, The method also includes a training phase: Each active node in the heterogeneous directed graph is treated as an agent, and a reinforcement learning framework with centralized training and distributed execution is used to train the heterogeneous graph network and each Actor network. During training, at each decision moment, each agent generates and executes hybrid actions through its respective Actor network based on the current local observations and the structural embedding output by the heterogeneous graph network, thereby obtaining a reward and entering the next state. The local observations, global states, actions, rewards, and next states of each agent are stored as experience data in the experience pool, wherein the global state is generated based on the local observations of each node and the real-time disaster information. The global state is received using a centralized Critic network, the global value is estimated, and the advantage function is calculated. The pruning target is optimized using a multi-agent proximal strategy. Based on the empirical data, the strategy loss and value loss are calculated, and the heterogeneous graph network, the Actor networks of each agent, and the centralized Critic network are jointly updated.

7. The heterogeneous multi-agent collaborative emergency material allocation method according to claim 6, characterized in that, The training phase also includes: adopting a course learning mechanism to dynamically adjust the complexity of the environment scenario according to the training progress, gradually increasing the intensity of demand surges, the frequency of road interruptions, the probability of temporary node activation, and the complexity of event combinations, so that the training transitions from a low-disturbance scenario to a high-disturbance scenario.

8. A heterogeneous multi-agent collaborative emergency material allocation system, characterized in that, The system includes: The data acquisition module is used to collect real-time status information of each node and edge based on the emergency logistics network, and at the same time collect real-time disaster information; The dynamic graph construction module is used to dynamically construct a heterogeneous directed graph to represent the emergency logistics network based on the state information. In the heterogeneous directed graph, a type identifier is marked for each node, a relationship identifier is marked for each edge, and the state of each node and edge is updated according to the real-time disaster information. The state coding module is used to generate local observation data for each node in the heterogeneous directed graph. The local observation data includes the corresponding node's inventory, unmet demand, forecast of the arrival of goods in transit, throughput occupancy, and risk of delay. The heterogeneous graph network module is used to input the heterogeneous directed graph and the local observations of each node into the heterogeneous graph network. It adopts corresponding input mapping for different node types in the heterogeneous directed graph and corresponding message passing functions for different edge types. The structural embedding of each node is obtained through multiple rounds of message aggregation. The strategy decision module is used to concatenate the local observations of the active nodes with allocation capabilities in the heterogeneous directed graph with the corresponding structural embeddings, and then input them into the Actor network corresponding to the node type of the active node to obtain hybrid actions. The hybrid actions include: discrete destination selection and continuous multi-material shipment ratio; wherein, the destination selection is to select a target node from the downstream nodes or choose to wait. The allocation action generation module is used to determine the effective destination from the downstream nodes of the current node in the heterogeneous directed graph based on the hybrid action, through the demand-aware action mask, and convert the shipment ratio into the actual shipment volume according to the downstream residual demand, transport edge capacity and arrival time limit, and generate the allocation action. The allocation execution module is used to execute the allocation action and update the inventory, in-transit materials and demand fulfillment of each node to realize the collaborative allocation of emergency materials by heterogeneous multi-agent systems.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

Images