A transferable multi-agv task allocation method based on transformer entity encoding

Abstract

This invention proposes a transferable multi-AGV task allocation method based on Transformer entity encoding. The steps include: establishing a system model for multi-AGV collaborative task allocation and constructing an entity state model; considering local perception, capacity constraints, and global collaborative allocation requirements in the multi-AGV system, establishing a distributed partially observable Markov decision process; designing a multi-AGV collaborative allocation algorithm for dynamic task allocation environments based on a multi-agent deep reinforcement learning framework that integrates entity encoding and Transformer multi-head self-attention mechanisms, and outputting task allocation decision results; jointly optimizing the task allocation decision results, and deploying the trained task allocation model in scenarios with different numbers of AGVs and task scales. This invention achieves efficient multi-AGV collaborative task allocation under capacity-constrained scenarios and improves the transferability of the model in scenarios with different numbers of AGVs and cross-scale tasks.

Description

Technical Field

[0001] This invention relates to the technical field of multi-agent collaborative decision-making in intelligent manufacturing and intelligent warehousing logistics scheduling, and in particular to a transferable multi-AGV task allocation method based on Transformer entity encoding. Background Technology

[0002] In intelligent warehousing and logistics environments, multi-AGV task allocation differs from single-AGV scheduling or static allocation problems. Each task typically corresponds to a fixed service location and has capacity requirements, meaning a specified number of AGVs must simultaneously or collaboratively bind together to complete the task. In such scenarios, each AGV can usually only obtain entity status information within a local area and cannot know the strategy parameters or decision intentions of other AGVs. Furthermore, parallel decision-making by multiple AGVs can lead to task capacity competition, causing multiple AGVs to potentially select the same task simultaneously, resulting in capacity conflicts, invalid binding, and decreased resource utilization. Therefore, how to achieve efficient collaborative allocation under local observation conditions, while balancing task completion utility and system response time, has become a pressing technical problem to be solved.

[0003] Existing multi-AGV task allocation methods can be broadly categorized into traditional optimization methods and learning-based methods. Traditional optimization methods mainly include mathematical programming methods, heuristic optimization methods, and rule-based distributed methods. These methods can achieve certain results in scenarios with high levels of structure and relatively limited environmental changes, but they typically rely on complete environmental information, accurate system models, or manually designed rules. When environmental conditions, task requirements, or system scale change, it is often necessary to re-solve, re-tune parameters, or redesign rules, thus making it difficult to meet the real-time and adaptability requirements of dynamic manufacturing scenarios.

[0004] In recent years, deep reinforcement learning and multi-agent reinforcement learning have provided new solutions for multi-AGV task allocation. Through centralized training and distributed execution frameworks, multi-AGV systems can learn collaborative strategies using global information during the training phase and make independent decisions based on local observations during the execution phase. However, existing learning methods still have significant shortcomings in multi-AGV task allocation scenarios: First, they typically concatenate AGV features and task features directly into a fixed-dimensional vector input to the network, lacking a unified modeling capability for the heterogeneous interaction and capacity competition relationships between AGVs and tasks, making it difficult to effectively characterize the dependencies between entities; second, under conditions of local observability and parallel decision-making, agents struggle to fully aggregate global entity context information, easily leading to inefficient competition; third, existing network structures are usually strongly coupled with the number of AGVs and tasks, often requiring adjustments to the network structure or retraining the model when the system scale changes, resulting in low cross-scenario deployment efficiency and insufficient generalization ability.

[0005] Especially in flexible manufacturing and smart warehousing scenarios, the number of AGVs and tasks dynamically changes with production plans, equipment status, and logistics needs. If the task allocation model relies on fixed input dimensions or fixed action output structures, it is difficult to reuse it directly when the scale of the scenario changes. The network structure must be redesigned or the parameters must be retrained, resulting in long deployment cycles and high adaptation costs, making it difficult to meet the needs of practical engineering applications for rapid migration and online deployment. Summary of the Invention

[0006] To address the technical problems of low efficiency and insufficient generalization ability in task allocation across scenarios for multi-AGV systems under capacity constraints and local observability conditions, this invention proposes a transferable multi-AGV task allocation method based on Transformer entity encoding. Under capacity constraints and local observability conditions, this method utilizes graph structured observation, entity encoding mechanism, Transformer multi-head self-attention mechanism, and entity-level value decomposition mechanism to achieve efficient collaborative task allocation and cross-scale transfer deployment of multi-AGVs in capacity-constrained scenarios, and improves the model's transfer deployment capability in scenarios with different numbers of AGVs and cross-scale tasks.

[0007] To achieve the above objectives, the technical solution of the present invention is implemented as follows: a transferable multi-AGV task allocation method based on Transformer entity encoding, the steps of which include:

[0008] S1: Establish a system model for multi-AGV collaborative task allocation, abstract AGVs and tasks as heterogeneous entity nodes, and construct an entity state model that includes information on position, speed, participation status, capacity characteristics and entity category.

[0009] S2: Considering the local perception, capacity constraints and global collaborative allocation requirements in a multi-AGV system, and combining the parallel decision-making characteristics in the task allocation process, a graph-structured observation space is constructed, and a distributed partially observable Markov decision process is established.

[0010] S3: Based on a multi-agent deep reinforcement learning framework that integrates entity encoding mechanism and Transformer multi-head self-attention mechanism, a multi-AGV collaborative allocation algorithm for dynamic task allocation environment is designed. Through a noisy Transformer agent network, the action value of candidate tasks is generated and the task allocation decision result is output.

[0011] S4: Entity-level value decomposition is performed through the entity-aware Transformer mixer. The task allocation decision results are jointly optimized and trained by combining centralized training and distributed execution. After the training meets the termination condition, the task allocation model is obtained and deployed in scenarios with different numbers of AGVs and task scales.

[0012] The method for constructing the entity state model is as follows: the system model in the multi-AGV collaborative task allocation scenario includes an edge server, M AGVs, and N tasks, and the M AGVs and N tasks are uniformly represented as an entity set. Time Entity status for:

[0013] ;

[0014] in, Representing entities Location; Representing entities For the AGV entity, the speed must satisfy... , This represents the maximum speed; for a task entity, the speed is... ; This indicates the participation status of an entity. For AGV entities, it indicates the binding status: 0 means the AGV entity is not bound to a task, and 1 means the AGV entity is bound to a task. For task entities, it indicates the completion status: 0 means the task entity has not been completed, and 1 means the task entity has been completed. This represents capacity-related characteristics. For AGV entities, it indicates the binding and locking status. For task entities, it indicates the ratio of the number of bound AGVs to the capacity requirement. Indicates entity type, Represents the AGV entity. Represents the task entity.

[0015] The distributed partially observable Markov decision process includes a graph-structured observation space, an action space, and a reward function.

[0016] exist time, The graph-structured observation space is represented as an entity feature matrix:

[0017] ;

[0018] in, For the first One AGV entity, It is the transpose operator for a matrix. Let be the set of real numbers. For feature dimension, For the number of entities, For entities exist Node features in a reference frame;

[0019] entity The node characteristics are:

[0020] ;

[0021] in, Representing entities Compared to The relative position, For entities Location, for Location; Representing entities Compared to relative velocity, For entities speed, for The speed of the entity; when the entity The relative speed when observing the AGV itself. When the entity When it is a task entity, relative velocity ; Indicates its own identifying characteristics, when an entity For the first When there are multiple AGV entities, ,otherwise ; The entity category indicates the characteristic, when the entity When it is an AGV entity, When the entity When it is a task entity, ; For capacity-related features, for AGV entities, it represents the bound / locked state; for task entities, it represents the current capacity occupancy state. Let the task... The capacity requirement is , The number of AGVs that are currently bound is , Indicates capacity ratio, Indicates the task completion status;

[0022] The total observation space of the system is represented as: ,in, Number of AGV entities;

[0023] exist time, Action selection based on local graph-structured observation space: ;in, Indicates the task entity Designated as The target task, The number of task entities;

[0024] All AGVs are Individual actions at any given moment collectively constitute the coordinated actions of the system. As a space for action;

[0025] exist At any given time, the reward function consists of a feasibility consistency penalty, a utility event reward, and an allocation progress reward:

[0026] ;

[0027] in, for time Feasibility consistency penalty item, for time Utility event reward items, for time The allocation of progress rewards.

[0028] The feasibility consistency penalty item is:

[0029] ;

[0030] in, The penalty coefficient for illegal actions. The penalty coefficient for full-capacity tasks. for The action, To exclude valid action sets after completing or exceeding the capacity of tasks, For the task Capacity requirements, For a moment Task The number of AGVs already bound. For indicator functions;

[0031] The reward for the utility event is as follows:

[0032] ;

[0033] in, Based on the reward coefficient, This is the marginal capacity coefficient. for The successful binding indicator function, This is the remaining capacity item;

[0034] The progress reward item is as follows:

[0035] ;

[0036] in, To gradually select the cost, For signal weights, and They are time points and time Assign task The value of the task-aware scoring function; the allocation of progress reward items only applies to unbound AGVs. For bound AGVs, the allocation of progress reward items... .

[0037] The implementation method of the multi-AGV collaborative allocation algorithm for dynamic task allocation environment is as follows:

[0038] S31: Construct an entity encoding layer to perform unified feature mapping on the entity feature matrix in the graph-structured observation space and output the entity embedding sequence;

[0039] S32: Construct the interactive modeling input sequence by concatenating the hidden state of the previous time step with the entity embedding sequence output by the entity encoding layer to obtain the input sequence;

[0040] S33: Design an entity interaction modeling module based on the Transformer multi-head self-attention mechanism to model the input sequence and obtain the updated entity interaction representation;

[0041] S34: Calculate the action value of candidate tasks based on entity interaction representation and output the multi-AGV task allocation decision results.

[0042] The method for calculating the action value of candidate tasks based on entity interaction representation is as follows:

[0043] exist At any given moment, the graph will be used to structure the observation space. entity feature matrix The input entity encoding layer encodes the features of each entity node through a parameter-shared linear transformation. The encoding result is:

[0044] ;

[0045] in, For entity node features, For the number of entities, and These are the encoding weights and biases, respectively. For the total embedding dimension, Dimensions of entity node features;

[0046] The entity embedding sequence output by the entity coding layer is: ;

[0047] exist At any given moment, the previous state will be hidden. Entity embedding sequence concatenated to the output of the entity coding layer Front-end, constructing the input sequence for interactive modeling: ;

[0048] The method for obtaining the updated entity interaction representation based on the Transformer multi-head attention mechanism is as follows:

[0049] Input sequence for interaction modeling Perform independent linear projections to obtain the query matrix, key matrix, and value matrix; calculate the attention weights based on the query matrix and key matrix; calculate the output of a single attention head based on the attention weights and value matrix.

[0050] The outputs of each attention point are concatenated, mapped using an output projection matrix, and then compared with the input sequence for interactive modeling. Perform residual connection fusion to obtain the updated entity interaction representation. ;

[0051] Through updated entity interaction representation Extracting candidate tasks corresponding to entity interaction representations entity representation ,noise The head represents the task entity through a shared linear layer. Mapping to corresponding The value of an individual action in a candidate task is calculated as follows:

[0052] ;

[0053] in, and These are the shared projection weights and biases, respectively. For the overall embedding dimension, actions For the selected target task.

[0054] The method for jointly optimizing and training the task allocation decision results is as follows:

[0055] S41: Initialize the parameters of the noisy Transformer agent network and the entity perception Transformer mixer, and create the target network; initialize the experience replay buffer; set the discount factor, soft update coefficient, priority index, importance sampling index, and noisy network parameters;

[0056] S42: At the beginning of each training round, initialize the environment and obtain the initial state; calculate the individual action value of the candidate task through a noisy Transformer agent network based on the entity feature matrix in the graph structured observation space and the hidden state of the previous time step; and aggregate the individual action values ​​into a joint action value based on the entity perception Transformer mixer.

[0057] S43: Select and execute joint actions based on the value of joint actions, obtain the global team reward and the next global state returned by the environment, construct state transition samples and store them in the experience replay buffer; sample a batch of samples from the experience replay buffer according to the sample sampling priority; calculate the temporal difference error of the samples by combining the target network and the Double Q-Learning mechanism and update the sample sampling priority;

[0058] S44: Construct training loss based on temporal difference error, use Adam optimizer with gradient pruning to update network parameters of noisy Transformer agent network and entity perception Transformer mixer, and perform soft update on target network parameters according to preset soft update coefficients;

[0059] S45: Repeat steps S42 to S44 until the preset number of training rounds is reached or the training termination condition is met; after training is completed, during the execution phase, each AGV outputs the task allocation decision result based on the local entity feature matrix and the trained noisy Transformer agent network parameters.

[0060] The method for aggregating individual action values ​​into joint action values ​​based on the entity-aware Transformer mixer is as follows:

[0061] The entity-aware Transformer mixer utilizes an entity encoding layer to integrate the states of each entity. The global state constituted Encoding as an entity embedding matrix ,in For the number of entities, For the embedding dimension; the input sequence of the entity-aware Transformer mixer is:

[0062] ;

[0063] in, for The hidden state; Number of AGV entities; Three cyclic supertokens are initialized to zero vectors at the start of each round and propagated step by step over time; the input sequence is... The output sequence is obtained after processing by the entity-aware Transformer mixer. ; will be before Output tokens Stacking forms a weight matrix ,in Let be the dimension of the hidden layer vectors in the value aggregation subnetwork; transform the three cyclic super tokens Bias generated by projection Weight matrix With bias All weight matrices are nonnegated using an absolute value function. ;

[0064] The value of the combined action is:

[0065] ;

[0066] in, For hidden layer output and , This is the transpose of the weight matrix. For bias, For individuals Value vector, A function of the exponential linear unit, This is the global state. The combined actions of all AGVs constitute the system's coordinated actions. Indicates network parameters; This indicates that the network parameters are The global state is Joint actions The value of combined actions.

[0067] The method for selecting joint actions based on joint action value is as follows:

[0068] Parameters of noisy Transformer agent networks Initialize and use it as the main network, then construct the target network parameters. ;

[0069] The main network is Choose joint actions based on their value at all times:

[0070] ;

[0071] in, This indicates that the network parameters are The global state is Joint actions The value of combined actions;

[0072] The method for calculating the temporal difference error of samples and updating the sample sampling priority by combining the target network and the Double Q-Learning mechanism is as follows:

[0073] Target network in Calculate the single-step temporal difference objective at each moment:

[0074] ;

[0075] in, For the overall team reward, among which For the number of AGV entities, for time The reward function, For round end indicator, Discount factor; This indicates that the network parameters are The global state is Joint actions The value of combined actions;

[0076] Introducing multi-step rewards:

[0077] ;

[0078] in, For multi-step return coefficients, For the number of steps in a multi-step reward, for Step-by-step time difference objective;

[0079] Based on multi-step returns The first in the computational experience replay buffer Temporal difference error of each sample:

[0080] ;

[0081] Based on timing difference error The absolute value calculation experience replay buffer Sampling priority of each sample:

[0082] ;

[0083] in, , The number of samples in the experience replay buffer. To prevent zero priority, This is the control coefficient for the priority distribution.

[0084] The training loss is:

[0085] ;

[0086] in, For importance sampling weights, The size of the experience buffer, For the first Sampling priority of each sample As an importance sampling index, For multi-step returns, This indicates that the network parameters are The global state is Joint actions The value of joint actions under these circumstances This indicates the expectation of the sampled batch. For experience replay buffer, For the reward function;

[0087] Soft update of target network parameters using soft update coefficients: ,in For the target network parameters, Main network parameters, This is the soft update coefficient.

[0088] The beneficial effects of this invention are as follows:

[0089] 1. This invention proposes a transferable multi-AGV task allocation method based on Transformer entity encoding. Under capacity constraints and local observability conditions, it can uniformly represent AGVs and tasks as heterogeneous entities and model the dynamic interaction relationships between entities through graph-structured observation space, thereby improving the collaborative task allocation capability of multi-AGV systems in complex dynamic environments.

[0090] 2. This invention introduces an entity interaction modeling method based on the Transformer multi-head self-attention mechanism, which can effectively learn the association and competition relationships between AGV entities and between AGV entities and task entities. Under local observation conditions, it realizes information aggregation and task matching relationship learning, reduces ineffective competition among multiple AGVs for the same task, and improves task allocation efficiency.

[0091] 3. This invention has strong scalability and portability. Through the entity encoding mechanism with parameter sharing and the entity-level action value decoupling mechanism, the network structure is decoupled from the number of AGVs, the number of tasks, and the scale of the scene. It can be migrated and deployed without redesigning the network structure in different scenarios with different numbers of AGVs and task scales, thereby improving the generalization ability and engineering application value of the model.

[0092] 4. This invention combines centralized training and distributed execution mechanisms, and introduces noise mechanisms and priority experience replay mechanisms, which can improve the exploration efficiency, sample utilization and training stability during the training process, thereby improving the overall performance and operational stability of multi-AGV systems in dynamic task allocation scenarios.

[0093] This invention enables unified entity modeling of AGVs and tasks, and decouples network structure from the number of entities through parameter-sharing entity encoding, Transformer-based entity relationship modeling, and entity-level action value decoupling mechanism, thereby achieving efficient collaborative task allocation and cross-scale scalable deployment under capacity constraints. Attached Figure Description

[0094] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0095] Figure 1 This is a diagram of the intelligent factory multi-AGV collaborative task allocation architecture of the present invention.

[0096] Figure 2 This is a schematic diagram of the multi-AGV task allocation algorithm for dynamic task allocation environments according to the present invention.

[0097] Figure 3 This is a flowchart illustrating the joint optimization of task allocation decision results according to the present invention. Detailed Implementation

[0098] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0099] like Figure 1 As shown, this invention proposes a portable multi-AGV task allocation method based on Transformer entity encoding. This method can perform unified entity modeling for AGVs and tasks, and decouples the network structure from the number of entities through parameter-shared entity encoding, Transformer-based entity relationship modeling, and an entity-level action value decoupling mechanism. This enables efficient collaborative task allocation and cross-scale portable deployment under capacity constraints. The invention specifically includes the following steps:

[0100] S1: Establish a system model for multi-AGV collaborative task allocation, abstract AGVs and tasks as heterogeneous entity nodes, and construct an entity state model that includes information on position, speed, participation status, capacity characteristics, and entity category.

[0101] like Figure 1 As shown, in order to describe the multi-AGV collaborative task allocation problem under capacity constraints and distributed decision-making conditions in an intelligent warehousing and logistics environment, a system model consisting of an edge server, multiple AGVs, and multiple transportation tasks is first established.

[0102] The system model for multi-AGV task allocation scenarios includes an edge server, M AGVs, and N tasks. The edge server is used for centralized training of the task allocation model during the training phase and supports model deployment during the execution phase. The M AGVs perform distributed task selection based on local observation information. The M AGVs and N tasks are uniformly represented as an entity set, and the state of any entity k at time step t is represented as... ,in, and Let the position vector and velocity vector of entity k be respectively. For the AGV entity, the velocity satisfies the following condition: , Indicates the maximum speed. The value is set according to the scenario; for task entities, speed... The location of the task is set according to the scene. This indicates the participation status of an entity. For AGV entities, it indicates the binding status: 0 means the AGV entity is not bound to a task, and 1 means the AGV entity is bound to a task. For task entities, it indicates the completion status: 0 means the task has not been completed, and 1 means the task has been completed. This represents capacity-related characteristics, specifically the bound / locked state for AGV entities, i.e., when the AGV enters the bound radius of the task. The binding relationship is triggered within the specified time. After successful binding, the AGV is locked to the current task and will no longer participate in the task selection for this round. This variable of the AGV entity is determined by... To characterize it, when constructing a unified entity feature matrix, the feature of the AGV entity is encoded in a fixed-dimensional form consistent with that of the task entity; for the task entity, it represents the ratio of the number of bound AGVs to the capacity requirement. Identify entity type, Represents the AGV entity. Represents the task entity.

[0103] Each task j has a fixed service location. and capacity requirements The capacity requirement refers to the number of AGVs that need to be bound simultaneously to complete the corresponding task. When an AGV enters the binding radius of the target task and the remaining capacity of the target task meets the requirement, a binding relationship is established. After successful binding, the AGV is locked to the target task and no longer participates in the task selection of the current round, thus forming a system model for multi-AGV collaborative task allocation under capacity constraints. The capacity constraint arises because of the parallel decision-making of multiple AGVs. When multiple AGVs simultaneously select the same task and the remaining capacity of that task is insufficient, capacity competition will occur. Concurrent binding exceeding the capacity limit will lead to ineffective movement and decreased resource utilization efficiency.

[0104] The system's optimization objective is to maximize task utility while minimizing completion time within a finite time domain. Allocation quality is evaluated using two normalized metrics. Normalized Average Task Utility (NATU) measures the trade-off between task completion benefits and movement costs. The utility of a task is defined as:

[0105] ,

[0106] in, For the task The completed revenue, for For the task Indicator function that completes valid binding. for The cumulative movement costs incurred during the approach to the mission The distance cost coefficient is used. The Normalized Average Task Utility (NATU) is defined as...

[0107] ,

[0108] in, and These are the theoretical lower and lower bounds of utility, respectively.

[0109] Normalized Average Task Completion (NATC) measures system responsiveness and is defined as follows:

[0110] ,

[0111] in, for The binding is successful at the specified time; if binding fails within the specified time range, then... . This represents the theoretical minimum task completion time during a single task allocation process. This represents the maximum allowed duration of a single task assignment process. The system optimization objective is to maximize NATU while minimizing NATC. Each AGV plans the optimal task selection based on local observations at each decision step, dynamically balancing task completion efficiency, movement cost, and responsiveness.

[0112] The above optimization problem must satisfy the following constraints:

[0113] ,

[0114] in, To bind the distance threshold, For the task Capacity requirements, for Always bound to the task The number of AGVs. Indicates the time after effective binding The entity participation state of the i-th AGV. Constraint C1 limits the maximum speed of the AGV to meet physical constraints; constraint C2 enforces the upper limit of task capacity to prevent over-allocation; constraint C3 defines the binding trigger condition, and a binding relationship can only be established when the distance between the AGV and the target task meets the binding radius requirement; constraint C4 ensures the persistence of the binding, and once the AGV reaches the target task at time i... Once a valid binding is achieved, its binding state remains unchanged at all subsequent moments; constraint C5 restricts the upper bound of the decision-making time domain. The system optimization objective does not directly participate in the loss function calculation of the reinforcement learning network, but serves as the basis for the overall performance evaluation of the task allocation problem; its specific implementation method is as follows: optimization requirements such as improving task utility, reducing completion time, and satisfying capacity constraints are transformed into utility event reward items, allocation progress reward items, and feasibility consistency penalty items in the reward function, respectively, and the system is indirectly driven to converge toward the optimization objective by maximizing the cumulative team reward.

[0115] S2: Considering the local perception, capacity constraints and global collaborative allocation requirements in a multi-AGV system, and combining the parallel decision-making characteristics in the task allocation process, a graph-structured observation space is constructed, and a distributed partially observable Markov decision process is established.

[0116] Since the multi-AGV system is under locally observable conditions, each AGV can only obtain entity state information within a local range and cannot know the strategy parameters and future actions of other AGVs. Therefore, this invention models the multi-AGV task allocation problem as a distributed partially observable Markov decision process in the following steps.

[0117] The distributed partially observable Markov decision process includes a graph-structured observation space, action space, and reward function. In a system consisting of M AGVs and N tasks, the M AGVs and N tasks are unified into an entity set, and the total number of entities is denoted as K=M+N. Each AGV independently selects tasks based on local observation information and achieves collaborative task allocation in capacity-constrained scenarios under multi-agent parallel decision-making conditions.

[0118] At time t, The graph-structured observation space is represented as an entity feature matrix, which is defined as follows:

[0119] ,

[0120] in, It is the transpose operator for a matrix. Let be the set of real numbers. For feature dimension, Indicates that entity k is in Node characteristics within a reference frame. In a factory environment, AGVs have their own sensors and can upload and download data via a server. For example, an AGV can obtain its own position and speed through sensors, and request the position and speed of other entities within its sensing radius from the server, thereby constructing... The entity feature matrix of the observation space.

[0121] The node characteristics of entity k are defined as follows:

[0122] ,

[0123] in, Indicates entity k relative to The relative position, express The location. Indicates entity k relative to relative velocity, express Speed: When entity k is the currently observed AGV itself hour( ), retaining its absolute speed To support its own motion state estimation; when entity k is a task entity, its velocity vector Position and velocity are both two-dimensional vectors, calculated by subtraction. The relative position and relative velocity with other entities. The self-identification feature is used to mark whether entity k is the currently observed AGV itself. =1 Otherwise, its own identifying features This enables the network to distinguish its own nodes from other entities under a shared parameter encoding mechanism. The entity category indicator feature is used to distinguish whether entity k is an AGV entity or a task entity. At that time, entity k is an AGV entity. When entity k is the task entity, the self-attention mechanism can differentiate between heterogeneous entity categories. and Although both represent entity categories, It is a representation in indicating the participation state of an entity, and Is entity k in Category representation under observation. Capacity-related features are used to characterize the AGV binding and locking status or the current capacity occupancy status of a task: Let the task... The capacity requirement is The number of AGVs already bound at time t is The capacity ratio and completion status are defined as follows: ,in, Indicates the task completion status; for AGV entities, Semantically representing the binding and locking state, it can be derived from... Characterization; when constructing a unified entity feature matrix, the feature of the AGV entity is encoded in a fixed-dimensional format consistent with that of the task entity. For the task entity, capacity ratio and completion status... The first dimension Indicates the binding progress, second dimension This indicates the task completion status. The representation organizes entity features into a matrix structure, where feature dimensions and the number of entities are independent, and features of the same type are processed uniformly by sharing parameters.

[0124] The total observation space of the system is represented as: ,in Represents the time t. indivual Entity feature matrix in the observation space.

[0125] At time t, The action is selected based on the current local observation status, and is denoted as: ,in, Indicates the task Designated as The current navigation target.

[0126] The individual actions of all AGVs at time t constitute the combined action of the system: .

[0127] System joint action Distributed control signals are generated at each moment. This coordinated action specifically refers to each AGV moving to the task target location according to its assigned task, and the control signals are the synthesized traction force described below. This is used to drive the evolution of a multi-AGV system towards the target state. For AGVs that have already locked their binding relationships, their decision channels will be closed to ensure that the entire system can converge to the complete allocation state within a limited time domain.

[0128] Discrete decision-making is mapped to continuous control through deterministic dynamics modules. Discrete decision-making refers to... Select action based on the current local observation status. This represents which task is being bound. When the decision specifies the target task... hour, The dynamics engine is based on the synthesis of traction force using relative position vectors:

[0129] ,

[0130] in, This is the sensitivity coefficient. for With the target task The distance between them for With the target task The distance between them. The velocity update equation is:

[0131] ,

[0132] in, Let be the speed of the i-th AGV at time t. Let be the speed of the i-th AGV at time t+1. For AGV quality, Let t be the time change corresponding to a given time. The position state evolves through velocity integral:

[0133] ,

[0134] in, Let i be the position of the i-th AGV at time t+1. Let be the position of the i-th AGV at time t.

[0135] By using the above method, the motion execution problem of AGVs moving to the target task location is simplified to generating traction force based on the target point, thereby moving to the target task location. Reinforcement learning focuses on task allocation, maintains physical feasibility, and realizes task selection and motion execution of multiple AGVs under local observation conditions.

[0136] The reward function is designed by comprehensively considering the system's task allocation efficiency, capacity constraint satisfaction, and allocation schedule.

[0137] To improve the efficiency and stability of collaborative task allocation among multiple AGVs under capacity constraints, the single-step reward function at time step t consists of a feasibility consistency penalty term, a utility event reward term, and an allocation progress reward term. Each part corresponds to a different optimization objective. The overall allocation performance of the system is maximized by combining these objectives, as expressed in:

[0138] ,

[0139] in, for Feasibility consistency penalty item for Utility event reward items, for The allocation of progress rewards.

[0140] First, the feasibility consistency penalty term is used to suppress illegal task selection and invalid competition for full-capacity tasks. At time step t, The feasibility consistency penalty term is expressed as:

[0141] ,

[0142] in, The penalty coefficient for illegal actions. The penalty coefficient for full-capacity tasks. and All are non-negative numbers. For the currently selected target task, express The action, To exclude valid action sets after completing or exceeding the capacity of tasks, For the task Capacity requirements, For task t The number of AGVs already bound. This is an indicator function; it takes a value of 1 if the condition within the parentheses is met, and a value of 0 if the condition is not met.

[0143] Secondly, the utility event reward item is used to provide a positive reward when the AGV is successfully bound to a task, at time step t. The successful binding indicator function is represented as:

[0144] ,

[0145] in, for With the target task The distance between them To be at time step t Task binding status, For time step t-1 Task binding status, The distance threshold is a non-negative number, and the task is defined as follows: The remaining capacity before binding is represented as The utility event reward item is then represented as:

[0146] ,

[0147] in, Based on the reward coefficient, This is the marginal capacity coefficient. and All are non-negative numbers. (Through the remaining capacity term) Higher rewards are given to tasks with more remaining slots, guiding AGVs to distribute to tasks that are under-allocated rather than those where competition is saturated.

[0148] Both the feasibility consistency term and the utility event reward term exhibit inherent sparsity: the feasibility consistency penalty is triggered only when the constraint is violated, and the utility event reward is only obtained at the binding moment, resulting in low exploration efficiency in the high-dimensional joint action space.

[0149] Next, a dense allocation progress reward item is introduced to continuously evaluate the task selection decisions of each AGV before binding.

[0150] The allocation schedule reward includes a stepwise selection cost and a differential allocation quality signal, wherein the stepwise selection cost is expressed as:

[0151] ,

[0152] in, For map dimensions, the map in this embodiment is set to 40 m × 40 m. A base cost coefficient is used to penalize excessively long allocation processes. This is the distance cost coefficient. and All are non-negative numbers. for The distance from time step t to the task. To be at time step t Location. This provides information related to... A persistent negative signal proportional to the current distance to the target task reflects the allocation cost of choosing a distant task.

[0153] The differential allocation quality signal is constructed based on a task-aware scoring function, which is expressed as:

[0154] ,

[0155] in, The capacity scoring coefficient is a non-negative number. The first term measures the spatial proximity to the target task, and the second term measures the remaining capacity availability of the target task. Unlike the stepwise selection cost, which reflects the absolute allocation cost at each time step, the differential form... By capturing changes in task selection quality across consecutive time steps, directional feedback is provided regarding whether the current allocation decision has improved. The complete allocation progress reward is then defined as follows:

[0156] ,

[0157] in, The signal weight is a non-negative number. and These represent the values ​​of the task-aware scoring function for task j at time step t and time step t-1, respectively. The allocation of progress rewards only applies to unbound AGVs; for bound AGVs, ... .

[0158] By designing the reward function described above, a unified constraint is achieved to suppress illegal selection, incentivize successful binding, and continuously optimize the task allocation process, thereby improving the efficiency of multi-AGV task allocation and cross-scenario adaptability.

[0159] S3: Based on a multi-agent deep reinforcement learning framework that integrates entity encoding mechanism and Transformer multi-head self-attention mechanism, a multi-AGV collaborative allocation algorithm for dynamic task allocation environment is designed. Through a noisy Transformer agent network, the action value of candidate tasks is generated and the task allocation decision result is output.

[0160] To address the capacity conflicts and efficiency losses caused by multiple AGVs competing for the same task under distributed decision-making conditions, this embodiment models the task allocation problem as a matching learning problem on an entity graph. AGVs and tasks are represented as graph nodes, and the association weights between nodes are learned through a self-attention mechanism. The algorithm's network architecture is as follows: Figure 2 As shown, cross-scale generalization is achieved through three key mechanisms: First, a parameter-sharing entity encoding layer is adopted to decouple encoder complexity from the number of entities; second, the collaborative relationship between AGVs and the matching and competition relationship between AGVs and tasks are jointly modeled through the Transformer multi-head self-attention mechanism; third, an entity-level action value decoupling mechanism is adopted to decouple network parameters from scene scale.

[0161] The implementation method of the multi-AGV collaborative allocation algorithm for dynamic task allocation environments is as follows:

[0162] S31: Construct an entity encoding layer to perform unified feature mapping on the entity feature matrix in the graph-structured observation space and output the entity embedding sequence.

[0163] At time step t, the graph structured observation space is... entity feature matrix The input entity encoding layer encodes the features of each entity node through a parameter-shared linear transformation. The encoding result is represented as:

[0164] ,

[0165] in, For entity node features, and These are the encoding weights and biases, respectively. For the total embedding dimension, Let be the dimension of the entity node features. The encoded entity embedding representation matrix sequence is denoted as . The entity encoding method maps different entities to a semantic space of a preset dimension, providing a unified input representation for entity interaction modeling. This design decouples encoder parameters from the number of entities.

[0166] S32: Construct the interactive modeling input sequence by concatenating the hidden state of the previous time step with the entity embedding sequence output by the entity encoding layer to obtain the input sequence.

[0167] At time step t, to introduce temporal dependencies, the hidden state from the previous time step is... Concatenate to the entity embedding sequence output by the entity coding layer The front-end constructs an input based on the Transformer multi-head self-attention mechanism, represented as... This is used to fuse historical information with current entity feature information. Among them, the hidden state from the previous time step... It is output by a neural network.

[0168] S33: Design an entity interaction modeling module based on the Transformer multi-head self-attention mechanism to model the input sequence and obtain the updated entity interaction representation.

[0169] The self-attention layer computes the interaction features between entities through a multi-head attention mechanism. For the first... Each attention head, for the input sequence Perform independent linear projections to obtain the query, key, and value matrices, which are represented as follows: , and .in, , and They represent the first Each attention head corresponds to a query projection matrix, a key projection matrix, and a value projection matrix. These parameter weights are continuously optimized through reinforcement learning. Attention weights are calculated based on the query and key matrices, and are represented as follows: ,in The feature dimension of a single attention head. Represents the total embedding dimension. The number of attention heads is indicated. Among the attention weights, the weights between AGV entities represent the cooperative relationship, and the weights between AGV entities and task entities represent the intensity of task competition. The output of each attention head is represented as The outputs of each attention head are concatenated and mapped using the output projection matrix, then fused with the input sequence via residual connection. This process allows for the introduction of new relational information learned by the attention mechanism while preserving the original entity information, resulting in an updated entity interaction representation.

[0170] ,

[0171] in, This represents the output projection matrix. The updated entity interaction representation. Extract the updated hidden state representation and task entity representation according to the sequence index, where index 0 corresponds to the concatenated hidden state, and indices 1 to... Corresponding AGV entity representation, index Corresponding task The entity representation; the updated hidden state representation and task entity representation are respectively represented as and It is used for calculating the value of task actions. Represents the updated entity interaction representation Chinese index The corresponding entity representation.

[0172] S34: Calculate the action value of candidate tasks based on entity interaction representation and output the multi-AGV task allocation decision results.

[0173] For candidate tasks Through updated entity interaction representation Extract the corresponding task entity representation. ,noise The head represents the task entity through a shared linear layer. Mapping to corresponding The value is represented as:

[0174] ,

[0175] in, and To share projection weights and biases, For the total embedding dimension, The final target task selected for the strategy. And select the largest one. The value is the action value of the target task.

[0176] This design decouples network parameters from the number of tasks. Models trained in the task scenario can be directly deployed to In this task scenario, no network structure adjustment is required. To enhance exploration capabilities in the high-dimensional task selection space, noise... The deterministic parameters of the head projection layer are reparameterized by introducing learnable random perturbations. Specifically, each weight element is represented as the sum of a trainable mean component and a trainable scale component modulated by factorized Gaussian noise: ,in For learnable mean parameters, For learnable noise intensity parameters, It is Gaussian noise. This mechanism replaces the traditional... The strategy allows the exploration intensity to be automatically adjusted through gradient descent.

[0177] well trained The network will output The target action with the highest value is the target task assigned to each AGV.

[0178] S4: Entity-level value decomposition is performed through the entity-aware Transformer mixer. The task allocation decision results are jointly optimized and trained by combining centralized training and distributed execution. After the training meets the termination condition, the task allocation model is obtained and deployed in scenarios with different numbers of AGVs and task scales.

[0179] like Figure 2 As shown, the algorithm's network architecture consists of two core modules: a noisy Transformer agent network and an entity-aware Transformer mixer. The noisy Transformer agent network is responsible for the distributed decision-making of each AGV, as described in S1-S3; the entity-aware Transformer mixer is responsible for global value aggregation and evaluation, that is, dynamically generating hybrid weights based on the global entity participation state, aggregating the individual action values ​​of each AGV into a joint action value, which is used to evaluate the overall merits of the multi-AGV joint task allocation decision, as described in S4.

[0180] The noisy Transformer agent network employs a parameter-sharing mechanism, where all AGVs share the same set of network parameters. Based on the entity feature matrix in the graph-structured observation space described in S2 and the hidden state from the previous time step described in S3, the action value of each AGV's candidate task is generated, and the action value with the highest value is selected. The value is the action value of the target task.

[0181] The entity-aware Transformer mixer aggregates the action values ​​of each AGV's target task based on the global state to evaluate the global value of the joint action. During the training phase, the entity-aware Transformer mixer has access to the global state to estimate the joint action value and optimize the agent network. During the execution phase, the entity-aware Transformer mixer does not participate in online decision-making; each AGV generates its own target task action value using only its local entity feature matrix and the trained shared agent network, thus achieving distributed execution.

[0182] like Figure 3 As shown, the method for jointly optimizing task allocation decision results according to the present invention includes the following steps:

[0183] S41: Initialize the parameters of the noisy Transformer agent network and the entity perception Transformer mixer, and create the target network; initialize the experience replay buffer; set the discount factor, soft update coefficient, priority index, importance sampling index, and noisy network parameters.

[0184] The noisy Transformer agent network employs orthogonal initialization to avoid gradient explosion, while the entity-aware Transformer mixer uses random initialization. The target network is a delayed copy of the main network, used during training to compute a more stable TD target value during Q-learning updates, thus implementing the Double Q-Learning mechanism. The experience replay buffer stores historical interaction data between the AGV and its environment for network training. The importance sampling index is gradually annealed to 1 during training, and the noisy network parameters are randomly initialized. In this embodiment, the discount factor is 0.99, the soft update coefficient is 0.005, and the priority index is 0.6.

[0185] S42: At the beginning of each training round, initialize the environment and obtain the initial state; calculate the individual action value of the candidate task through a noisy Transformer agent network based on the entity feature matrix in the graph structured observation space and the hidden state of the previous time step; and aggregate the individual action values ​​into a joint action value based on the entity perception Transformer mixer.

[0186] The entity-aware Transformer mixer first processes the states of each entity. The global state constituted Using an entity encoding layer neural network, the global state is... Encode to obtain the entity embedding matrix ,in For the embedding dimension. The global state encompasses the capacity usage of all tasks, the spatial distribution of all AGVs, and the task completion progress. The input sequence of the entity-aware Transformer mixer is defined as:

[0187] ,

[0188] in, for The hidden state encodes the local decision-making intent. Three cyclic supertokens, initialized to a zero vector at the start of the round and then... Propagation across time steps. Input sequence The output sequence is obtained after processing by the Transformer mixing module. .forward Output tokens Stacking forms a weight matrix ,in The dimension of the hidden layer vector in the value aggregation subnetwork represents the entity perception. The Transformer mixer is a holistic network module used to generate joint action values ​​based on the global state and individual action values; the value aggregation subnetwork within this module is responsible for integrating the individual actions of each AGV. Value mapping to joint action value ); Transformed super token Bias generated by projection Weight matrix With bias All weight matrices are nonnegated using an absolute value function to satisfy the monotonicity constraint of the Individual-Global-Max (IGM) principle. .

[0189] The process by which the entity-aware Transformer mixer aggregates the individual action values ​​of candidate tasks into a joint action value can be represented as follows:

[0190] ,

[0191] ,

[0192] in, For individuals The value vector, i.e., the target task output by the noisy Transformer agent network in step S34. value, For hidden layer output, This is the global state. The combined actions of all AGVs constitute the system's coordinated actions. Represents network parameters, This indicates that the network parameters are When the global state is Joint actions In this case, the value of the combined action calculated using the above formula is expressed using an exponential linear unit. A nonlinear transformation is introduced as the activation function. Using this method, a joint value evaluation of the multi-AGV task allocation decision results is performed during the intensive training phase.

[0193] S43: Select and execute joint actions based on the value of joint actions, obtain the global team reward and the next global state returned by the environment, construct state transition samples and store them in the experience replay buffer; sample a batch of samples from the experience replay buffer according to the sample sampling priority; calculate the temporal difference error of the samples by combining the target network and the Double Q-Learning mechanism and update the sample sampling priority.

[0194] The training and update method for jointly optimizing the task allocation decision results is as follows:

[0195] During the training phase, network parameters are optimized by minimizing temporal difference error based on the Q-learning framework. First, the parameters of the noisy Transformer agent network are optimized. Initialize it as the main network and construct the target network parameters. The target network is a delayed replica of the primary network; target network parameters. Regularly update parameters from the main network parameters using soft methods. To ensure training stability, a Double Q-Learning mechanism is employed to construct joint action selection and temporal difference objectives. The main network selects a joint action at the next time step based on the joint action value, as represented by:

[0196] ,

[0197] The target network computation single-step temporal difference objective is represented as:

[0198] ,

[0199] in, As a reward for the entire team, For round end indicator, This is the discount factor. This indicates that the network parameters are The global state is Joint actions The value of joint actions under certain circumstances.

[0200] To reduce training variance and integrate short-term and long-term reward information, a multi-step reward is introduced, represented as:

[0201] ,

[0202] in, This represents the multi-step return coefficient, with a value ranging from 0 to 1. This indicates the number of steps in a multi-step reward. This represents the n-step time-difference objective, which is obtained by weighted summation of time-difference objectives of different time lengths.

[0203] The first step is obtained by sampling from the experience replay buffer based on multi-step reward calculation. The temporal difference error of each sample is expressed as:

[0204] ,

[0205] And based on the timing difference error, the first Each sample is assigned a sampling priority, expressed as follows:

[0206] ,

[0207] in, , The number of samples in the current experience replay buffer. To prevent zero priority, The control coefficients represent the priority distribution.

[0208] Sampling priorities are assigned based on the absolute value of the temporal difference error of each sample in the empirical replay buffer. The larger the absolute value of the temporal difference error, the higher the sampling priority of the sample, and the greater the probability of it being sampled during training. Sample batches are then transferred from the empirical replay buffer according to this sampling priority, and the importance sampling weights described in S44 are applied. Correction is performed for sampling bias caused by priority sampling. This includes adjusting the importance sampling weight. This is used to reduce the impact of oversampling of high-priority samples on the loss function, thereby correcting sampling bias.

[0209] S44: The training loss is constructed based on the temporal difference error. The Adam optimizer with gradient pruning is used to update the network parameters of the noisy Transformer agent network and the entity perception Transformer mixer. The target network parameters are then softly updated according to the preset soft update coefficient.

[0210] The network parameters are updated using the temporal difference error mean square loss with importance sampling weights. The training loss is expressed as:

[0211] ,

[0212] in, For importance sampling weights, The size of the experience buffer, The importance sampling index is gradually annealed to 1 during training to fully correct for sampling bias. To calculate the expectation of the sampled batch, based on the training loss, the network parameters of the noisy Transformer agent network and the entity-aware Transformer mixer are updated using the Adam optimizer with gradient clipping to prevent gradient explosion.

[0213] The target network parameters are then softly updated according to a preset soft update coefficient. Specifically, during each update, the target network parameters are not directly and completely copied from the main network, but rather slowly move closer to the main network. ,in, For the target network parameters, Main network parameters, This represents the soft update coefficient. This soft update mechanism improves the stability of the training process. Through the above method, the training and updating of a multi-AGV task allocation model based on entity-level value decomposition combined with centralized training and distributed execution is achieved.

[0214] S45: Repeat steps S42 to S44 until the preset number of training rounds is reached or the training termination condition is met; after training is completed, during the execution phase, each AGV outputs the task allocation decision result based on the local entity feature matrix and the trained noisy Transformer agent network parameters.

[0215] The portability of the task allocation model is achieved through the following method: the entity encoding layer uses a parameter sharing method to uniformly encode AGV entities and task entities. The action value of each AGV for the candidate task is represented by the corresponding task entity and calculated independently through shared value mapping parameters, thereby decoupling the network parameters from the number of AGVs, the number of tasks, and the scale of the scene. Therefore, in target scenes with different numbers of AGVs and task scales, the trained task allocation model can maintain a unified entity encoding method and action value generation method without redesigning the network structure, and can achieve cross-scale migration and deployment.

[0216] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A Transformer entity coding-based transferable multi-AGV task allocation method, characterized in that, Includes the following steps: S1: Establish a system model for multi-AGV collaborative task allocation, abstract AGVs and tasks as heterogeneous entity nodes, and construct an entity state model that includes information on position, speed, participation status, capacity characteristics and entity category. S2: Considering the local perception, capacity constraints and global collaborative allocation requirements in a multi-AGV system, and combining the parallel decision-making characteristics in the task allocation process, a graph-structured observation space is constructed, and a distributed partially observable Markov decision process is established. S3: Based on a multi-agent deep reinforcement learning framework that integrates entity encoding mechanism and Transformer multi-head self-attention mechanism, a multi-AGV collaborative allocation algorithm for dynamic task allocation environment is designed. Through a noisy Transformer agent network, the action value of candidate tasks is generated and the task allocation decision result is output. S4: Entity-level value decomposition is performed through the entity-aware Transformer mixer. The task allocation decision results are jointly optimized and trained by combining centralized training and distributed execution. After the training meets the termination condition, the task allocation model is obtained and deployed in scenarios with different numbers of AGVs and task scales.

2. The transferable multi-AGV task allocation method based on Transformer entity encoding according to claim 1, characterized in that, The method for constructing the entity state model is: the system model in the multi-AGV cooperative task allocation scene includes an edge server, M AGVs and N tasks, and the M AGVs and the N tasks are uniformly represented as an entity set, Time entity The state is: ； in, Representing entities Location; Representing entities For the AGV entity, the speed must satisfy... , This represents the maximum speed; for a task entity, the speed is... ; This indicates the participation status of an entity. For AGV entities, it indicates the binding status: 0 means the AGV entity is not bound to a task, and 1 means the AGV entity is bound to a task. For task entities, it indicates the completion status: 0 means the task entity has not been completed, and 1 means the task entity has been completed. This represents capacity-related characteristics. For AGV entities, it indicates the binding and locking status. For task entities, it indicates the ratio of the number of bound AGVs to the capacity requirement. Indicates entity type, Represents the AGV entity. Represents the task entity.

3. The Transformer entity encoding-based transferable multi-AGV task allocation method according to claim 2, characterized in that, The distributed partially observable Markov decision process includes a graph-structured observation space, an action space, and a reward function. In At The graph structured observation space of the entity is represented as an entity feature matrix: ； in, For the first One AGV entity, It is the transpose operator for a matrix. Let be the set of real numbers. For feature dimension, For the number of entities, For entities exist Node features in a reference frame; Entity The node features of the entity are: ； in, Representing entities Compared to The relative position, For entities Location, for Location; Representing entities Compared to relative velocity, For entities speed, for The speed of the entity; when the entity The relative speed when observing the AGV itself. When the entity When it is a task entity, relative velocity ; Indicates its own identifying characteristics, when an entity For the first When there are multiple AGV entities, ,otherwise ; The entity category indicates the characteristic, when the entity When it is an AGV entity, When the entity When it is a task entity, ; For capacity-related features, for AGV entities, it represents the bound / locked state; for task entities, it represents the current capacity occupancy state. Let the task... The capacity requirement is , The number of AGVs that are currently bound is , Indicates capacity ratio, Indicates the task completion status; The total observation space of the system is represented as: ,in, Number of AGV entities; exist time, Action selection based on local graph-structured observation space: ;in, Indicates the task entity Designated as The target task, The number of task entities; All AGVs are Individual actions at any given moment collectively constitute the coordinated actions of the system. As a space for action; exist At any given time, the reward function consists of a feasibility consistency penalty, a utility event reward, and an allocation progress reward: ； in, for time Feasibility consistency penalty item for time Utility event reward items, for time The allocation of progress rewards.

4. The transferable multi-AGV task allocation method based on Transformer entity encoding according to claim 3, characterized in that, The feasibility consistency penalty item is: ； in, The penalty coefficient for illegal actions. The penalty coefficient for full-capacity tasks. for The action, To exclude valid action sets after completing or exceeding the capacity of tasks, For the task Capacity requirements, For a moment Task The number of AGVs already bound. For indicator functions; The reward for the utility event is: ； in, Based on the reward coefficient, This is the marginal capacity coefficient. for The successful binding indicator function, This is the remaining capacity item; The allocation progress reward item is: ； in, To gradually select the cost, For signal weights, and They are time points and time Assign task The value of the task-aware scoring function; the allocation of progress reward items only applies to unbound AGVs. For bound AGVs, the allocation of progress reward items... .

5. The transferable multi-AGV task allocation method based on Transformer entity encoding according to claim 1 or 4, characterized in that, The implementation method of the multi-AGV collaborative allocation algorithm for dynamic task allocation environment is as follows: S31: Construct an entity encoding layer to perform unified feature mapping on the entity feature matrix in the graph-structured observation space and output the entity embedding sequence; S32: Construct the interactive modeling input sequence by concatenating the hidden state of the previous time step with the entity embedding sequence output by the entity encoding layer to obtain the input sequence; S33: Design an entity interaction modeling module based on the Transformer multi-head self-attention mechanism to model the input sequence and obtain the updated entity interaction representation; S34: Calculate the action value of candidate tasks based on entity interaction representation and output the multi-AGV task allocation decision results.

6. The transferable multi-AGV task allocation method based on Transformer entity encoding according to claim 5, characterized in that, The method for calculating the action value of candidate tasks based on entity interaction representation is as follows: exist At any given moment, the graph will be used to structure the observation space. entity feature matrix The input entity encoding layer encodes the features of each entity node through a parameter-shared linear transformation. The encoding result is: ； in, For entity node features, For the number of entities, and These are the encoding weights and biases, respectively. For the total embedding dimension, Dimensions of entity node features; The entity embedding sequence output by the entity coding layer is: ; exist At any given moment, the previous state will be hidden. Entity embedding sequence concatenated to the output of the entity coding layer Front-end, constructing the input sequence for interactive modeling: ; The method for obtaining the updated entity interaction representation based on the Transformer multi-head attention mechanism is as follows: Input sequence for interaction modeling Perform independent linear projections to obtain the query matrix, key matrix, and value matrix; calculate the attention weights based on the query matrix and key matrix; calculate the output of a single attention head based on the attention weights and value matrix. The outputs of each attention point are concatenated, mapped using an output projection matrix, and then compared with the input sequence for interactive modeling. Perform residual connection fusion to obtain the updated entity interaction representation. ; Through updated entity interaction representation Extracting candidate tasks corresponding to entity interaction representations entity representation ,noise The head represents the task entity through a shared linear layer. Mapping to corresponding The value of an individual action in a candidate task is calculated as follows: ； in, and These are the shared projection weights and biases, respectively. For the overall embedding dimension, actions For the selected target task.

7. The transferable multi-AGV task allocation method based on Transformer entity encoding according to claim 1 or 6, characterized in that: The method for jointly optimizing and training the task allocation decision results is as follows: S41: Initialize the parameters of the noisy Transformer agent network and the entity perception Transformer mixer, and create the target network; initialize the experience replay buffer; set the discount factor, soft update coefficient, priority index, importance sampling index, and noisy network parameters; S42: At the beginning of each training round, initialize the environment and obtain the initial state; calculate the individual action value of the candidate task through a noisy Transformer agent network based on the entity feature matrix in the graph structured observation space and the hidden state of the previous time step; and aggregate the individual action values ​​into a joint action value based on the entity perception Transformer mixer. S43: Select and execute joint actions based on the value of joint actions, obtain the global team reward and the next global state returned by the environment, construct state transition samples and store them in the experience replay buffer; sample a batch of samples from the experience replay buffer according to the sample sampling priority; calculate the temporal difference error of the samples by combining the target network and the Double Q-Learning mechanism and update the sample sampling priority; S44: Construct training loss based on temporal difference error, use Adam optimizer with gradient pruning to update network parameters of noisy Transformer agent network and entity perception Transformer mixer, and perform soft update on target network parameters according to preset soft update coefficients; S45: Repeat steps S42 to S44 until the preset number of training rounds is reached or the training termination condition is met. After training is completed, during the execution phase, each AGV outputs the task allocation decision result based on the local entity feature matrix and the trained noisy Transformer agent network parameters.

8. The transferable multi-AGV task allocation method based on Transformer entity encoding according to claim 7, characterized in that, The method for aggregating individual action values ​​into joint action values ​​based on the entity-aware Transformer mixer is as follows: The entity-aware Transformer mixer utilizes an entity encoding layer to integrate the states of each entity. The global state constituted Encoding as an entity embedding matrix ,in For the number of entities, For the embedding dimension; the input sequence of the entity-aware Transformer mixer is: ； in, for The hidden state; Number of AGV entities; Three cyclic supertokens are initialized to zero vectors at the start of each round and propagated step-by-step over time; the input sequence is... The output sequence is obtained after processing by the entity-aware Transformer mixer. ; will be before Output tokens Stacking forms a weight matrix ,in Let be the dimension of the hidden layer vectors in the value aggregation subnetwork; transform the three cyclic super tokens Bias generated by projection Weight matrix With bias All weight matrices are nonnegated using an absolute value function. ; The value of the combined action is: ； in, For hidden layer output and , This is the transpose of the weight matrix. For bias, For individuals Value vector, A function of the exponential linear unit, This is the global state. The combined actions of all AGVs constitute the system's coordinated actions. Indicates network parameters; This indicates that the network parameters are The global state is Joint actions The value of combined actions.

9. The transferable multi-AGV task allocation method based on Transformer entity encoding according to claim 8, characterized in that, The method for selecting joint actions based on joint action value is as follows: Parameters of noisy Transformer agent networks Initialize and use it as the main network, then construct the target network parameters. ; The main network is Choose joint actions based on their value at all times: ； in, This indicates that the network parameters are The global state is Joint actions The value of joint actions under these circumstances; The method for calculating the temporal difference error of samples and updating the sample sampling priority by combining the target network and the Double Q-Learning mechanism is as follows: Target network in Calculate the single-step temporal difference objective at each moment: ； in, For the overall team reward, among which For the number of AGV entities, for time The reward function, For round end indicator, Discount factor; This indicates that the network parameters are The global state is Joint actions The value of joint actions under these circumstances; Introducing multi-step rewards: ； in, For multi-step return coefficients, For the number of steps in a multi-step reward, for Step-by-step time difference objective; Based on multi-step returns The first in the computational experience replay buffer Temporal difference error of each sample: ； Based on timing difference error The absolute value calculation experience replay buffer Sampling priority of each sample: ； in, , The number of samples in the experience replay buffer. To prevent zero priority, This represents the control coefficient for the priority distribution.

10. The transferable multi-AGV task allocation method based on Transformer entity encoding according to claim 9, characterized in that, The training loss is: ； in, For importance sampling weights, The size of the experience buffer, For the first Sampling priority of each sample The importance sampling index, For multi-step returns, This indicates that the network parameters are The global state is Joint actions The value of joint actions under these circumstances This indicates the expectation of the sampled batch. For experience replay buffer, For the reward function; Soft update of target network parameters using soft update coefficients: ,in For the target network parameters, Main network parameters, This is the soft update coefficient.

Images