Storage robot distribution method and system

Abstract

The invention discloses a storage robot distribution method and system, and relates to the technical field of intelligent storage logistics. According to the method, the states of robots, the states of workstations and the fatigue state and the pressure state of a human picker in the warehousing system are obtained firstly, then real-time local observation of all the robots is deduced based on a pre-trained multi-agent reinforcement learning strategy network, distribution actions are generated, and finally the robots are dynamically distributed to the corresponding workstations. Therefore, collaborative optimization of order throughput maximization and human picker occupational health risk minimization is realized. Thus, through accurate capturing and dynamic decision making of multi-dimensional real-time state information by a multi-agent reinforcement learning strategy network, a robot distribution strategy can be adaptive to fatigue-pressure coupling change of a picker, and malignant evolution of the physiological state of the picker is effectively slowed down while load unbalance of a workstation is avoided, so that the accuracy and the reliability of the picker are improved. And the effect of bidirectionally improving the operation efficiency and human health is achieved.

Description

Technical Field

[0001] This application relates to the field of intelligent warehousing and logistics technology, and in particular to a method and system for distributing warehousing robots. Background Technology

[0002] In modern warehousing and logistics, the Robotic Mobile Order Fulfillment System (RMFS) has become a core piece of equipment for improving operational efficiency. It eliminates the inefficiency of personnel movement in the traditional "person-to-goods" operation through the "goods-to-person" model. However, the bottleneck of system operating efficiency has shifted to the human picking operation. The rationality of the robot allocation strategy directly determines the workload of the pickers, thereby affecting their physical and mental state.

[0003] Existing robot allocation schemes have significant shortcomings: most strategies focus only on economic indicators such as order throughput, employing simplified human performance models and failing to fully consider the dynamic evolution and two-way reinforcement mechanisms of picker fatigue and stress; some studies focusing on human factors are also limited to a single dimension, failing to characterize the synergistic effect of fatigue and stress and the dynamics of recovery. Non-optimal allocation strategies easily lead to a vicious cycle of physical fatigue and mental stress among pickers, resulting in problems such as continuous decline in efficiency, order backlog, and increased risk of workplace injuries, making it difficult to achieve synergistic optimization of operational efficiency and human health. At the same time, the RMFS contains a large-scale swarm of mobile robots, and the system state has randomness (such as the uncertainty of pickers' physiological state), which places stringent requirements on real-time assignment decisions for large-scale swarm robots, and existing technical methods are unable to meet this real-time decision-making requirement. Summary of the Invention

[0004] To address the above problems, this application provides a warehouse robot allocation method and system, including the following: In a first aspect, this application provides a method for allocating warehouse robots, the method comprising: The system acquires real-time status information of the warehousing system, which includes at least the robot's own status, the workstation status, and the physiological status of the human picker, including fatigue and stress. The real-time state information is processed based on a pre-trained multi-agent reinforcement learning policy network to generate a robot allocation policy; According to the robot allocation strategy, robots are dynamically allocated to corresponding workstations to achieve synergistic optimization of maximizing order throughput and minimizing occupational health risks for human pickers. The occupational health risks include the cumulative duration of high-pressure conditions and the cumulative amount of fatigue.

[0005] Optionally, the multi-agent reinforcement learning policy network is trained using a centralized training-distributed execution architecture, and the training process includes: Initialize the replay buffer, model parameters, and hyperparameters. The model parameters are single parameters of the neural networks of hybrid networks, robot networks, and hypernetworks, and are updated during the training process. The hyperparameters are fixed control parameters preset before training and are not updated during the training process. The warehouse simulation environment is reset to its initial state, and the global initial state is obtained. After entering the iterative training loop, each robot selects an action based on the current strategy in each step. The current strategy includes an exploration mechanism. All robots synchronously execute the selected actions to form a joint action. After receiving the joint action, the environment returns the global reward and the next state. The system stores the current state, joint action, reward, next state, and termination flag as experience tuples in a centralized replay buffer. When the preset training period is reached, data is sampled from the replay buffer, the action value Q of the prediction network and the target network is calculated, and the loss function is minimized through gradient descent to update the model parameters; the parameters of the target network and the prediction network are periodically synchronized until the training reaches the preset termination condition.

[0006] Optionally, the physiological state of the human picker is quantitatively characterized by a kinetic model, which includes: The pressure response model is based on the Yerkes-Dodson law, and the pressure level is linearly related to the workstation queue length. The efficiency modulation factor adopts a Gaussian response model. The fatigue dynamic model shows that fatigue accumulates exponentially during the working phase and decays exponentially during the rest phase, with fatigue accumulation being amplified by stress. The stress-fatigue-efficiency coupling model integrates the impact of stress response and fatigue dynamics on picking efficiency, achieving a quantitative correlation between physiological state and work efficiency.

[0007] Optionally, the network structure of the multi-agent reinforcement learning policy network includes an agent network and a hybrid network. The agent network adopts a deep recurrent Q-network architecture, integrating gated recurrent units to capture temporal dependencies, inputting local observation information and outputting action value Q-value estimates for each action. The hybrid network integrates a masking mechanism and a multi-head attention module to aggregate the individual Q-values ​​of each agent into a global Q-value, achieving synergy between distributed decision-making and global optimization.

[0008] Optionally, the processing procedure of the hybrid network includes: In the state-aware feature encoding stage, the features of the robot and the workstation are encoded separately, and non-standby robots are masked through a masking mechanism. In the multi-head attention mechanism stage, the encoded features are mapped to multiple subspaces to dynamically model the relationship between robot requirements and workstation states. In the hybrid output stage, individual Q-values ​​are weighted and aggregated based on attention features, and a state-aware bias term is introduced to generate a global Q-value.

[0009] Optionally, the robot's own status in the real-time status information includes: robot position coordinates, target shelf coordinates, and busy status; The workstation status includes queue length, number of robots in transit, and workstation identifier; The fatigue state of the human pickers is characterized by a cumulative fatigue quantification value, and the stress state is characterized by a stress level quantification value.

[0010] Optionally, the reward function of the multi-agent reinforcement learning policy network adopts a composite reward mechanism, including a global reward item and an auxiliary reward item. The global reward item integrates throughput deviation, stress exposure degree and fatigue rest events. The auxiliary reward item is designed differently according to the action type, and the assigned action is penalized according to the workstation congestion degree and stress state, and the waiting action is given targeted reward according to the system state.

[0011] Optionally, the optimization objective of the robot allocation strategy is defined by a composite objective function: ,in This refers to the order throughput within the operating cycle T. The duration of total pressure. This represents the total accumulated fatigue. , , The configurable weighting coefficients allow for the adjustment of optimization weights for efficiency and health risks.

[0012] Optionally, the real-time status information of the warehousing system is obtained through a discrete event simulation framework. The simulation framework models a dynamic interaction process and triggers an allocation decision when the robot returns to the storage area to wait for a new task. The dynamic interaction process includes robot movement, shelf transportation, order picking, and physiological status updates.

[0013] Secondly, this application provides a warehouse robot distribution system, which includes: The acquisition unit is used to acquire real-time status information of the warehousing system. The real-time status information includes at least the robot's own status, the workstation status, and the physiological status of the human picker. The physiological status includes fatigue and stress. The processing unit is used to process the real-time state information based on a pre-trained multi-agent reinforcement learning policy network to generate a robot allocation policy. The allocation unit is used to dynamically allocate robots to corresponding workstations according to the robot allocation strategy, so as to achieve synergistic optimization of maximizing order throughput and minimizing occupational health risks of human pickers. The occupational health risks include the cumulative duration of high-pressure conditions and the cumulative amount of fatigue.

[0014] Optionally, the multi-agent reinforcement learning policy network is trained using a centralized training-distributed execution architecture, and the training process of the processing unit for the multi-agent reinforcement learning policy network includes: Initialize the replay buffer, model parameters, and hyperparameters. The model parameters are single parameters of the neural networks of hybrid networks, robot networks, and hypernetworks, and are updated during the training process. The hyperparameters are fixed control parameters preset before training and are not updated during the training process. The warehouse simulation environment is reset to its initial state, and the global initial state is obtained. After entering the iterative training loop, each robot selects an action based on the current strategy in each step. The current strategy includes an exploration mechanism. All robots synchronously execute the selected actions to form a joint action. After receiving the joint action, the environment returns the global reward and the next state. The system stores the current state, joint action, reward, next state, and termination flag as experience tuples in a centralized replay buffer. When the preset training period is reached, data is sampled from the replay buffer, the action value Q of the prediction network and the target network is calculated, and the loss function is minimized through gradient descent to update the model parameters; the parameters of the target network and the prediction network are periodically synchronized until the training reaches the preset termination condition.

[0015] Optionally, the physiological state of the human picker is quantitatively characterized by a dynamic model, which includes: a stress response model based on the Yerkes-Dodson law, where stress level is linearly related to workstation queue length, and the efficiency modulation factor adopts a Gaussian response model; a fatigue dynamic model, where fatigue accumulates exponentially during the working phase and decays exponentially during the rest phase, and fatigue accumulation is amplified by stress state; and a stress-fatigue-efficiency coupling model, which integrates the influence of stress response and fatigue dynamics on picking efficiency to achieve a quantitative correlation between physiological state and work efficiency.

[0016] Optionally, the network structure of the multi-agent reinforcement learning policy network includes an agent network and a hybrid network. The agent network adopts a deep recurrent Q-network architecture, integrating gated recurrent units to capture temporal dependencies, inputting local observation information and outputting action value Q-value estimates for each action. The hybrid network integrates a masking mechanism and a multi-head attention module to aggregate the individual Q-values ​​of each agent into a global Q-value, achieving synergy between distributed decision-making and global optimization.

[0017] Optionally, the processing procedure of the hybrid network includes: In the state-aware feature encoding stage, the features of the robot and the workstation are encoded separately, and non-standby robots are masked through a masking mechanism. In the multi-head attention mechanism stage, the encoded features are mapped to multiple subspaces to dynamically model the relationship between robot requirements and workstation states. In the hybrid output stage, individual Q-values ​​are weighted and aggregated based on attention features, and a state-aware bias term is introduced to generate a global Q-value.

[0018] Optionally, the robot's own status in the real-time status information includes: robot position coordinates, target shelf coordinates, and busy status; The workstation status includes queue length, number of robots in transit, and workstation identifier; The fatigue state of the human pickers is characterized by a cumulative fatigue quantification value, and the stress state is characterized by a stress level quantification value.

[0019] Optionally, the reward function of the multi-agent reinforcement learning policy network adopts a composite reward mechanism, including a global reward item and an auxiliary reward item. The global reward item integrates throughput deviation, stress exposure degree and fatigue rest events. The auxiliary reward item is designed differently according to the action type, and the assigned action is penalized according to the workstation congestion degree and stress state, and the waiting action is given targeted reward according to the system state.

[0020] Optionally, the optimization objective of the robot allocation strategy is defined by a composite objective function: ,in This refers to the order throughput within the operating cycle T. The duration of total pressure. This represents the total accumulated fatigue. , , The configurable weighting coefficients allow for the adjustment of optimization weights for efficiency and health risks.

[0021] Optionally, the acquisition unit is specifically used to acquire real-time status information of the warehousing system through a discrete event simulation framework. The simulation framework models a dynamic interaction process and triggers an allocation decision when the robot returns to the storage area to wait for a new task. The dynamic interaction process includes robot movement, shelf transportation, order picking, and physiological status updates.

[0022] Thirdly, this application provides an apparatus comprising a memory and a processor, the memory for storing instructions or code, and the processor for executing the instructions or code to cause the apparatus to perform the warehouse robot allocation method described in any of the implementations of the first aspect.

[0023] Fourthly, this application provides a computer-readable storage medium storing code, wherein when the code is executed, a device running the code implements the warehouse robot allocation method described in any of the implementations of the first aspect.

[0024] This application provides a method for allocating warehouse robots. When executing the method, firstly, real-time status information of the warehouse system is acquired. This real-time status information includes at least the robot's own status, workstation status, and the physiological status of the human picker. The physiological status includes fatigue and stress. Then, the real-time status information is processed based on a pre-trained multi-agent reinforcement learning policy network to generate a robot allocation strategy. Finally, robots are dynamically allocated to corresponding workstations according to the robot allocation strategy, achieving a synergistic optimization of maximizing order throughput and minimizing occupational health risks for human pickers. The occupational health risks include the cumulative duration of high-pressure states and the cumulative amount of fatigue. Thus, through the accurate capture and dynamic decision-making of multi-dimensional real-time status information by the multi-agent reinforcement learning policy network, the robot allocation strategy can adapt to the fatigue-stress coupling changes of the picker. This avoids workstation load imbalance while effectively mitigating the deteriorating physiological state of the picker, achieving a dual improvement in operational efficiency and human health. In this way, the duration of high pressure and fatigue accumulation of pickers can be significantly reduced, while maintaining or even increasing the system's order throughput. It can also enhance the robustness of the warehousing system to dynamic disturbances such as workstation failures and efficiency heterogeneity, providing more practical technical support for human-machine collaboration in warehouse automation. Attached Figure Description

[0025] To more clearly illustrate the technical solutions in this embodiment or the prior art, the drawings used in the description of the embodiment or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0026] Figure 1 A flowchart illustrating a warehouse robot allocation method provided in this application embodiment; Figure 2 A diagram illustrating the basic patterns of fatigue and stress development in humans during a sorting work cycle, provided as an embodiment of this application; Figure 3 This is a schematic diagram of a warehouse robot distribution system provided in an embodiment of this application. Detailed Implementation

[0027] As described in the background section of this application, existing warehouse robot allocation strategies fail to fully consider the dynamic synergistic effect of picker fatigue and stress, resulting in a difficulty in balancing system efficiency and human health.

[0028] To address the aforementioned technical problems, this application provides a warehouse robot allocation method. The method includes: acquiring real-time status information of the warehouse system, the real-time status information including at least the robot's own status, workstation status, and the physiological status of human pickers, the physiological status including fatigue and stress; processing the real-time status information based on a pre-trained multi-agent reinforcement learning policy network to generate a robot allocation strategy; and dynamically allocating robots to corresponding workstations according to the robot allocation strategy to achieve synergistic optimization of maximizing order throughput and minimizing occupational health risks for human pickers, the occupational health risks including the cumulative duration of high-pressure states and the cumulative amount of fatigue.

[0029] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0030] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0031] Figure 1 This is a flowchart illustrating a warehouse robot allocation method provided in an embodiment of this application. (In conjunction with...) Figure 1 As shown, the warehouse robot allocation method provided in this application embodiment may include: S101. Obtain real-time status information of the warehousing system. The real-time status information includes at least the robot's own status, the workstation status, and the physiological status of the human picker. The physiological status includes fatigue and stress.

[0032] The physiological state of the human picker is quantitatively characterized by a kinetic model, which includes: Stress Response Model: Based on the Yerkes-Dodson Law, which presents an inverted U-shaped relationship between stress and performance, such as... Figure 2 As shown, Figure 2This application provides a diagram illustrating the basic patterns of fatigue and stress development in humans during a picking work cycle. Figure 2 (a) illustrates the relationship between specific stress levels and human performance.

[0033] Stress levels are categorized into four states: under-motivated (extremely short shelf queues, low picker engagement and efficiency), optimal stress (moderate queues, high cognitive focus and maximized efficiency), overload (super-optimal queues, cognitive fatigue and decreased efficiency), and excessive workload (increasing stress and accelerated performance decline driven by anxiety). The relationship between the length of the workstation queue and the length of the queue is linear, as shown in the formula: ; in It is the proportional coefficient of the pressure response model (a preset parameter in the model, which needs to be calibrated through experiments or historical data, used to quantify the conversion relationship between workstation queue length and pressure level). Indicates the first The length of the shelf queue at each workstation (i.e., the number of shelves to be processed at that workstation).

[0034] Efficiency Modulation Factor , representing the percentage effect of pressure on efficiency, is characterized using a Gaussian response model, and the formula is: ; , and It is a constant parameter, and . Indicates the optimal pressure level, when When the efficiency modulation factor is 1, it means that the picking efficiency is at its theoretical maximum.

[0035] Fatigue Dynamics Model: Human pickers exhibit nonlinear fatigue accumulation characteristics during warehousing operations, consistent with the dynamic characteristics of neuromuscular attrition models, such as... Figure 2 As shown in (b), Figure 2 (b) shows the fatigue-recovery curves during work and rest. The cumulative process is modeled exponentially, with the formula being: Among them, the benchmark fatigue rate With stress-induced accelerators Together they regulate the rate of fatigue accumulation. Quantify decision interval Internal fatigue increment, Characterizing the reinforcing effect of stress on fatigue accumulation, Indicates the duration of the current work phase. Indicates the first The cumulative fatigue value of human pickers at each work stage is used to measure the degree of fatigue at the end of the current work stage. Indicates the first The cumulative fatigue value at the end of each stage.

[0036] During the rest phase, fatigue decays exponentially but retains residual accumulation (due to incomplete physiological recovery), as shown in the formula: , This represents the residual fatigue level of the human pickers at the end of the rest phase. This represents the cumulative fatigue quantification at the start of the rest phase (i.e., at the end of the work phase). For performance recovery rate parameters, A fixed rest period is preset (a constant value scheme is adopted in this embodiment). Meanwhile, the inhibitory effect of fatigue accumulation on picking efficiency is characterized by a negative exponential model, and the efficiency formula for the working stage is: The formula for recovery efficiency after rest is: ,in Indicates the first Next rest, Indicates the first Rest (duration) Recovery efficiency after ) For the first Segment picking operation (duration) Efficiency after initial conditions. hour (The maximum theoretical processing speed of human pickers).

[0037] The pressure-fatigue-efficiency coupling model integrates the effects of pressure response and fatigue dynamics on picking efficiency, assuming that the effects of pressure and fatigue on efficiency are independent. The model formula is as follows: ; This model achieves a quantitative correlation between physiological state and work efficiency. It is integrated into the human factors state simulation module of the simulator to support subsequent learning methods.

[0038] The robot's own status in the real-time status information includes: robot position coordinates (movement distance is calculated using Manhattan distance), target shelf coordinates, and busy status (binary state). , Indicating that the order task is being executed, The workstation status includes queue length (number of shelves to be processed, service is accepted in a first-come, first-served (FCFS) sequence, assuming no upper limit on buffer capacity), number of robots in transit, and workstation identifier; the fatigue status of the human picker is quantified by fatigue accumulation value. Characterization (when) Reaching the preset threshold When necessary, a rest mechanism is automatically triggered (e.g., a 10-minute short break). Stress status is quantified by stress level values. The representation is a binary state. This indicates excessive stress. This indicates that the person is not under stress (based on thresholded physiological signals, such as pupil diameter to measure stress and heart rate variability to measure fatigue, and following a predetermined classification protocol to achieve state discrimination).

[0039] S102. The real-time state information is processed based on a pre-trained multi-agent reinforcement learning policy network to generate a robot allocation policy.

[0040] The multi-agent reinforcement learning policy network is based on the theory of decentralized partially observable Markov decision process (Dec-POMDP). Specifically, this application formalizes the robot assignment problem (RAP) into a decentralized partially observable Markov decision process (Dec-POMDP). The model tuple... In this model, the state space integrates the physiological states of the robot, workstation, and human pickers; the action space focuses on the robot's allocation behavior to the workstation; and the reward function integrates order throughput and picker stress / fatigue constraints, fully adapting to the decision-making needs of dynamic robot allocation and achieving collaborative optimization of operational efficiency and human health under distributed decision-making. Formal modeling addresses the problem of "local robot observation and global collaborative decision-making" in warehousing systems, specifically including: Define the model as a tuple The specific details of each element are as follows: state space ( S ): Global state An integrated representation of the robot's own state ( ):robot state vector Includes position coordinates Target shelf coordinates and a dual busy state ( ). Indicating that the order task is being executed, This indicates that the robot is in an idle state and ready to perform a new transport task. Workstation status ( ):workstation Feature set , These represent the robot queue length, the number of robots in transit, and the workstation identifier, respectively. Human state ( Picker In time sequence Physiological states are characterized by binary pairs: in, This indicates the state of fatigue (0: normal physiological condition, 1: fatigued state). The stress state is indicated by (0: psychologically normal, 1: under stress). This discretized modeling is based on engineering feasibility: continuous fatigue quantity. With pressure Since direct measurement is difficult in practical systems, state discrimination is achieved by thresholding physiological signals (such as pupil diameter to measure pressure and heart rate variability to measure fatigue) and following a predetermined classification protocol. This binary representation is compatible with low-cost, deployable monitoring hardware and facilitates integration into online decision-making frameworks.

[0041] Action space Action space The set of discrete assignable behaviors available to each robotic agent is defined as follows: .in Assign action sets to the core. For auxiliary behavior set: Corresponding robot to workstation Targeted allocation operation; The robot is temporarily unavailable due to performing other system tasks. This is a regulatory action, only necessary when all pickers are simultaneously in a critical physiological state (e.g., When activated, it can reduce the load instantly by pausing the allocation of tasks, while ensuring the long-term resilience of the system.

[0042] Transition probability P : Describes the evolution of the system state after an action is performed; reward function R The reward function of the multi-agent reinforcement learning strategy network adopts a composite reward mechanism, which is designed specifically for the dual objectives of robot allocation: the global reward item integrates throughput deviation, stress exposure level, and fatigue rest events, while the auxiliary reward item imposes penalties on allocation actions to high-pressure / congested workstations and rewards strategic waiting actions. Through this hierarchical reward mechanism, the robot allocation strategy can balance the core objectives of maximizing order throughput and minimizing occupational health risks of pickers.

[0043] The reward function includes a global reward and a secondary reward, and the formula is as follows: ; in , These are configurable linear weighting coefficients. Indicates the first The first workstation (or the first) The quantitative value of human sorting worker stress level at each time point is derived from the formula. The calculation reflects the linear relationship between workstation queue length and picker stress. Indicates that all robots are in the first... A combined action executed step by step, that is, a set of actions of each robot in a multi-agent system (such as a combination of actions such as "robot R1 → workstation W3" or "robot R2 enters waiting state").

[0044] Global Rewards ( The formula comprehensively characterizes long-term system performance:

[0045] in, Decision interval The number of orders actually completed by the internal system; The number of human pickers; Theoretically maximum processing capacity, , Characterization workstation The theoretical peak picking efficiency; For time period Internal pickers The total duration of being under high pressure; The fatigue rest event indicator (equal to 1 if and only if the picker) exist Internally triggered forced rest mechanism); Multi-objective optimization weight configuration constraints ( Adjusting throughput weight, (Separately adjust the health risk weights for stress and fatigue).

[0046] Auxiliary reward items ( Provides real-time action feedback at the robot level to mitigate the latency issue of global rewards:

[0047] In the formula A collection of robots that perform assigned actions. Differentiated definitions based on action type: Assigning actions ( Penalties are applied proportionally based on the congestion and stress levels of the target workstation. ,in ( For workstations (queue length); It is a binary pressure indicator.

[0048] Waiting action ( ): Encourage strategic waiting behavior in critical situations.

[0049]

[0050] in, , , The length of the queue. This represents the total load of the pressure workstation. , This represents the total load of the pressure workstation. ; This is the pressure load weighting coefficient. This is the relaxation load weighting coefficient.

[0051] Key threshold definition: Full pressure state: ( (Number of available pickers); System idle status: .

[0052] This tiered reward mechanism achieves three key objectives through collaboration: firstly, stress-sensitive penalty items. The penalty intensity for assigning robots to overloaded and high-voltage workstations is dynamically increased, forming an adaptive inhibition mechanism that responds to real-time human physiological states; secondly, Components strategically prioritize mitigating the load on critical workstations, prioritizing interventions while simultaneously activating stress indicators. ) and standardized load overcapacity ( The key site; thirdly, Components through Normalization ensures baseline operational efficiency at non-stressed workstations, preventing underutilized workstations from being overlooked while maintaining balanced system resource utilization. These mechanisms work together to provide fine-grained action feedback based on physiological states, enabling strategies to navigate the fundamental trade-off between throughput optimization and occupational health.

[0053] Observation space Ω: Intelligent agent At the moment of decision Through observation function Obtain local observations The architecture employs a limited observation design to avoid information overload: This indicates that the intelligent agent can only perceive its own state vector, the load data of the entire workstation and the human condition of the picker, and cannot directly obtain information from other robots. The observation function O defines the mapping relationship from the robot's own state to the local observation Ω. For example, the robot obtains the queue length of the nearby workstation through its onboard IoT sensors and obtains the stress level of its dedicated picker through its onboard physiological monitoring equipment (such as a heart rate monitor). It should be noted that each picker is equipped with a physiological monitoring device. After the device signal is analyzed at the picking station, the corresponding real-time physiological state information is obtained. Then, a wireless communication device can be used to send out the picker's state. This observation only includes the state of the robot itself and the workstation and picker it directly interacts with, without introducing information from other robots, in order to compress the state dimension.

[0054] Number of agents n The total number of robots involved in the allocation decision can be dynamically configured according to the warehouse size (e.g., 100 robots for a medium-sized warehouse and 200 robots for a large warehouse). Discount factor This is used to weigh the importance of current and future rewards. In this embodiment, it is set to 0.99, which is consistent with the hyperparameters used in model training.

[0055] In practical applications, each robot acts as a distributed autonomous intelligent agent, collaboratively optimizing the global allocation target within the Dec-POMDP decision framework. At the same time, the fatigue-stress coupling dynamics of human pickers are embedded into the model as part of the observable state, realizing distributed intelligent allocation in a partially observable environment, taking into account both operational efficiency and the occupational health of pickers.

[0056] The Dec-POMDP approach refactors the Robot Assignment Problem (RAP) into a sequential decision process, transforming the original multi-objective optimization problem into a strategy for all robots to learn stress-fatigue perception. The problem is as follows. The original multi-objective optimization problem is: The weighting coefficient These are configurable parameters, where Adjusting the system efficiency weight, and By separately adjusting the health risk weights of stress duration and fatigue accumulation, and through coefficient allocation, synergistic optimization of operational efficiency and human well-being can be achieved. : T Total number of orders completed within the time period High pressure duration, : Cumulative fatigue level.

[0057] This robot learning stress-fatigue perception strategy maximizes discounted cumulative rewards through a global action-value function, effectively balancing long-term throughput and occupational health risks. In the formula For joint action - observing historical trajectories, For joint action vectors, This is the discount factor. The formula represents the historical trajectory. Execute joint actions Continue to follow the strategy The expected long-term rewards that can be obtained. A larger value indicates a more superior collective strategy, and the reward function maintains mathematical consistency with the function of the multi-objective optimization problem.

[0058] Optimal allocation strategy The following conditions must be met:

[0059] Through agent networks The function approximation learns gradually in the process of dynamically adapting to human physiological states. .

[0060] A multi-agent reinforcement learning method integrates an attention mechanism and employs a centralized training-distributed execution architecture. The training process includes: initializing the model parameters and hyperparameters in the replay buffer, whereby the model parameters include the parameters of the hybrid network robot network and the hypernetwork; resetting the warehouse simulation environment, obtaining the initial state, and entering iterative training; each robot selects an action based on its current exploration strategy, executes the joint action, observes the reward and the next state, and stores the state transition data in the replay buffer; when a preset training period is reached, data is sampled from the replay buffer, the action value Q of the prediction network and the target network is calculated, and the loss function is minimized through gradient descent to update the model parameters; the parameters of the target network and the prediction network are periodically synchronized until the training reaches a preset termination condition.

[0061] The multi-agent reinforcement learning policy network is trained using a centralized training-distributed execution architecture. The training process includes: initialization phase: initializing the replay buffer. D (capacity is) N Model parameters θ (Including parameters of hybrid networks, robotic networks, and hypernetworks) and hyperparameters (e.g., discount factors) Minimum batch size Adam optimization algorithm learning rate Exploration rate initial value Final value Explore steps (Step), set the initial step size to 0, and let the target network parameters... .

[0062] Iterative training begins: When the step size is less than the maximum step size, the warehouse simulation environment (based on a discrete event framework implemented using the Python SimPy library, modeling dynamic interactions within RMFS, including random shelf retrieval task generation, demand allocation, workload distribution across workstations, and human picker fatigue accumulation) is reset to obtain the initial state. ,set up .

[0063] In-round interaction phase: when In a non-terminating state and when k ≤ the round limit, each robot Based on the current exploration strategy, -greedy exploration strategy according to Linear decay, This represents the current training iteration number. This represents the preset total number of exploration steps. This decay strategy enables an adaptive and smooth transition from exploration-oriented to exploitation-oriented training.

[0064] Select action: If the random number is less than Select a random action; otherwise, press [button]. Choose the optimal action, among which The action chosen by agent i at step k. For action-observation historical trajectory, For the current joint action, Let i be the action-value function of agent i. Execute the joint action. Observation reward R k Next state Transition data Store to replay buffer D, update step size, step size = step size + 1.

[0065] Model parameter update phase: when the step size is the preset training period When the value is an integer multiple of D, sample M state transition data points from the playback buffer D; for each transition data point, calculate the global Q-value of the prediction network. , The current global state-action history trajectory (including the action-observation history of all agents), where... The joint action at the current moment (the set of actions of all agents). For the local observation state of agent i, The model parameters of the prediction network, including those of the agent network and the hybrid network, are updated via gradient descent. This is compared with the global Q-value of the target network. , For the global state-action history trajectory at the next moment, For the joint action in the next moment (all agents in state) (selected action set) For agent i to perform actions The local observation state after the next state (local information of the next state). The model parameters of the target network are copies of the parameters of the predicted network, used for stable training and periodically updated synchronously.

[0066] Minimize the loss function using gradient descent:

[0067] For the first i The instant reward corresponding to each sampled data point Discount factor, used to weigh the importance of current rewards against future rewards.

[0068] Update model parameters When the step size is an integer multiple of the preset update interval, synchronize the parameters of the target network and the prediction network, and let... .

[0069] Training terminates, and the above process is repeated until the training reaches the preset termination conditions, such as core indicators such as order completion volume, fatigue accumulation, and stress duration converging stably within 200 training rounds, with 50 steps per round. Performance evaluation is quantified and monitored every 5000 steps using the average return value within a 3-hour operating time window.

[0070] The network structure of the multi-agent reinforcement learning strategy network includes an agent network and a hybrid network. This network structure is specifically designed to address the core pain points of dynamic robot allocation in warehousing scenarios (distributed decision-making, human condition perception, global throughput, and collaborative optimization of occupational health), differing from general multi-agent reinforcement learning network architectures. The agent network inputs include robot position information, workstation conditions, and the fatigue / stress state of the picker, outputting a Q-value estimate of the robot's allocation actions. The hybrid network uses a masking mechanism to shield non-standby robots and a multi-head attention module to model the robot-workstation-human condition association. Its module configuration and input / output design are all centered around the robot allocation strategy generation requirements. Validated across multiple warehousing scenarios, the multi-agent reinforcement learning network structure designed specifically for the dynamic allocation problem of warehousing robots in this application achieves a 23.8% increase in order throughput and reduces the duration of picker stress by more than 26% in medium-sized warehousing scenarios compared to traditional strategies such as random allocation, distance-based allocation, and load balancing, with fatigue accumulation approaching the theoretical optimum. In large-scale scenarios, the order throughput of the robot allocation strategy supported by this network structure is twice that of the load balancing strategy, and the stress efficiency ratio (SER) and fatigue efficiency ratio (FER) are significantly better than those of random allocation and distance-based allocation strategies, verifying the adaptability and superiority of this network structure for the robot allocation problem. The agent network adopts a deep recurrent Q-network (DRQN) architecture, integrating gated recurrent units (GRUs) to capture temporal dependencies in sequential decision-making (the gating mechanism of the GRU can selectively retain and update historical states, filter transient noise, and learn robust temporal representations). The input consists of local observation information (including robot pose coordinates, running state, workstation conditions, and human picker state, i.e., fatigue level). With pressure value The input data is mapped into a high-dimensional continuous representation through the embedding layer. Then, the GRU module (containing 64 hidden nodes) extracts temporal features. The generated features are further transformed by subsequent fully connected layers, and finally, the Q-value estimates of each available action are output.

[0071] The hybrid network integrates a masking mechanism and a multi-head attention module to aggregate the individual Q-values ​​of each agent into a global Q-value, achieving synergy between distributed decision-making and global optimization. The process includes three stages: In the state-aware feature encoding stage, dual-path feature encoding is implemented for both the robot and the workstation. For each robot, its two-dimensional spatial coordinates and assigned actions are concatenated, and a high-dimensional demand vector is generated through a multilayer perceptron (MLP) encoder with layer normalization layers. Fr (The output dimension matches the embedding requirements of the subsequent multi-head attention layer), a binary qualification masking mechanism is used to block non-standby robots. The masking formula is: ; This mechanism ensures that only robots that need to make immediate decisions participate in attention calculations, effectively avoiding interference from inactive agents.

[0072] The workstation encoder integrates multi-dimensional status characteristics, workstation attributes, human factors, and picker fatigue. With pressure value The encoding vector can represent the real-time service capability. When the queue is congested or the pressure increases, a certain dimension of the encoding vector decays, reducing its selection priority in attention decision-making.

[0073] Multi-head attention mechanism stage: This stage dynamically models the relationship between robot requirements and workstation states using a parallel multi-head attention architecture. Encoded robot features (representation space-behavior joint requirement information) and workstation features (encoded service capabilities and human factor state parameters) are mapped to multiple feature subspaces, generating three matrices: query (Q), key (K), and value (V). Each attention head is dedicated to extracting specific interaction patterns, such as head... Focus on spatial proximity measurement, head (Enhanced capacity matching assessment). Attention head. h The weights are calculated using the following compatibility metric formula: ,in The attention head dimension hyperparameter is used for gradient stabilization; the mask matrix is ​​also used. Filtering out non-standby robots based on binary qualification identifiers No. h The query matrix for each attention head, generated from the robot's feature encoding, is used to represent the robot's demand information. No. h The transpose of the key matrix of each attention head, generated from the feature encoding of the workstation, is used to represent the workstation's state information. A binary eligibility mask matrix is ​​used to mask out non-standby robots (if a robot is unavailable, the mask value at the corresponding position is 0; otherwise, it is 1), ensuring that attention calculations are only performed on assignable robots. An optional adaptive head weight adjustment mechanism can be introduced: the global state is compressed into a feature vector using a state encoder. The specific weights are output from the multilayer perceptron (MLP) weight generation network. This enables adaptive allocation of multi-head importance in response to environmental dynamics. Finally, attention features are generated through weighted fusion. This feature fully encapsulates the adaptability of robot allocation strategies under global state awareness.

[0074] In the hybrid output stage, individual Q-values ​​are weighted and aggregated based on attention features, and a state-aware bias term is introduced to generate a global Q-value, as shown in the formula. ;in Characterization Robot Attention feature vectors, This is a bias derived from global state coding. For the first j The individual Q-value of a robot is an estimate of the value of its actions based on its own local observations. This mathematical model achieves collaborative optimization of distributed agent decision-making and global resource allocation. Specifically, when the attention mechanism exhibits load distribution bias (such as robots excessively clustering at low-load workstations), the bias term... By integrating global load metric parameters for dynamic correction, the risk of congestion caused by local optima is effectively avoided, ensuring overall system load balance.

[0075] S103. According to the robot allocation strategy, the robot is dynamically allocated to the corresponding workstation to achieve synergistic optimization of maximizing order throughput and minimizing occupational health risks of human pickers. The occupational health risks include the cumulative duration of high-pressure conditions and the cumulative amount of fatigue.

[0076] The robot allocation strategy is generated based on real-time observations made by the agent network after training. The optimization objective of the robot allocation strategy is defined by a composite objective function: ,in For the operating cycle T The order throughput within the period is calculated by using a discrete event simulation framework to count the total number of valid orders completed by each workstation within the period. The validity of the orders must meet the dual constraints of picking accuracy (preset threshold ≥ 99.5%) and delivery timeliness (single order processing time ≤ preset period threshold). In the formula Characterizing the decision interval Number of completed orders within ( ; ), This represents the total number of decision intervals within the operating cycle. H This represents the total number of human pickers. This metric directly measures the allocation strategy. The overall output efficiency of the system.

[0077] The total pressure duration, i.e., the cycle. T All human pickers are under high pressure (pressure level quantification) , The cumulative duration of the pressure threshold set based on industry health standards is sampled and statistically analyzed at the second level through the real-time status monitoring module. In the formula Characterizing pickers h In time interval The duration of internal high pressure.

[0078] The total accumulated fatigue, i.e., the period. T Cumulative fatigue value of all human pickers The time integral is defined as the actual working period for each picker, and the fatigue attenuation during rest periods is calculated using a fatigue dynamic model. Simultaneous deduction, in the formula For performance recovery rate parameters, To preset a fixed rest period; In the formula For pickers h During the period The cumulative fatigue value within the range. It should be noted that the system uses variables... Real-time monitoring of fatigue levels of each picker; when Reaching the preset threshold When this happens, a rest mechanism (such as a 10-minute short break) will be automatically triggered to ensure the recovery of personnel's functions. α 1. α 2. α 3 represents the configurable weight coefficient.

[0079] The real-time status information of the warehousing system is obtained through a discrete event simulation framework. This framework is built on the Python SimPy library, and its core function is to model the dynamic interaction process of multiple agents within the warehousing system. Specifically, this includes: Dynamic interaction process modeling: Robot movement: Based on the warehousing map grid model, combined with path planning algorithms, the optimal path from the robot's current position to the target shelf / workstation is calculated, and the robot's position coordinates (positioning accuracy ≤ 0.5m) and movement time (calculated based on the robot's rated speed (e.g., 1.2m / s) and path length) are updated in real time; Shelf transportation: When the robot reaches the target shelf position, a shelf binding event is triggered, and the system records the shelf coordinates and binding time; During transportation, the robot's load status (empty / full load) is monitored in real time. If path congestion occurs (e.g., multiple robots intersecting), the driving order is adjusted through a conflict avoidance algorithm to ensure transportation continuity; Order picking: After the workstation receives the shelf delivered by the robot, the human picker performs the picking operation according to the order list. The system synchronously collects the picking time (the time interval from the shelf arrival to the picking completion) and the picking quantity, combined with a pressure-fatigue-efficiency coupling model. Real-time updates of the picker's physiological status ( and ); Physiological state updates: A combination of timed sampling (sampling interval = 1 min) and event-triggered sampling (such as completion of picking tasks, changes in workstation queue length exceeding a threshold) is used to calculate cumulative fatigue and stress levels based on a kinetic model. ( When the fatigue threshold is reached (e.g., 0.8), a rest event is automatically triggered, and the start and end times of the rest are recorded. During this period, the picking task assignment for the picker is suspended.

[0080] Allocation decision trigger condition: When the robot completes the shelf transportation task and returns to the storage area (the preset robot standby area), and is in an idle state (busy state). b When =1), the system triggers an allocation decision event. At this time, the discrete event simulation framework will consider the robot's own state (position coordinates, idle state indicator), workstation state (queue length, number of robots in transit, workstation identifier), and the physiological state of the human picker. , The status information (whether the robot is in a resting state) is packaged into real-time status information and transmitted to the trained robot allocation strategy model to start the robot allocation strategy generation process.

[0081] The above are some specific implementations of a warehouse robot allocation method provided in the embodiments of this application. Based on this, this application also provides a corresponding system. The system provided in the embodiments of this application will be described below from the perspective of functional modularization.

[0082] Figure 3 This is a schematic diagram of a warehouse robot distribution system provided in an embodiment of this application. (Combined with...) Figure 3 As shown in the embodiment of this application, the warehouse robot distribution system 300 includes: The acquisition unit 310 is used to acquire real-time status information of the warehousing system. The real-time status information includes at least the robot's own status, the workstation status, and the physiological status of the human picker. The physiological status includes fatigue and stress. Processing unit 320 is used to process the real-time state information based on a pre-trained multi-agent reinforcement learning policy network to generate a robot allocation policy; The allocation unit 330 is used to dynamically allocate robots to corresponding workstations according to the robot allocation strategy, so as to achieve synergistic optimization of maximizing order throughput and minimizing occupational health risks of human pickers. The occupational health risks include the cumulative duration of high-pressure conditions and the cumulative amount of fatigue.

[0083] In one implementation of this application, the multi-agent reinforcement learning policy network is trained using a centralized training-distributed execution architecture. The training process of the processing unit training the multi-agent reinforcement learning policy network includes: Initialize the replay buffer model parameters and hyperparameters, including the parameters of the hybrid network robot network and the hypernetwork; Reset the warehouse simulation environment, obtain the initial state and enter iterative training; each robot selects actions based on the current exploration strategy, executes joint actions and observes the reward and the next state, and stores the state transition data in the replay buffer; When the preset training period is reached, data is sampled from the replay buffer, the action value Q of the prediction network and the target network is calculated, and the loss function is minimized through gradient descent to update the model parameters; the parameters of the target network and the prediction network are periodically synchronized until the training reaches the preset termination condition.

[0084] In one implementation of this application, the physiological state of the human picker is quantitatively characterized by a dynamic model, which includes: a stress response model based on the Yerkes-Dodson law, where stress level is linearly related to workstation queue length and the efficiency modulation factor adopts a Gaussian response model; a fatigue dynamic model, where fatigue accumulates exponentially during the working phase and decays exponentially during the rest phase, and fatigue accumulation is amplified by stress state; and a stress-fatigue-efficiency coupling model, which integrates the influence of stress response and fatigue dynamics on picking efficiency to achieve a quantitative correlation between physiological state and work efficiency.

[0085] In one implementation of this application, the network structure of the multi-agent reinforcement learning policy network includes an agent network and a hybrid network. The agent network adopts a deep recurrent Q-network architecture, integrating gated recurrent units to capture temporal dependencies, inputting local observation information and outputting action value Q-value estimates for each action. The hybrid network integrates a masking mechanism and a multi-head attention module to aggregate the individual Q-values ​​of each agent into a global Q-value, achieving synergy between distributed decision-making and global optimization.

[0086] In one implementation of this application, the processing procedure of the hybrid network includes: In the state-aware feature encoding stage, the features of the robot and the workstation are encoded separately, and non-standby robots are shielded through a masking mechanism. In the multi-head attention mechanism stage, the encoded features are mapped to multiple subspaces to dynamically model the relationship between robot requirements and workstation states. In the hybrid output stage, individual Q-values ​​are weighted and aggregated based on attention features, and state perception values ​​are introduced to represent stress states through stress level quantification values.

[0087] In one implementation of this application, the reward function of the multi-agent reinforcement learning policy network adopts a composite reward mechanism, including a global reward item and an auxiliary reward item. The global reward item integrates throughput deviation, pressure exposure degree, and fatigue rest events. The auxiliary reward item is designed differently according to the action type, and the assigned action is penalized according to the workstation congestion and pressure state, while the waiting action is given targeted reward according to the system state.

[0088] In one implementation of this application, the optimization objective of the robot allocation strategy is defined by a composite objective function: ,in This refers to the order throughput within the operating cycle T. The duration of total pressure. This represents the total accumulated fatigue. , , The configurable weighting coefficients allow for the adjustment of optimization weights for efficiency and health risks.

[0089] In one implementation of this application, the acquisition unit is specifically used to acquire real-time status information of the warehousing system through a discrete event simulation framework. The simulation framework models a dynamic interaction process and triggers an allocation decision when the robot returns to the storage area to wait for a new task. The dynamic interaction process includes robot movement, shelf transportation, order picking, and physiological status updates.

[0090] This application also provides corresponding devices and computer storage media for implementing the solutions provided in this application.

[0091] The device includes a memory and a processor. The memory stores instructions or code, and the processor executes the instructions or code to cause the device to perform the method described in any embodiment of this application.

[0092] The computer storage medium stores code, and when the code is run, the device running the code implements the method described in any embodiment of this application.

[0093] As can be seen from the above description of the embodiments, those skilled in the art can clearly understand that all or part of the steps in the methods of the above embodiments can be implemented by means of software plus a general-purpose hardware platform. Based on this understanding, the technical solution of this application can be embodied in the form of a software product. This computer software product can be stored in a storage medium, such as a read-only memory (ROM) / RAM magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, a server, or a network communication device such as a router) to execute the methods described in various embodiments or some parts of the embodiments of this application.

[0094] It is understood that in the specific embodiments of this application, the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data storage for analysis, data display, etc.) involved require user permission or consent when the above embodiments of this application are applied to specific products or technologies, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0095] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process method article or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process method article or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process method article or apparatus that includes said element.

[0096] It should also be noted that the various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, for the device and system embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the description of the method embodiments. The device and system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components indicated as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of the solution in this embodiment according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0097] The above description is merely one specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A warehouse robot allocation method, characterized in that, The method includes: The system acquires real-time status information of the warehousing system, which includes at least the robot's own status, the workstation status, and the physiological status of the human picker, including fatigue and stress. The real-time state information is processed based on a pre-trained multi-agent reinforcement learning policy network to generate a robot allocation policy; According to the robot allocation strategy, robots are dynamically allocated to corresponding workstations to achieve synergistic optimization of maximizing order throughput and minimizing occupational health risks for human pickers. The occupational health risks include the cumulative duration of high-pressure conditions and the cumulative amount of fatigue.

2. The method according to claim 1, characterized in that, The multi-agent reinforcement learning policy network is trained using a centralized training-distributed execution architecture. The training process includes: Initialize the replay buffer, model parameters, and hyperparameters. The model parameters are single parameters of the neural networks of hybrid networks, robot networks, and hypernetworks, and are updated during the training process. The hyperparameters are fixed control parameters preset before training and are not updated during the training process. The warehouse simulation environment is reset to its initial state, and the global initial state is obtained. After entering the iterative training loop, each robot selects an action based on the current strategy in each step. The current strategy includes an exploration mechanism. All robots synchronously execute the selected actions to form a joint action. After receiving the joint action, the environment returns the global reward and the next state. The system stores the current state, joint action, reward, next state, and termination flag as experience tuples in a centralized replay buffer. When the preset training period is reached, data is sampled from the replay buffer, the action value Q of the prediction network and the target network is calculated, and the loss function is minimized through gradient descent to update the model parameters; the parameters of the target network and the prediction network are periodically synchronized until the training reaches the preset termination condition.

3. The method according to claim 1, characterized in that, The physiological state of the human picker is quantitatively characterized by a kinetic model, which includes: The pressure response model is based on the Yerkes-Dodson law, and the pressure level is linearly related to the workstation queue length. The efficiency modulation factor adopts a Gaussian response model. The fatigue dynamic model shows that fatigue accumulates exponentially during the working phase and decays exponentially during the rest phase, with fatigue accumulation being amplified by stress. The stress-fatigue-efficiency coupling model integrates the impact of stress response and fatigue dynamics on picking efficiency, achieving a quantitative correlation between physiological state and work efficiency.

4. The method according to claim 1, characterized in that, The network structure of the multi-agent reinforcement learning policy network includes an agent network and a hybrid network. The agent network adopts a deep recurrent Q-network architecture, integrating gated recurrent units to capture temporal dependencies, inputting local observation information and outputting action value Q-value estimates for each action. The hybrid network integrates a masking mechanism and a multi-head attention module to aggregate the individual Q-values ​​of each agent into a global Q-value, realizing the synergy between distributed decision-making and global optimization.

5. The method according to claim 4, characterized in that, The processing steps of the hybrid network include: In the state-aware feature encoding stage, the features of the robot and the workstation are encoded separately, and non-standby robots are masked through a masking mechanism. In the multi-head attention mechanism stage, the encoded features are mapped to multiple subspaces to dynamically model the relationship between robot requirements and workstation states. In the hybrid output stage, individual Q-values ​​are weighted and aggregated based on attention features, and a state-aware bias term is introduced to generate a global Q-value.

6. The method according to claim 1, characterized in that, The robot's own status in the real-time status information includes: robot position coordinates, target shelf coordinates, and busy status; The workstation status includes queue length, number of robots in transit, and workstation identifier; The fatigue state of the human pickers is characterized by a cumulative fatigue quantification value, and the stress state is characterized by a stress level quantification value.

7. The method according to claim 1, characterized in that, The reward function of the multi-agent reinforcement learning policy network adopts a composite reward mechanism, including a global reward item and an auxiliary reward item. The global reward item integrates throughput deviation, stress exposure degree and fatigue rest events. The auxiliary reward item is designed differently according to action type, and the assigned action is penalized according to the workstation congestion degree and stress state, while the waiting action is given targeted reward according to the system state.

8. The method according to claim 1, characterized in that, The optimization objective of the robot allocation strategy is defined by a composite objective function: ,in This refers to the order throughput within the operating cycle T. The duration of total pressure. This represents the total accumulated fatigue. , , The configurable weighting coefficients allow for the adjustment of optimization weights for efficiency and health risks.

9. The method according to claim 1, characterized in that, The real-time status information of the warehousing system is obtained through a discrete event simulation framework. The simulation framework models the dynamic interaction process and triggers the allocation decision when the robot returns to the storage area to wait for a new task. The dynamic interaction process includes robot movement, shelf transportation, order picking, and physiological status updates.

10. A warehouse robot distribution system, characterized in that, The system includes: The acquisition unit is used to acquire real-time status information of the warehousing system. The real-time status information includes at least the robot's own status, the workstation status, and the physiological status of the human picker. The physiological status includes fatigue and stress. The processing unit is used to process the real-time state information based on a pre-trained multi-agent reinforcement learning policy network to generate a robot allocation policy. The allocation unit is used to dynamically allocate robots to corresponding workstations according to the robot allocation strategy, so as to achieve synergistic optimization of maximizing order throughput and minimizing occupational health risks of human pickers. The occupational health risks include the cumulative duration of high-pressure conditions and the cumulative amount of fatigue.

Images