Intelligent workshop logistics transportation resource self-organization scheduling method considering order freezing and unfreezing

Abstract

The invention discloses an intelligent workshop logistics transportation resource self-organization scheduling method considering order freezing and unfreezing, and relates to the technical field of logistics transportation, and the method comprises the steps: building a Markov decision process model; simulating a time-space evolution process of a road network load by using a preset depth time-space diagram neural differential equation, and outputting a congestion index of a road network node in a future time window; constructing a collaborative scheduling framework; outputting a scheduling action set in combination with an action selection strategy; establishing a multi-target mixed reward function; training the Markov decision process model by using a multi-agent near-end strategy optimization algorithm and a course learning strategy; and the order table and the automatic guided vehicle state table are read in real time, a scheduling instruction is generated, and a road network state is monitored in real time to trigger a deadlock fusing mechanism. According to the method, the continuous evolution trend of the congestion index of each network node in the future time period can be predicted, a potential AGV deadlock risk area is identified in advance, and the deadlock occurrence rate of a logistics system is reduced from the source.

Description

Technical Field

[0001] This invention relates to the field of logistics and transportation technology, and more specifically, to a self-organizing scheduling method for intelligent workshop logistics and transportation resources that takes into account order freezing and unfreezing. Background Technology

[0002] With the deepening of industrial development and the iterative upgrading of intelligent manufacturing technologies, flexible job shops, with their flexible production processes and efficient resource allocation, have become the core production form in the modern discrete manufacturing field. In the operation system of an intelligent flexible job shop, the production activities executed by processing equipment and the logistics transfer activities executed by automated guided vehicles (AGVs) are highly coupled, forming a production-logistics collaborative (PLC) system that integrates production cycle control and precise material delivery. The level of collaborative scheduling of this system directly determines the overall production efficiency and resource utilization rate of the workshop.

[0003] However, in actual production conditions, dynamic disturbances such as order changes, equipment failures, and material shortages occur frequently. Among these, order freezing and unfreezing are typical types of dynamic disturbances affecting the stability of production-logistics collaborative systems. Existing technologies still have significant deficiencies in their response mechanisms to these types of dynamic disturbances, making it difficult for the system to adapt to actual production needs. The specific technical bottlenecks are as follows:

[0004] 1. Lack of mechanisms to handle traffic surges caused by order freezing / thawing: Existing workshop scheduling methods are mostly based on the ideal assumption that orders arrive with a Poisson or uniform distribution. Mainstream scheduling algorithms such as genetic algorithms and standard reinforcement learning do not design specific strategies to address the freezing / thawing characteristics of orders. In actual production, orders are often temporarily frozen due to factors such as material supply delays, substandard process quality, or adjustments to process plans. When the freeze is lifted, a large number of frozen orders are released in a concentrated manner, creating a significant order traffic surge. Because existing algorithms cannot identify this concentrated release pattern as a special disturbance, unfrozen orders are instantly sent to the production-logistics collaboration system, causing a sharp increase in logistics transfer demand in a short period, exceeding the capacity threshold of the workshop road network. This leads to problems such as AGV congestion and path deadlock, ultimately significantly extending the overall workshop completion time.

[0005] 2. Insufficient Capability of Predicting the Spatiotemporal Evolution of Road Network Congestion: Existing workshop road network traffic prediction models, such as those based on recurrent neural networks (RNNs) and graph convolutional neural networks (GCNs), generally use discrete time steps for data sampling and model training, making it difficult to accurately capture the continuous spatiotemporal evolution of road network traffic pressure. The traffic surge caused by the concentrated unfreezing of orders has a significant spatial diffusion effect. Congestion will gradually spread from the processing area corresponding to the unfreezing of orders to the surrounding road network, forming a nationwide congestion risk. However, existing discrete time step prediction models cannot effectively simulate this type of nonlinear and continuous congestion diffusion process, causing the logistics agent (LA) to only be able to respond passively after congestion occurs, unable to perceive the congestion risk in remote areas in advance and to carry out forward-looking route planning and resource scheduling.

[0006] 3. Lack of a closed-loop feedback mechanism for production scheduling and logistics scheduling: Traditional workshop control architectures employ a hierarchical decision-making model, decoupling production scheduling (order dispatch) from logistics scheduling (AGV dispatch). When the production side executes order unfreezing and distribution operations, it only uses the idle status of processing equipment as the basis for decision-making, completely ignoring the real-time load of the road network. This fragmented decision-making model is the core root cause of frequent bottlenecks in workshop logistics. Although some existing research has explored production-logistics collaborative scheduling using multi-agent deep reinforcement learning (MADRL) methods, in scenarios where order unfreezing causes traffic surges, related research has not yet constructed a closed-loop feedback mechanism of "road network load status → production order release rhythm." This makes it impossible to achieve the reverse transmission of road network pressure signals to the production decision-making end, thus making it difficult to fundamentally avoid the problem of logistics system overload caused by concentrated order distribution.

[0007] No effective solutions have yet been proposed to address the problems in the relevant technologies. Summary of the Invention

[0008] To address the problems in related technologies, this invention proposes a self-organizing scheduling method for intelligent workshop logistics transportation resources that considers order freezing and unfreezing, in order to overcome the aforementioned technical problems existing in the current related technologies.

[0009] Therefore, the specific technical solution adopted by the present invention is as follows:

[0010] According to one aspect of the present invention, a self-organizing scheduling method for intelligent workshop logistics transportation resources considering order freezing and unfreezing is provided, the method comprising:

[0011] S1. Map the physical environment of the smart workshop into a workshop road network map, use dynamic equations to quantify the logistics increment caused by order freezing and unfreezing, and establish a Markov decision process model.

[0012] S2. Based on the established Markov decision process model, the historical traffic characteristics and unfreezing increment characteristics obtained in advance are used as inputs. The pre-set deep spatiotemporal graph neural differential equation is used to simulate the spatiotemporal evolution of road network load and output the congestion index of road network nodes in the future time window.

[0013] S3. Based on the action space and congestion index of the Markov decision process model, construct a collaborative scheduling framework, which includes a target agent, a production agent, and a logistics agent.

[0014] S4. Generate a mask vector corresponding to the order freeze status based on the collaborative scheduling framework, and use the output congestion index to set a dynamic pruning threshold. Combined with the action selection strategy, output a set of scheduling actions.

[0015] S5. Based on the output-based scheduling action set, establish a multi-objective hybrid reward function;

[0016] S6. Based on the state transition equation of the Markov decision process model and the reward value of the multi-objective mixed reward function, the Markov decision process model is trained using the multi-agent proximal policy optimization algorithm and the course learning strategy.

[0017] S7. Based on the trained Markov decision process model, read the order table and the automated guided vehicle status table in real time, generate scheduling instructions, and monitor the road network status in real time to trigger the deadlock circuit breaker mechanism to achieve self-organized scheduling of logistics transportation resources.

[0018] Optionally, the physical environment of the smart workshop is mapped as a workshop road network map, and dynamic equations are used to quantify the logistics increment caused by order freezing and thawing. A Markov decision process model is then established, including:

[0019] S11. Obtain the physical environment information of the smart workshop, and construct the workshop road network map using the road network in the physical environment information as nodes, logistics paths as edges, and physical distance or passage cost as edge weights.

[0020] S12. Based on the constructed workshop road network map, define the order set and determine the state of each order at any time. Establish a dynamic equation to quantify the logistics increment caused by order freezing and unfreezing.

[0021] S13. Based on the constructed workshop road network map and the quantified logistics increment, define the state space, action space and state transition equations to establish a Markov decision process model.

[0022] Optionally, based on the established Markov decision process model, the pre-acquired historical traffic characteristics and unfreezing increment characteristics are used as inputs. A pre-defined deep spatiotemporal graph neural differential equation is used to simulate the spatiotemporal evolution of the road network load, outputting the congestion index of road network nodes within the future time window, including:

[0023] S21. Based on the dynamic equations in the Markov decision process model, calculate and extract the unfreezing increment features of orders;

[0024] S22. Obtain the historical traffic characteristics of the road network, and concatenate the historical traffic characteristics with the unfreezing incremental characteristics at the channel level to generate the input tensor;

[0025] S23. Based on the generated input tensor, establish a dynamic adjacency matrix, which includes an adaptive adjacency matrix, a pattern similarity matrix, and a spatial similarity matrix.

[0026] S24. Substitute the established dynamic adjacency matrix into the preset deep spatiotemporal graph neural differential equation to construct a road network congestion prediction model to simulate the spatiotemporal evolution of road network load as order unfreezing increments, and output the congestion index of road network nodes within the future time window.

[0027] Optionally, establishing a dynamic adjacency matrix includes:

[0028] Based on any node that has thawed and its corresponding downstream node, extract the corresponding historical traffic features;

[0029] Based on the extracted historical flow features and thawing increment features, the historical flow features and thawing increment features are mapped into an enhanced hidden state vector using linear transformation and nonlinear activation function.

[0030] By combining the enhanced hidden state vector obtained from the mapping with the preset unfreezing mode distribution vector, the influence intensity coefficient of each node on the downstream node is calculated using the attention mechanism, and the influence intensity coefficient is normalized by the Softmax function to generate the elements of the dynamic adjacency matrix.

[0031] Using the generated dynamic adjacency matrix as the core input, a graph convolutional network is used to propagate the spatial representation of congestion features to achieve congestion feature transmission.

[0032] Optionally, the production intelligent agent includes:

[0033] The production agent includes a state space and an action space. The state space includes the maximum value of the queue length of orders that are currently in the unfrozen state but have not been scheduled and the congestion index. When selecting orders to be processed, the action space must be constrained by a dynamic release threshold.

[0034] When the maximum value of the congestion index increases due to the unfreezing of batch orders, the dynamic release threshold is simultaneously lowered to achieve adaptive flow control of the unfreezing increment.

[0035] Optionally, a mask vector corresponding to the order freeze state is generated based on the collaborative scheduling framework, and a dynamic pruning threshold is set using the output congestion index. Combined with the action selection strategy, the output scheduling action set includes:

[0036] S41. Based on the production agents in the collaborative scheduling framework, construct the order freeze state vector and generate the corresponding mask vector based on the order freeze state vector;

[0037] S42. The generated mask vector is superimposed on the logical value layer for calculating the action probability of the production agent, and a dynamic pruning threshold is set in combination with the congestion index of the road network nodes.

[0038] S43. Based on the congestion index of road network nodes, calculate the congestion risk value of each logistics path, identify logistics paths with congestion risk values ​​higher than the dynamic pruning threshold as risk paths, and perform scheduling probability truncation on risk paths.

[0039] S44. Based on the logistics agent in the collaborative scheduling framework, calculate the comprehensive cost of the automated guided vehicle (AGV) performing the unfreezing task, select the optimal AGV with the goal of minimizing the comprehensive cost, and avoid the logistics path corresponding to the target congestion index through the optimal AGV.

[0040] S45. Combining the execution results of the risk path scheduling probability truncation with the optimal automated guided vehicle logistics path avoidance results, output the scheduling action set.

[0041] Optionally, the multi-objective hybrid reward function includes:

[0042] Based on the output scheduling action set, a completion time reward function, a logistics cost reward function, a congestion penalty function, and a deadlock penalty function are defined. Combined with preset completion time reward weights, logistics cost reward weights, and congestion penalty weights, a multi-objective hybrid reward function is constructed.

[0043] Optionally, based on the state transition equation of the Markov decision process model and the reward value of the multi-objective mixed reward function, the Markov decision process model is trained using a multi-agent proximal policy optimization algorithm and a curriculum learning strategy, including:

[0044] S61. Using the state transition equation of the Markov decision process model as the training environment, the state and congestion index of the target agent, production agent and logistics agent are integrated to construct the global state.

[0045] S62. Based on the pre-set critic network, with the reward value of the multi-objective mixed reward function as the optimization objective and the global state as the input, the corresponding loss function is constructed by calculating the deviation between the value estimate and the true target value, and the critic network is optimized using the loss function.

[0046] S63. Based on the optimized critic network and the local states of each agent, the generalized advantage estimation algorithm is used to calculate the advantage function, and combined with the preset actor network, the policy network of each agent is updated with minimizing the advantage function as the optimization objective.

[0047] S64. Based on the update results of the policy network of each agent, the agents are trained in segments under different intensity intelligent workshop load environments using the course learning strategy to complete the training of the Markov decision process model. The segmented training includes basic capability training, robustness training and extreme adaptive training.

[0048] Optionally, segmented training includes:

[0049] Based on the update results of the policy networks of each agent, the parameters of the unfreezing event generation function are adjusted to simulate intelligent workshop load environments of different intensities.

[0050] Set the thawing event density and thawing increment variance, and train each agent based on the settings until the preset termination condition is reached.

[0051] The unfreezing increment is set to follow a Poisson distribution of the basic probability, and each agent is trained based on the set result to adapt to the input of the neural differential equation.

[0052] The unfreezing increment is set as a superposition of the base load and the pulse load, and the mean and variance of the pulse load are gradually increased to drive each agent to execute adaptive flow control and action selection strategies in order to complete the training of the Markov decision process model.

[0053] Optionally, based on the trained Markov decision process model, the order table and automated guided vehicle (AGV) status table are read in real time to generate scheduling instructions, and the road network status is monitored in real time to trigger a deadlock circuit breaker mechanism, so as to achieve self-organized scheduling of logistics transportation resources, including:

[0054] S71. Deploy the trained Markov decision process model to the intelligent workshop control system, and read the order table in the manufacturing execution system and the automatic guided vehicle status table in the data acquisition and monitoring system in real time through the industrial Ethernet interface.

[0055] S72. Based on the read order table and the automated guided vehicle status table, by comparing the order status bits at adjacent times, identify the set of orders whose order status has changed from frozen to unfrozen, and calculate the corresponding change-unfrozen incremental features according to the dynamic equation.

[0056] S73. Based on the reading results, map the coordinates of the automated guided vehicles and the length of the machine queue in the smart workshop to the state vector of the Markov decision process model, and input the change and unfreeze incremental features into the collaborative scheduling framework to generate automated guided vehicle scheduling instructions.

[0057] S74. Parse the generated automated guided vehicle (AGV) scheduling instructions, obtain the control message, and send the control message to the robot control system and the production-logistics collaboration system to drive the AGV to perform logistics transportation tasks. The control message includes the source point, destination point, and path node sequence.

[0058] S75. Based on the execution results of the automated guided vehicles, monitor the average traffic speed of the road network in real time, and obtain the actual congestion level of the road network based on the average traffic speed;

[0059] S76. Compare and analyze the actual congestion level with the output congestion index. If the actual congestion level exceeds the congestion index, trigger the downgrade mode, stop the unfreezing of new orders, and continue until the road network is cleared, so as to realize the self-organized scheduling of logistics transportation resources.

[0060] According to another aspect of the present invention, a self-organizing scheduling system for intelligent workshop logistics transportation resources that takes into account order freezing and unfreezing is also provided, the system comprising:

[0061] The decision model building module is used to map the physical environment of the smart workshop into a workshop road network map, use dynamic equations to quantify the logistics increment caused by order freezing and unfreezing, and establish a Markov decision process model.

[0062] The congestion index output module is used to simulate the spatiotemporal evolution of road network load based on the established Markov decision process model, taking the pre-acquired historical traffic characteristics and unfreezing increment characteristics as input, and using the preset deep spatiotemporal graph neural differential equation to output the congestion index of road network nodes within the future time window.

[0063] The scheduling framework construction module is used to construct a collaborative scheduling framework based on the action space and congestion index of the Markov decision process model. The collaborative scheduling framework includes a target agent, a production agent, and a logistics agent.

[0064] The scheduling action output module is used to generate a mask vector corresponding to the order freeze status based on the collaborative scheduling framework, and to set a dynamic pruning threshold using the output congestion index, combined with the action selection strategy, to output a set of scheduling actions.

[0065] The reward function establishment module is used to establish a multi-objective hybrid reward function based on the output-based scheduling action set;

[0066] The decision model training module is used to train the Markov decision process model using a multi-agent proximal policy optimization algorithm and a course learning strategy, based on the state transition equation of the Markov decision process model and the reward value of the multi-objective mixed reward function.

[0067] The self-organizing scheduling module is used to read the order table and the automated guided vehicle status table in real time based on the trained Markov decision process model, generate scheduling instructions, and monitor the road network status in real time to trigger the deadlock circuit breaker mechanism, so as to realize the self-organizing scheduling of logistics transportation resources.

[0068] The beneficial effects of this invention are as follows:

[0069] 1. This invention introduces deep spatiotemporal graph neural differential equations, using the incremental characteristics of order unfreezing as input variables, which differs from the traditional discrete time step model. Relying on the extrapolation capability of neural differential equations in the continuous time domain, it accurately depicts the spatiotemporal diffusion trajectory of unfreezing flow waves in the workshop road network. In the early stage of order unfreezing, it can predict the continuous evolution trend of congestion index of each road network node in the future period, identify potential AGV deadlock risk areas in advance, and reduce the deadlock rate of the logistics system from the source.

[0070] 2. This invention constructs a collaborative scheduling framework, endowing the production intelligence agent with adaptive flow control capabilities based on road network congestion prediction results, and establishing a negative feedback closed-loop mechanism between order release rate and road network congestion index; when the road network is predicted to be overloaded, the production intelligence agent automatically tightens the issuance threshold of unfrozen orders, dynamically adjusts the order release rhythm, smooths out instantaneous traffic peaks, avoids the collapse of the logistics system caused by order unfreezing surges, and improves the operational stability of the workshop under high load conditions.

[0071] 3. This invention achieves a dynamic balance between production efficiency and logistics costs through a hierarchical intelligent agent architecture and a multi-objective hybrid reward function within a collaborative scheduling framework; it eliminates invalid decision-making actions of intelligent agents by using a frozen state mask to avoid ineffective resource consumption; it employs a congestion pruning strategy to reduce the probability of AGVs selecting high-risk paths, guiding AGVs to automatically avoid order unfreezing hotspots and balancing the load on the road network; the target intelligent agent can dynamically adjust strategy weights based on real-time operating conditions, flexibly switching between delivery priority and cost priority modes, ensuring that the workshop balances order processing speed and logistics energy consumption control during sudden unfreezing events, thereby improving overall completion efficiency.

[0072] 4. This invention uses a multi-agent near-end strategy optimization algorithm and a course learning strategy to gradually transition the training scenario from a sparse thawing scenario to a high-density, high-variance batch thawing scenario. Through phased training, the production and logistics intelligent agent system can master all-condition response strategies such as routine scheduling, emergency diversion, and degraded operation, ultimately achieving the technical goal of shortening the overall completion time of the workshop and making the logistics deadlock rate under high load conditions approach zero. Attached Figure Description

[0073] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0074] Figure 1 This is a flowchart of a self-organizing scheduling method for intelligent workshop logistics transportation resources that considers order freezing and unfreezing according to an embodiment of the present invention. Detailed Implementation

[0075] To further illustrate the various embodiments, the present invention provides accompanying drawings, which are part of the disclosure of the present invention. These drawings are mainly used to illustrate the embodiments and can be used in conjunction with the relevant descriptions in the specification to explain the operating principles of the embodiments. With reference to these drawings, those skilled in the art should be able to understand other possible implementation methods and the advantages of the present invention.

[0076] According to an embodiment of the present invention, a self-organizing scheduling method for intelligent workshop logistics transportation resources that takes into account order freezing and unfreezing is provided.

[0077] The present invention will now be further described in conjunction with the accompanying drawings and specific embodiments, such as... Figure 1 As shown, according to an embodiment of the present invention, a self-organizing scheduling method for intelligent workshop logistics transportation resources considering order freezing and unfreezing includes:

[0078] S1. Map the physical environment of the smart workshop into a workshop road network map, use dynamic equations to quantify the logistics increment caused by order freezing and unfreezing, and establish a Markov decision process model.

[0079] In this optional embodiment, the physical environment of the smart workshop is mapped as a workshop road network map, the logistics increment caused by order freezing and thawing is quantified using dynamic equations, and a Markov decision process model is established, including:

[0080] S11. Obtain the physical environment information of the smart workshop, and construct the workshop road network map using the road network in the physical environment information as nodes, logistics paths as edges, and physical distance or passage cost as edge weights.

[0081] S12. Based on the constructed workshop road network map, define the order set and determine the state of each order at any time. Establish a dynamic equation to quantify the logistics increment caused by order freezing and unfreezing.

[0082] S13. Based on the constructed workshop road network map and the quantified logistics increment, define the state space, action space and state transition equations to establish a Markov decision process model.

[0083] It should be further explained that establishing a Markov Decision Process (MDP) model for collaborative scheduling of production logistics in an intelligent workshop specifically includes creating a workshop road network map, defining the dynamic equations for order freezing / thawing, and constructing a hierarchical state space and action space. The workshop road network map maps the physical environment of the workshop into a weighted directed graph G, where nodes represent workstations, intersections, and stops, edges represent logistics paths, and the weights of the edges represent physical distances or travel costs, thereby establishing the spatial constraint boundaries for scheduling. The dynamic equations for order freezing / thawing define that orders have two states: frozen and thawing. The thawing event is defined as the incremental input of logistics demand at a specific node in the road network. This quantifies the dynamic disturbances of non-stationary environments on the scheduling system. The hierarchical state space and action space are based on the workshop road network diagram and dynamic equations. The state space of the MDP is defined to include road network congestion characteristics, order freeze status, and resource load distribution. The action space is defined to include production task assignment constrained by the current state and logistics path planning influenced by road network weights. Specific mathematical definitions are as follows:

[0084] (1) Workshop road network map:

[0085] Define the workshop logistics network as a workshop road network map. ; A collection of road network nodes (including workstations, intersections, and charging stations); For road segments; Let be the adjacency weight matrix, where This represents the physical distance or travel cost from node i to j.

[0086] (2) Dynamic equations:

[0087] Define order set Any order The state at time t is determined by the binary tuple. describe: This refers to the completion rate of the process. This is a frozen state indicator variable (1 for frozen, 0 for normal).

[0088] Define the instantaneous traffic increment generated when an order is unfrozen. The sum of state transitions at node n:

[0089] ;

[0090] In the formula, For the current subset of orders located at node n, For indicator functions, For orders The weight of logistics demand (depending on workpiece volume or priority).

[0091] Order sets provide basic data, and dynamic equations drive the evolution of data states. The resulting state differences are transformed into logistics increments, which ultimately change the load distribution of the physical road network.

[0092] (3) MDP state space :

[0093] Define the global state of the system at time t. For the set of triples:

[0094] ;

[0095] Environmental conditions : Contains a vector of predicted road network congestion indices Real-time unfreezing incremental distribution; order status : A vector containing the frozen status of all orders and progress vector Resource status Includes machine load matrix AGV Spatiotemporal Distribution .

[0096] (4) MDP action space :

[0097] Define hierarchical action space Target action : Trade-off factor vector Used to dynamically adjust the reward function; production actions Workpiece-machine matching matrix, subject to frozen state constraints (That is, frozen workpieces cannot be selected); logistics actions Task-AGV assignment matrix, where The logistics task set generated for PA This is the set of available AGVs.

[0098] (5) State transition equation:

[0099] Define the system state evolution as following a probability distribution :

[0100] ;

[0101] in, For random environmental disturbances (such as new freeze events or equipment failures); For MDP state space; This is the action space for MDP.

[0102] Furthermore, the fundamental purpose of constructing a Markov Decision Process (MDP) model is to map the multi-source, heterogeneous, and nonlinearly coupled physical entities and dynamic events in a smart workshop into a unified and computable mathematical state space, providing a standardized interactive environment for deep reinforcement learning algorithms. This goal is specifically achieved through three aspects of design: digital abstraction of physical entities, that is, abstracting discretely distributed resources such as machines, AGVs, and road networks into graph structures and state matrices, solving the problem that the physical environment is difficult for algorithms to understand directly; quantitative representation of non-stationary disturbances, targeting the core phenomenon of order freezing / thawing, transforming it from a qualitative production event into a quantitative state transition probability and flow increment input through dynamic equations, enabling the algorithm to numerically perceive environmental changes; and mathematical establishment of decision boundaries, clarifying the state space and action space of the intelligent agent, and defining a feasible solution space for multi-agent collaborative scheduling.

[0103] The constructed MDP model serves as the core architecture and system kernel of the self-organizing scheduling system, permeating the entire process. It provides underlying support for the prediction, decision-making, and training modules through three mechanisms: topology modeling, state constraints, and evolutionary guidance. Its functions are specifically reflected in the following aspects:

[0104] As the topological framework and external excitation source for congestion prediction, when conducting spatiotemporal extrapolation of road network congestion based on Deep Spatiotemporal Graph Neural Differential Equations (DSTGNDE), the MDP model completes the spatial topological anchoring and data input injection of the prediction module by constructing a workshop road network graph G=(V,E,W) (where V is the node set, E is the edge set, and W is the edge weight) and a thawing dynamic equation. Among them, the road network graph provides a topological benchmark for the evolution of the neural differential equation, clarifies the spatial connection relationship between nodes, and ensures that the congestion propagation model driven by fluid dynamics or queuing theory has a physically interpretable node-edge interaction paradigm. The logistics increment ΔQ, which is quantitatively calculated by the dynamic equation, directly serves as the external excitation feature of DSTGNDE, providing a time-series driving signal of order release-logistics change for congestion trend prediction.

[0105] As a state noise reduction and action constraint mechanism for collaborative decision-making, when the near-end policy optimization agent generates scheduling instructions, the MDP model achieves the filtering of invalid information and the selection of physically feasible actions before decision-making through the design of state space S and action space A. On the one hand, state space S extracts key features and filters out noise from workshop multimodal data (such as equipment status, order queue, and road network load), mapping high-dimensional raw data into low-dimensional decision features such as congestion index, frozen state position, and path reachability, reducing the agent's observation complexity. On the other hand, action space A and its constraint rules (such as freezing orders from scheduling and prohibiting the selection of unreachable paths) directly generate a mask matrix, forcing the agent to output actions that meet physical legality, and preventing invalid instructions such as scheduling frozen orders and planning unreachable paths from entering the execution layer.

[0106] As a closed-loop evolutionary environment and training guideline for feedback and evolution, the Markov Policy Gradient (MAPPO) algorithm uses the state transition function P and reward function R to simulate action-state evolution and transfer value guided by self-organization goals in policy updates. The state transition function describes the dynamic process of the workshop state from s to s' after executing action a (such as the change in path load caused by AGV movement and the queue growth caused by order unfreezing), providing continuous trajectory samples for MAPPO to support policy gradient estimation. The reward function R aims to minimize global completion time and suppress deadlock rate, transforming physical indicators such as workshop efficiency and stability into numerical reward signals, guiding the agent to iterate towards Pareto optimality in self-organization and cooperation. If the evolution and reward definitions of MDP are removed, MAPPO training will lose the physical logical anchor of action-state changes and the quantitative guidance of self-organization goals, causing policy updates to fall into aimless random exploration or local optima that contradict production needs.

[0107] S2. Based on the established Markov decision process model, the historical traffic characteristics and unfreezing increment characteristics obtained in advance are used as inputs. The pre-set deep spatiotemporal graph neural differential equation is used to simulate the spatiotemporal evolution of road network load and output the congestion index of road network nodes in the future time window.

[0108] In this optional embodiment, based on the established Markov decision process model, the pre-acquired historical traffic characteristics and unfreezing increment characteristics are used as inputs. A preset deep spatiotemporal graph neural differential equation is used to simulate the spatiotemporal evolution of the road network load, outputting the congestion index of road network nodes within the future time window, including:

[0109] S21. Based on the dynamic equations in the Markov decision process model, calculate and extract the unfreezing increment features of orders.

[0110] S22. Obtain the historical traffic characteristics of the road network, and concatenate the historical traffic characteristics with the unfrozen incremental characteristics at the channel level to generate the input tensor.

[0111] S23. Based on the generated input tensor, establish a dynamic adjacency matrix, which includes an adaptive adjacency matrix, a pattern similarity matrix, and a spatial similarity matrix.

[0112] In this optional embodiment, establishing the dynamic adjacency matrix includes:

[0113] Based on any node that has thawed and its corresponding downstream node, extract the corresponding historical traffic features;

[0114] Based on the extracted historical flow features and thawing increment features, the historical flow features and thawing increment features are mapped into an enhanced hidden state vector using linear transformation and nonlinear activation function.

[0115] By combining the enhanced hidden state vector obtained from the mapping with the preset unfreezing mode distribution vector, the influence intensity coefficient of each node on the downstream node is calculated using the attention mechanism, and the influence intensity coefficient is normalized by the Softmax function to generate the elements of the dynamic adjacency matrix.

[0116] Using the generated dynamic adjacency matrix as the core input, a graph convolutional network is used to propagate the spatial representation of congestion features to achieve congestion feature transmission.

[0117] S24. Substitute the established dynamic adjacency matrix into the preset deep spatiotemporal graph neural differential equation to construct a road network congestion prediction model to simulate the spatiotemporal evolution of road network load as order unfreezing increments, and output the congestion index of road network nodes within the future time window.

[0118] It should be noted that the road network congestion prediction model uses historical traffic characteristics of the road network and the incremental increase in order unfreezing. As a joint input, the continuous spatiotemporal evolution of the load state of road network nodes is simulated through neural differential equations, and the congestion index vector of road network nodes within the future time window is output. The specific calculation formula is as follows:

[0119] Phase 1, Feature Fusion:

[0120] Define the input tensor of the road network congestion prediction model Historical traffic characteristics of the road network Incremental characteristics of order unfreezing Channel-level splicing: ,in This characterizes the intensity of potential logistics demand generated by the unfrozen orders at each node at the current moment; among which ( ) indicates feature splicing.

[0121] The second stage involves learning dynamic graph structures.

[0122] Construct a dynamic adjacency matrix affected by the thawing distribution. This matrix is ​​composed of an adaptive adjacency matrix. Pattern similarity matrix spatial similarity matrix It is formed by fusion, as detailed below:

[0123] (1) Adaptive adjacency matrix Used to capture potential implicit dependencies between nodes:

[0124] ,in , An embedding matrix for learnable nodes. is the saturation coefficient, ReLU is the piecewise linear activation function, T is the matrix transpose, N is the flow feature number, and d is the matrix dimension.

[0125] (2) Pattern similarity matrix This is used to capture similar traffic load patterns caused by batch thawing; a thawing distribution feature vector is introduced. Participating in the calculation:

[0126] ;

[0127] in, The hidden state features of the nodes are represented by Softmax, which is a normalized exponential activation function, and D is the matrix dimension.

[0128] (3) Spatial similarity matrix Used to capture the spatial correlation of the current state of a node:

[0129] ;

[0130] (4) Dynamic graph fusion:

[0131] ;

[0132] FC stands for fully connected layer, and Mask is the masking operation.

[0133] The third stage, continuous spatiotemporal evolution:

[0134] Using Neural Differential Equations (NODEs) to simulate road network load conditions As the input continues to evolve with the thawing increment, its derivative is defined as a graph-diffusion convolution process:

[0135] ;

[0136] in This is the state transition function, which characterizes the continuous impact of the unfreezing order increment on the road network state; This is virtual time, and K is the convolutional layer diffusion depth. It is the identity matrix, used to calculate the state at future time T through a neural differential equation solver:

[0137] ;

[0138] in The output congestion index vector quantitatively characterizes the impact of thawing behavior on the future load of the road network.

[0139] Furthermore, the dynamic graph construction process specifically includes two cascaded stages: incremental weight activation and weight-based congestion propagation, as detailed below:

[0140] Phase 1: Establish a dynamic weight update mechanism based on the unfreezing increment:

[0141] Define dynamic connection weights for any node i that thaws and its downstream node j. The calculation formula is as follows:

[0142] Historical traffic characteristics of node i With thawing increment characteristics The enhanced hidden state vector is obtained through linear transformation and nonlinear activation mapping. :

[0143] ;

[0144] in This represents the concatenation operation; ReLU is a piecewise linear activation function. , These are learnable parameters.

[0145] Introducing the thawing mode distribution vector The influence strength coefficient of node i on node j is calculated using the attention mechanism. :

[0146] ;

[0147] in To query the key-value projection matrix, T is the matrix transpose. Scaling factor It is the enhanced hidden state vector of node j.

[0148] The elements of the normalized dynamic adjacency matrix are generated using the Softmax function. :

[0149] ;

[0150] Where exp(⋅) is the natural index. It is the original association strength between node i and node k at time t. For sets All elements k in the summation.

[0151] When node i experiences a thawing event, it causes When it increases, its hidden state is enhanced. A vector offset occurs, causing it to match the mode containing the same thawing pattern. The vector similarity of the downstream node j increases, thereby affecting the connection weight. The value increases.

[0152] Phase Two: Congestion Feature Transmission Process Based on Dynamic Weights

[0153] Using the updated dynamic adjacency matrix Spatial representation propagation (SRP) of congestion features is performed using graph convolutional networks:

[0154] ;

[0155] in The input features are the graph convolutional features of the l-th layer. For convolution kernel parameters, It is the identity matrix;

[0156] Due to Phase One As the value increases, the high load feature at node i caused by thawing is given a larger transfer coefficient in the convolution operation, so that the output feature of downstream node j contains the high-intensity congestion feature component from node i, realizing the logistics agent's perception of remote congestion.

[0157] S3. Based on the action space and congestion index of the Markov decision process model, construct a collaborative scheduling framework, which includes a target agent, a production agent, and a logistics agent.

[0158] In this optional embodiment, the production agent includes:

[0159] The production agent includes a state space and an action space. The state space includes the maximum value of the queue length of orders that are currently in the unfrozen state but have not been scheduled and the congestion index. When selecting orders to be processed, the action space must be constrained by a dynamic release threshold.

[0160] When the maximum value of the congestion index increases due to the unfreezing of batch orders, the dynamic release threshold is simultaneously lowered to achieve adaptive flow control of the unfreezing increment.

[0161] It should be further explained that the collaborative scheduling framework is a nested hierarchical deep reinforcement learning (NHDRL) collaborative scheduling framework, comprising an upper-layer target agent (OA), a middle-layer production agent (PA), and a lower-layer logistics agent (LA). The production agent establishes a negative feedback mechanism between order release rate and road network congestion index, while the logistics agent establishes a dynamic mapping mechanism between transportation path weights and regional load distribution. The decision logic of the production agent includes an adaptive flow control mechanism for unfreezing increments, as detailed below:

[0162] PA's state space Includes the length of the order queue that is currently unfrozen but not yet scheduled. and the predicted peak congestion index of the future road network PA's motion space Selecting workpieces At that time, it is limited by the dynamic release threshold. :

[0163] ;

[0164] in This represents the theoretical maximum vehicle capacity of the road network. Sensitivity coefficient This is the congestion warning threshold; when the unfreezing of batch orders leads to a predicted value... When it rises, As the number of tasks decreases, the PA agent proactively reduces the frequency of task issuance for unfreezing orders, thereby controlling the total number of tasks entering the logistics system and preventing the road network from becoming oversaturated.

[0165] S4. Generate a mask vector corresponding to the order freeze status based on the collaborative scheduling framework, set a dynamic pruning threshold using the output congestion index, and output a set of scheduling actions in combination with the action selection strategy.

[0166] In this optional embodiment, a mask vector corresponding to the order freeze state is generated based on the collaborative scheduling framework, and a dynamic pruning threshold is set using the output congestion index. Combined with the action selection strategy, the output scheduling action set includes:

[0167] S41. Based on the production agents in the collaborative scheduling framework, construct the order freeze state vector and generate the corresponding mask vector based on the order freeze state vector;

[0168] S42. The generated mask vector is superimposed on the logical value layer for calculating the action probability of the production agent, and a dynamic pruning threshold is set in combination with the congestion index of the road network nodes.

[0169] S43. Based on the congestion index of road network nodes, calculate the congestion risk value of each logistics path, identify logistics paths with congestion risk values ​​higher than the dynamic pruning threshold as risk paths, and perform scheduling probability truncation on risk paths.

[0170] S44. Based on the logistics agent in the collaborative scheduling framework, calculate the comprehensive cost of the automated guided vehicle (AGV) performing the unfreezing task, select the optimal AGV with the goal of minimizing the comprehensive cost, and avoid the logistics path corresponding to the target congestion index through the optimal AGV.

[0171] S45. Combining the execution results of the risk path scheduling probability truncation with the optimal automated guided vehicle logistics path avoidance results, output the scheduling action set.

[0172] It should be further explained that the action selection strategy utilizes the order freeze state to generate a hard constraint mask matrix, eliminating unschedulable objects; and uses the output congestion index... Generate soft-constraint pruning thresholds to reduce the probability of tasks being selected in high-load regions, as follows:

[0173] (1) Construct the hard constraint mask for order freezing:

[0174] For a production agent (PA), define the order freeze state vector at time t. ,in =1 indicates freezing, and a mask vector is constructed. :

[0175] ;

[0176] When calculating the probability distribution of production actions, the mask is superimposed on the Logits layer before Softmax: This forces the model to reduce the probability of selecting frozen workpieces to 0, ensuring the executability of scheduling instructions.

[0177] (2) Constructing soft-constraint pruning for road network congestion:

[0178] For logistics agents (LA), based on the output future road network congestion index vector Construct a dynamic pruning threshold : ,in They are respectively The mean and standard deviation, This is the adjustment coefficient.

[0179] For any candidate path p, calculate its congestion risk value. ;like If the path is deemed high-risk, its execution probability is truncated.

[0180] ;

[0181] in System state parameters The threshold is set, and ModelOutput is the decision probability output by the model. This step actively eliminates unfreezing task transportation schemes that would lead to road network oversaturation, while ensuring connectivity.

[0182] A multi-dimensional cost evaluation rule was adopted on the logistics agent (LA) side:

[0183] Calculate the unfreezing task for each candidate AGV (Automated Guided Vehicle). The overall cost :

[0184] ;

[0185] in Perform unfreezing task for AGV driving distance, Let k be the congestion index for the predicted path node k. For the predicted future load status of the AGV, , These are trainable weights.

[0186] Intelligent agents based on minimization The principle of selecting AGVs and planning to avoid high-altitude areas. The path of the node is determined to alleviate local resource contention caused by the unfreezing increment.

[0187] S5. Based on the output-based scheduling action set, establish a multi-objective hybrid reward function.

[0188] In this optional embodiment, the multi-objective hybrid reward function includes:

[0189] Based on the output scheduling action set, a completion time reward function, a logistics cost reward function, a congestion penalty function, and a deadlock penalty function are defined. Combined with preset completion time reward weights, logistics cost reward weights, and congestion penalty weights, a multi-objective hybrid reward function is constructed.

[0190] It should be further explained that the multi-objective hybrid reward function consists of four linearly weighted parts: completion efficiency reward, logistics cost reward, unfreezing congestion penalty, and deadlock avoidance penalty. It aims to guide the agent to maintain the road network load in a linearly unsaturated region while meeting the order unfreezing requirements, as detailed below:

[0191] ;

[0192] in, , The weight of each reward;

[0193] Completion time bonus Defined as:

[0194] ;

[0195] Where |O| represents the total number of orders. Let t be the number of operations completed for order j. Let J be the total number of processes for order j.

[0196] Logistics cost incentives :

[0197] ;

[0198] Where N is the number of AGVs. Let be the distance traveled by AGVk at time t. For the waiting time, and These are the energy consumption coefficients per unit distance traveled and per unit time spent waiting, respectively.

[0199] Congestion penalties Defined as:

[0200] ;

[0201] in, Let n be the current vehicle load. Design capacity for nodes, >1 represents a non-linear penalty exponent. The indicator function is used to apply a weighted penalty to the congestion status of the region where the unfreeze event occurred (UnfreezeNodes), where Nodes is the set of all nodes and UnfreezeNodes is the set of unfrozen nodes;

[0202] Deadlock penalty Defined as:

[0203] ;

[0204] Where M is a maximal positive number (penalty value). Let k be the instantaneous speed of AGV. This is the minimum average velocity threshold for determining system deadlock.

[0205] S6. Based on the state transition equation of the Markov decision process model and the reward value of the multi-objective mixed reward function, the Markov decision process model is trained using the multi-agent proximal policy optimization algorithm and the curriculum learning strategy.

[0206] In this optional embodiment, training the Markov decision process model using a multi-agent proximal policy optimization algorithm and a course learning strategy, based on the state transition equation of the Markov decision process model and the reward value of the multi-objective mixed reward function, includes:

[0207] S61. Using the state transition equation of the Markov decision process model as the training environment, the state and congestion index of the target agent, production agent and logistics agent are integrated to construct the global state.

[0208] S62. Based on the pre-set critic network, with the reward value of the multi-objective mixed reward function as the optimization objective and the global state as the input, the corresponding loss function is constructed by calculating the deviation between the value estimate and the true target value, and the critic network is optimized using the loss function.

[0209] S63. Based on the optimized critic network and the local states of each agent, the generalized advantage estimation algorithm is used to calculate the advantage function, and combined with the preset actor network, the policy network of each agent is updated with minimizing the advantage function as the optimization objective.

[0210] S64. Based on the update results of the policy network of each agent, the agents are trained in segments under different intensity intelligent workshop load environments using the course learning strategy to complete the training of the Markov decision process model. The segmented training includes basic capability training, robustness training and extreme adaptive training.

[0211] In this optional embodiment, segmented training includes:

[0212] Based on the update results of the policy networks of each agent, the parameters of the unfreezing event generation function are adjusted to simulate intelligent workshop load environments of different intensities.

[0213] Set the thawing event density and thawing increment variance, and train each agent based on the settings until the preset termination condition is reached.

[0214] The unfreezing increment is set to follow a Poisson distribution of the basic probability, and each agent is trained based on the set result to adapt to the input of the neural differential equation.

[0215] The unfreezing increment is set as a superposition of the base load and the pulse load, and the mean and variance of the pulse load are gradually increased to drive each agent to execute adaptive flow control and action selection strategies in order to complete the training of the Markov decision process model.

[0216] It should be further explained that the Multi-Agent Proximal Policy Optimization (MAPPO) algorithm and the Curriculum Learning (CurriculumLearning) strategy are as follows:

[0217] (1) The joint training implementation based on multi-agent proximal policy optimization (MAPPO) adopts a centralized training and distributed execution (CTDE) architecture:

[0218] Critic Network Update: Building Global State , , , It refers to the states of three agents. It is the congestion coefficient; Critic network The loss function aims to minimize the value estimation error and is as follows:

[0219] ;

[0220] in For calculating the multi-objective mixed reward value, ϕ is the learnable critic network parameter, and B is the number of samples sampled from the experience replay buffer during each training iteration. (This is a commentator's network) The value estimate of the global state of the i-th sample, where γ is the discount factor. It is the old critic network's value estimate of the global state of the i-th sample at the next time step.

[0221] Actor Network Update: Various Intelligent Agents Only observe local state ; Calculate the advantage function using generalized advantage estimation (GAE) Update the policy network :

[0222] ;

[0223] in Here, E[⋅] is the actor network loss function, and E[⋅] is the expectation. The ratio of new to old strategies. It is the dominant function. It is a clipping function. It's a clipping hyperparameter. It is the coefficient of the entropy regularization term. It is a measure of the stochasticity of agent k's policy; this process enables the prediction model (through state) Joint optimization of participation and decision-making models.

[0224] (2) Implementation of curriculum learning strategies based on the evolution of thawing distribution variance and density:

[0225] Design training courses with increasing difficulty. By controlling the thaw event generation function The parameters are used to simulate load environments of varying intensities; the thawing increment distribution is defined to follow a Gaussian mixture distribution. ,in It's weight. It is an expectation. It is variance; the implementation steps are as follows:

[0226] Phase 1: Basic Skills Training (Low Density, Low Variance):

[0227] Set the density of thawing events (Only random single-item unfreezing), variance The training agent is designed to master basic workpiece-machine matching and obstacle-free path planning capabilities. Termination conditions: deadlock rate is 0, and average completion time converges.

[0228] Phase Two: Robust Training (Medium Density, Medium Difference):

[0229] Increase event density And introduce moderate-intensity random fluctuations; set , It is a Poisson distribution. It is the base probability, used to train the agent to adapt to the predicted input of the DSTGNDE model and learn avoidance strategies under normal congestion conditions.

[0230] Phase Three: Extreme Adaptive Training (High Density, High Variance):

[0231] Simulating a batch thawing scenario, the thawing increment is set as a non-stationary pulse process:

[0232] ;

[0233] BaseLoad is the base load. It is the pulse time period, gradually increasing the pulse average value. (Simulated thawing volume is large) and variance, (Simulated thawing uncertainty) It is an indicator function; it forces the agent to learn adaptive flow control mechanisms and path pruning strategies until deadlock does not occur under 90% load conditions.

[0234] S7. Based on the trained Markov decision process model, read the order table and the automated guided vehicle status table in real time, generate scheduling instructions, and monitor the road network status in real time to trigger the deadlock circuit breaker mechanism to achieve self-organized scheduling of logistics transportation resources.

[0235] In this optional embodiment, based on the trained Markov decision process model, the order table and the automated guided vehicle status table are read in real time to generate scheduling instructions, and the road network status is monitored in real time to trigger a deadlock circuit breaker mechanism, so as to realize the self-organized scheduling of logistics transportation resources, including:

[0236] S71. Deploy the trained Markov decision process model to the intelligent workshop control system, and read the order table in the manufacturing execution system and the automatic guided vehicle status table in the data acquisition and monitoring system in real time through the industrial Ethernet interface.

[0237] S72. Based on the read order table and the automated guided vehicle status table, by comparing the order status bits at adjacent times, identify the set of orders whose order status has changed from frozen to unfrozen, and calculate the corresponding change-unfrozen incremental features according to the dynamic equation.

[0238] S73. Based on the reading results, map the coordinates of the automated guided vehicles and the length of the machine queue in the smart workshop to the state vector of the Markov decision process model, and input the change and unfreeze incremental features into the collaborative scheduling framework to generate automated guided vehicle scheduling instructions.

[0239] S74. Parse the generated automated guided vehicle (AGV) scheduling instructions, obtain control messages, and send the control messages to the robot control system and the production-logistics collaboration system to drive the AGV to perform logistics transportation tasks. The control messages include the source point, destination point, and path node sequence.

[0240] S75. Based on the execution results of the automated guided vehicles, monitor the average traffic speed of the road network in real time, and obtain the actual congestion level of the road network based on the average traffic speed;

[0241] S76. Compare and analyze the actual congestion level with the output congestion index. If the actual congestion level exceeds the congestion index, trigger the downgrade mode, stop the unfreezing of new orders, and continue until the road network is cleared, so as to realize the self-organized scheduling of logistics transportation resources.

[0242] It should be further explained that, based on the trained Markov decision process model, the order table and the automated guided vehicle status table are read in real time to generate scheduling instructions, and the deadlock circuit breaker mechanism is triggered in real time to monitor the road network status, so as to realize the self-organized scheduling of logistics transportation resources. Specifically, this includes:

[0243] (1) Real-time data mapping and state reconstruction:

[0244] The system reads the order table from the MES system and the equipment status table from the SCADA system in real time via the industrial Ethernet interface.

[0245] Freeze Identification: Compare the order status bits at time t with those at time t-1 to identify the set of orders whose status has changed from frozen to thawed, and calculate the thaw increment feature. .

[0246] State Tensor Construction: Mapping the AGV coordinates and machine queue length of the physical workshop to the state vectors of the MDP. and will Input into the pre-trained road network congestion prediction model.

[0247] (2) Prediction-Decision Collaborative Reasoning:

[0248] Feedforward prediction: Road network congestion prediction model based on Calculate the probability distribution of road network congestion in the next T steps. .

[0249] Hierarchical decision-making:

[0250] Target Agent (OA) Layer: Based on the current system's average late delivery rate, determine the reward weight vector that favors timely delivery or cost reduction.

[0251] Production Agent (PA) Layer: Receiver Calculate the dynamic release threshold If the current number of unfrozen orders > If so, the issuance of some orders will be temporarily suspended according to priority, i.e., an empty action will be output.

[0252] Logistics Agent (LA) Layer: For valid tasks issued by the PA, combined with... Perform path search and generate AGV scheduling instructions.

[0253] (3) Closed-loop execution and anomaly feedback:

[0254] Command issuance: The scheduling command is parsed into a control message consisting of a "source point, destination point, and path node sequence" and sent to the RCS (robot control system) and PLC (production-logistics collaboration system).

[0255] Deadlock Circuit Breaker: Real-time acquisition of the instantaneous speed of all active AGVs in the workshop to calculate the actual average travel speed of the road network. Using the speed-congestion mapping function Converting physical speed into a normalized level of actual congestion, where For free flow velocity; As a congestion-sensitive factor, it is usually taken as The actual congestion level is compared with the predicted congestion value from the DSTGNDE model. If the deviation exceeds a preset threshold... If the prediction fails or there is a physical anomaly, the degradation mode is immediately triggered, forcibly stopping the unfreezing and issuance of new orders until the actual average traffic speed returns to a safe level, thereby preventing the system from falling into an irreversible physical deadlock.

[0256] Furthermore, this invention constructs a Markov decision process model and defines order unfreezing as a non-stationary incremental logistics input. It then uses a road network congestion prediction model to deduce the evolution characteristics of road network congestion in continuous time and space. This congestion prediction result serves as a dynamic feedforward signal to the collaborative decision framework, driving the production agent to dynamically adjust the order release threshold based on congestion peaks, and the logistics agent to perform path planning pruning based on congestion distribution. The system state after decision execution triggers a multi-objective hybrid reward function, and the reward signal is used to jointly update the prediction and decision model parameters through a multi-agent near-end policy optimization algorithm, ultimately achieving closed-loop self-organizing scheduling from data perception to physical execution.

[0257] A specific embodiment of a self-organizing scheduling method for intelligent workshop logistics transportation resources that considers order freezing and unfreezing is as follows:

[0258] I. Application Scenario Construction: Flexible Machining Workshop for Automotive Engine Cylinder Blocks

[0259] This embodiment selects the intelligent workshop for machining engine cylinder blocks of an automotive parts manufacturing company as the physical object.

[0260] 1. Physical environment layout: The workshop area is A rectangular space containing:

[0261] Nodes: There are a total of N=60 topology nodes, including 30 CNC machining centers (divided into three sections: roughing, finishing, and cleaning), 1 raw material automated warehouse, 1 finished product warehouse, 2 AGV charging stations, and 26 road network intersections.

[0262] Logistics resources: Equipped with 15 lurking lifting AGVs, with a maximum load capacity of 1.5 tons and a maximum operating speed of 1.2 m / s.

[0263] Order characteristics: The daily order volume is approximately 500 units, and each cylinder requires 4-6 processes.

[0264] 2. Specific freezing / thawing condition descriptions:

[0265] Background: The workshop is implementing just-in-time production; at 09:00 AM, the quality inspection system detected a risk of porosity in a cast blank with batch number B20231015.

[0266] Freeze Event: The MES (Manufacturing Execution System) triggers a command to instantly set the status of 60 work-in-process orders bound to this batch of blanks to freeze. (Frozen); These orders, which were originally located in the blank warehouse waiting area and the rough processing buffer area, are now all suspended.

[0267] Thawing Surge: At 09:45 AM, manual re-inspection confirmed that this batch only had surface oxidation and no internal pores, thus it was deemed qualified; MES issued a batch release instruction, and the status of 60 orders was updated. All will turn to 0 within 1 second (thawed).

[0268] Challenge: Without this invention, the system would instantly generate 60 handling tasks; 15 AGVs would simultaneously rush to the raw material warehouse and the rough processing area, causing the main loop load rate of the workshop to instantly exceed 100%, resulting in a serious traffic deadlock.

[0269] II. Detailed Explanation of Implementation Steps:

[0270] 1. Constructing a digital twin and dynamic mapping of the workshop:

[0271] Road network mapping: Mapping the physical map of the workshop into a weighted directed graph. The initial weight W is the physical distance.

[0272] Dynamic monitoring: At 09:45:00, the system detected the blank storage node (node ​​ID: This can cause huge changes in order status.

[0273] Incremental calculation: according to the formula Calculation The instantaneous increase in logistics demand at the location is (45 orders need to be shipped from the warehouse), preliminary processing area node The increment is .

[0274] 2. Congestion evolution prediction based on DSTGNDE:

[0275] Feature fusion: Combining road network traffic features from the past 30 minutes Compared to the current thawing increment (Only in) and (Non-zero values ​​are present at the splicing point)

[0276] Dynamic graph evolution: Thawing mode vectors in the model This was identified as a typical raw material-preliminary processing thawing pattern; pattern similarity matrix. Automatic enhancement by point to The connection weights of all segments on the path.

[0277] Spatiotemporal simulation: Neural differential equation calculations show that without intervention, key intersections in the road network... Congestion Index It will reach a peak of 0.98 (extreme congestion) at t+3min.

[0278] 3. Source flow control of production agents (PAs):

[0279] Perception: PA obtains the predicted peak value .

[0280] Decision: Substitute into the dynamic threshold formula Assume road network capacity (Total number of AGVs), Sensitivity coefficient .

[0281] ;

[0282] Execution: Although 60 orders were unfrozen, PA only issued the 4 highest priority orders within this decision cycle (10 seconds), leaving the remaining 56 orders in the virtual queue to avoid impacting the road network.

[0283] 4. Congestion avoidance path planning for logistics intelligent agents (LA):

[0284] Perception: LA acquires The heat map revealed that the main road area was a high-risk zone.

[0285] Decision: Calculate alternative routes. Although the main road has the shortest distance, it incurs congestion costs. Extremely high.

[0286] Execution: LA directed the assigned AGV to select the outer loop for transportation. Although the single journey increased by 40 meters, the estimated travel time was reduced by 5 minutes (avoiding a deadlock).

[0287] 5. Closed-loop control and feedback:

[0288] As four orders are processed, the predicted congestion value $R_{peak}$ for the next moment decreases slightly, and PA gradually increases the release volume (e.g., to 6 per cycle).

[0289] The system successfully smoothed out the originally sharp "traffic surge" into a gentle "traffic flow" without any deadlocks occurring throughout the process.

[0290] III. Comparison of Experimental Data and Explanation of Attached Figures:

[0291] To verify the effect, a comparative experiment was conducted on the Simulation platform, as shown in Table 1.

[0292] Experimental group: This invention (DSTGNDE prediction + NHDRL synergy).

[0293] Control group 1: FIFO rule order placement + Dijkstra's shortest path.

[0294] Control group 2: Traditional multi-agent reinforcement learning (no unfrozen feature input, no flow control mechanism).

[0295] Table 1 Comparison of performance indicators under different thawing intensities

[0296]

[0297] According to another embodiment of the present invention, a self-organizing scheduling system for intelligent workshop logistics transportation resources that takes into account order freezing and unfreezing is also provided. The system includes:

[0298] The decision model building module is used to map the physical environment of the smart workshop into a workshop road network map, use dynamic equations to quantify the logistics increment caused by order freezing and unfreezing, and establish a Markov decision process model.

[0299] The congestion index output module is used to simulate the spatiotemporal evolution of road network load based on the established Markov decision process model, taking the pre-acquired historical traffic characteristics and unfreezing increment characteristics as input, and using the preset deep spatiotemporal graph neural differential equation to output the congestion index of road network nodes within the future time window.

[0300] The scheduling framework construction module is used to construct a collaborative scheduling framework based on the action space and congestion index of the Markov decision process model. The collaborative scheduling framework includes a target agent, a production agent, and a logistics agent.

[0301] The scheduling action output module is used to generate a mask vector corresponding to the order freeze status based on the collaborative scheduling framework, and to set a dynamic pruning threshold using the output congestion index, combined with the action selection strategy, to output a set of scheduling actions.

[0302] The reward function establishment module is used to establish a multi-objective hybrid reward function based on the output-based scheduling action set;

[0303] The decision model training module is used to train the Markov decision process model using a multi-agent proximal policy optimization algorithm and a course learning strategy, based on the state transition equation of the Markov decision process model and the reward value of the multi-objective mixed reward function.

[0304] The self-organizing scheduling module is used to read the order table and the automated guided vehicle status table in real time based on the trained Markov decision process model, generate scheduling instructions, and monitor the road network status in real time to trigger the deadlock circuit breaker mechanism, so as to realize the self-organizing scheduling of logistics transportation resources.

[0305] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A self-organizing scheduling method for intelligent workshop logistics transportation resources considering order freezing and unfreezing, characterized in that, The method includes: S1. Map the physical environment of the smart workshop into a workshop road network map, use dynamic equations to quantify the logistics increment caused by order freezing and unfreezing, and establish a Markov decision process model. S2. Based on the established Markov decision process model, the historical traffic characteristics and unfreezing increment characteristics obtained in advance are used as inputs. The pre-set deep spatiotemporal graph neural differential equation is used to simulate the spatiotemporal evolution of road network load and output the congestion index of road network nodes in the future time window. S3. Based on the action space and congestion index of the Markov decision process model, construct a collaborative scheduling framework, which includes a target agent, a production agent, and a logistics agent. S4. Generate a mask vector corresponding to the order freeze status based on the collaborative scheduling framework, and use the output congestion index to set a dynamic pruning threshold. Combined with the action selection strategy, output a set of scheduling actions. S5. Based on the output-based scheduling action set, establish a multi-objective hybrid reward function; S6. Based on the state transition equation of the Markov decision process model and the reward value of the multi-objective mixed reward function, the Markov decision process model is trained using the multi-agent proximal policy optimization algorithm and the course learning strategy. S7. Based on the trained Markov decision process model, read the order table and the automated guided vehicle status table in real time, generate scheduling instructions, and monitor the road network status in real time to trigger the deadlock circuit breaker mechanism, so as to realize the self-organized scheduling of logistics transportation resources.

2. The intelligent workshop logistics transportation resource self-organizing scheduling method considering order freezing and unfreezing as described in claim 1, characterized in that, The process of mapping the physical environment of the smart workshop into a workshop road network map, quantifying the logistics increment caused by order freezing and thawing using dynamic equations, and establishing a Markov decision process model includes: S11. Obtain the physical environment information of the smart workshop, and construct the workshop road network map using the road network in the physical environment information as nodes, logistics paths as edges, and physical distance or passage cost as edge weights. S12. Based on the constructed workshop road network map, define the order set and determine the state of each order at any time. Establish a dynamic equation to quantify the logistics increment caused by order freezing and unfreezing. S13. Based on the constructed workshop road network map and the quantified logistics increment, define the state space, action space and state transition equations to establish a Markov decision process model.

3. The intelligent workshop logistics transportation resource self-organizing scheduling method considering order freezing and unfreezing as described in claim 1, characterized in that, The established Markov decision process model takes pre-acquired historical traffic characteristics and unfreezing increment characteristics as input, uses a preset deep spatiotemporal graph neural differential equation to simulate the spatiotemporal evolution of road network load, and outputs the congestion index of road network nodes within the future time window, including: S21. Based on the dynamic equations in the Markov decision process model, calculate and extract the unfreezing increment features of orders; S22. Obtain the historical traffic characteristics of the road network, and concatenate the historical traffic characteristics with the unfreezing incremental characteristics at the channel level to generate the input tensor; S23. Based on the generated input tensor, establish a dynamic adjacency matrix, which includes an adaptive adjacency matrix, a pattern similarity matrix, and a spatial similarity matrix. S24. Substitute the established dynamic adjacency matrix into the preset deep spatiotemporal graph neural differential equation to construct a road network congestion prediction model to simulate the spatiotemporal evolution of road network load as order unfreezing increments, and output the congestion index of road network nodes within the future time window.

4. The intelligent workshop logistics transportation resource self-organizing scheduling method considering order freezing and unfreezing according to claim 3, characterized in that, The establishment of the dynamic adjacency matrix includes: Based on any node that has thawed and its corresponding downstream node, extract the corresponding historical traffic features; Based on the extracted historical flow features and thawing increment features, the historical flow features and thawing increment features are mapped into an enhanced hidden state vector using linear transformation and nonlinear activation function. By combining the enhanced hidden state vector obtained from the mapping with the preset unfreezing mode distribution vector, the influence intensity coefficient of each node on the downstream node is calculated using the attention mechanism, and the influence intensity coefficient is normalized by the Softmax function to generate the elements of the dynamic adjacency matrix. Using the generated dynamic adjacency matrix as the core input, a graph convolutional network is used to propagate the spatial representation of congestion features to achieve congestion feature transmission.

5. The intelligent workshop logistics transportation resource self-organizing scheduling method considering order freezing and unfreezing according to claim 1, characterized in that, The production intelligent agent includes: The production agent includes a state space and an action space. The state space includes the length of the order queue that is currently in a thawing state but has not been scheduled and the maximum value of the congestion index. When selecting orders to be processed, the action space must be constrained by a dynamic release threshold. When the maximum value of the congestion index increases due to the unfreezing of batch orders, the dynamic release threshold is simultaneously lowered to achieve adaptive flow control of the unfreezing increment.

6. The intelligent workshop logistics transportation resource self-organizing scheduling method considering order freezing and unfreezing according to claim 1, characterized in that, The process involves generating a mask vector corresponding to the order freeze state based on the collaborative scheduling framework, setting a dynamic pruning threshold using the output congestion index, and combining this with an action selection strategy to output a set of scheduling actions, including: S41. Based on the production agents in the collaborative scheduling framework, construct the order freeze state vector and generate the corresponding mask vector based on the order freeze state vector; S42. The generated mask vector is superimposed on the logical value layer for calculating the action probability of the production agent, and a dynamic pruning threshold is set in combination with the congestion index of the road network nodes. S43. Based on the congestion index of road network nodes, calculate the congestion risk value of each logistics path, identify logistics paths with congestion risk values ​​higher than the dynamic pruning threshold as risk paths, and perform scheduling probability truncation on risk paths. S44. Based on the logistics agent in the collaborative scheduling framework, calculate the comprehensive cost of the automated guided vehicle (AGV) performing the unfreezing task, select the optimal AGV with the goal of minimizing the comprehensive cost, and avoid the logistics path corresponding to the target congestion index through the optimal AGV. S45. Combining the execution results of the risk path scheduling probability truncation with the optimal automated guided vehicle logistics path avoidance results, output the scheduling action set.

7. The intelligent workshop logistics transportation resource self-organizing scheduling method considering order freezing and unfreezing according to claim 1, characterized in that, The multi-objective hybrid reward function includes: Based on the output scheduling action set, a completion time reward function, a logistics cost reward function, a congestion penalty function, and a deadlock penalty function are defined. Combined with preset completion time reward weights, logistics cost reward weights, and congestion penalty weights, a multi-objective hybrid reward function is constructed.

8. The intelligent workshop logistics transportation resource self-organizing scheduling method considering order freezing and unfreezing according to claim 1, characterized in that, The step of training the Markov decision process model using a multi-agent proximal policy optimization algorithm and a curriculum learning strategy, based on the state transition equation of the Markov decision process model and the reward value of the multi-objective hybrid reward function, includes: S61. Using the state transition equation of the Markov decision process model as the training environment, the state and congestion index of the target agent, production agent and logistics agent are integrated to construct the global state. S62. Based on the pre-set critic network, with the reward value of the multi-objective mixed reward function as the optimization objective and the global state as the input, the corresponding loss function is constructed by calculating the deviation between the value estimate and the true target value, and the critic network is optimized using the loss function. S63. Based on the optimized critic network and the local states of each agent, the generalized advantage estimation algorithm is used to calculate the advantage function, and combined with the preset actor network, the policy network of each agent is updated with minimizing the advantage function as the optimization objective. S64. Based on the update results of the policy network of each agent, the agents are trained in segments under different intensity intelligent workshop load environments using a course learning strategy to complete the training of the Markov decision process model. The segmented training includes basic capability training, robustness training and extreme adaptive training.

9. A self-organizing scheduling method for intelligent workshop logistics transportation resources considering order freezing and unfreezing, as described in claim 8, is characterized in that... The segmented training includes: Based on the update results of the policy networks of each agent, the parameters of the unfreezing event generation function are adjusted to simulate intelligent workshop load environments of different intensities. Set the thawing event density and thawing increment variance, and train each agent based on the settings until the preset termination condition is reached. The unfreezing increment is set to follow a Poisson distribution of the basic probability, and each agent is trained based on the set result to adapt to the input of the neural differential equation. The unfreezing increment is set as a superposition of the base load and the pulse load, and the mean and variance of the pulse load are gradually increased to drive each agent to execute adaptive flow control and action selection strategies in order to complete the training of the Markov decision process model.

10. A self-organizing scheduling method for intelligent workshop logistics transportation resources considering order freezing and unfreezing, as described in claim 1, is characterized in that... The process, based on a trained Markov decision process model, involves real-time reading of the order table and automated guided vehicle (AGV) status table to generate scheduling instructions and real-time monitoring of the road network status to trigger a deadlock circuit breaker mechanism, thereby achieving self-organized scheduling of logistics transportation resources. This includes: S71. Deploy the trained Markov decision process model to the intelligent workshop control system, and read the order table in the manufacturing execution system and the automatic guided vehicle status table in the data acquisition and monitoring system in real time through the industrial Ethernet interface. S72. Based on the read order table and the automated guided vehicle status table, by comparing the order status bits at adjacent times, identify the set of orders whose order status has changed from frozen to unfrozen, and calculate the corresponding change-unfrozen incremental features according to the dynamic equation. S73. Based on the reading results, map the coordinates of the automated guided vehicles and the length of the machine queue in the smart workshop to the state vector of the Markov decision process model, and input the change and unfreeze incremental features into the collaborative scheduling framework to generate automated guided vehicle scheduling instructions. S74. Parse the generated automated guided vehicle (AGV) scheduling instruction, obtain the control message, and send the control message to the robot control system and the production-logistics collaboration system to drive the AGV to perform logistics transportation tasks. The control message includes the source point, the destination point, and the path node sequence. S75. Based on the execution results of the automated guided vehicles, monitor the average traffic speed of the road network in real time, and obtain the actual congestion level of the road network based on the average traffic speed; S76. Compare and analyze the actual congestion level with the output congestion index. If the actual congestion level exceeds the congestion index, trigger the downgrade mode, stop the unfreezing of new orders, and continue until the road network is cleared, so as to realize the self-organized scheduling of logistics transportation resources.

Images