Intelligent beam yard multi-torpedo tank dynamic collaborative material distribution control method and system

Abstract

The application provides a kind of wisdom beam field multi-torpedo can dynamic collaborative material distribution control method and system, based on graph structure extraction each torpedo can fusion space-time characteristic vector;The fusion space-time characteristic vector of each node, current observation state, environmental state and action set are input into Actor-Critic neural network, introduce action mask to output the action with safety and the maximum reward function from action set as the current best action of each torpedo can.The application adopts lightweight PPO algorithm architecture, introduces multi-head graph attention mechanism, gives the system real-time perception and autonomous learning ability, not only can effectively solve the real-time path planning and conflict obstacle avoidance of variable quantity torpedo can in complex environment, but also can realize the efficient cooperation of multi-source and multi-destination task under the premise of ensuring the absolute safety of multi-vehicle operation, significantly improve the control accuracy, execution efficiency and production resilience of distribution system.

Description

Technical Field

[0001] This invention relates to the field of large-scale material distribution in smart beam yards, and more specifically, to a method and system for dynamic collaborative material delivery control of multiple torpedo tanks in smart beam yards. Background Technology

[0002] In the scenario of large-scale material preparation in a smart beam yard, as the number of torpedo canisters increases, the traditional PLC fixed logic and manual intervention mode exposes drawbacks such as high complexity of collaborative scheduling, poor adaptability to dynamic environments, and limited system scalability. This can easily lead to control disorder and torpedo canister congestion. Moreover, it is difficult to balance high positioning accuracy and absolute safety when pursuing delivery speed, and it cannot meet the needs of modern production for efficient and flexible scheduling. Summary of the Invention

[0003] This invention addresses the technical problems existing in the prior art by providing a smart beam yard torpedo tank dynamic collaborative material delivery control method and system, which can overcome the existing problems of collaborative operation of multiple torpedo tanks in complex scenarios.

[0004] According to a first aspect of the present invention, a method for dynamic collaborative material delivery control of multiple torpedo tanks in a smart beam yard is provided, comprising: The operating area of ​​the smart beam yard is abstracted as a graph structure, and the discrete torpedo canisters are abstracted as dynamic nodes in the graph structure. Based on the heterogeneous feature fusion network, the fused spatiotemporal feature vector of each torpedo tank in the graph structure is extracted. The fused spatiotemporal feature vector is formed by concatenating the vehicle feature vector and the context feature vector aggregated by the multi-head attention mechanism. The fused spatiotemporal feature vector of each torpedo canister, the current observation state of each torpedo canister, the environmental state and action set within the operating area of ​​the smart beam yard are input into the Actor-Critic neural network. Based on the Actor-Critic neural network, an action mask mechanism is introduced to output the action with the highest safety and reward function from the action set, which is taken as the current best action for each torpedo canister. Control each torpedo canister to operate according to the current optimal action.

[0005] According to a second aspect of the present invention, a smart beam yard multi-torpedo tank dynamic collaborative material delivery control system is provided, comprising: The construction module is used to construct the operating area of ​​the smart beam yard as a graph structure, and to abstract discrete torpedo canisters as dynamic nodes in the graph structure. The extraction module is used to extract the fused spatiotemporal feature vector of each torpedo tank in the graph structure based on the heterogeneous feature fusion network. The fused spatiotemporal feature vector is formed by concatenating the vehicle feature vector and the context feature vector aggregated by the multi-head attention mechanism. The output module is used to input the fused spatiotemporal feature vector of each torpedo can, the current observation state of each torpedo can, the environmental state and action set within the operating area of ​​the smart beam yard into the Actor-Critic neural network. Based on the Actor-Critic neural network, an action mask mechanism is introduced to output the action with the highest safety and the highest reward function from the action set, which is the current best action for each torpedo can. The control module is used to control each torpedo canister to operate according to the current optimal action.

[0006] This invention provides a dynamic collaborative material delivery control method and system for multiple torpedo canisters in a smart beam yard. By adopting a lightweight PPO (Proximal Policy Optimization) algorithm architecture and introducing a multi-head graph attention mechanism, the system is endowed with real-time perception and autonomous learning capabilities. This not only effectively solves the real-time path planning and conflict avoidance of a variable number of torpedo canisters in complex environments, but also achieves efficient collaboration of multi-source and multi-destination tasks while ensuring the absolute safety of multi-vehicle operation, significantly improving the control accuracy, execution efficiency and production resilience of the delivery system. Attached Figure Description

[0007] Figure 1 Flowchart of automatic control process for batching plant in smart beam yard; Figure 2 A schematic diagram illustrating the trajectory of a torpedo canister. Figure 3 This is a schematic diagram of the torpedo tank's operating cycle. Figure 4 Control flow chart for torpedo can #1; Figure 5 Control flow chart for torpedo can #2; Figure 6 Control flow chart for torpedo canister #3; Figure 7 A flowchart of a dynamic collaborative material delivery control method for torpedo canisters in a smart beam yard, provided as an embodiment of the present invention; Figure 8 Diagram of the Actor-Critic neural network architecture; Figure 9 The entire network architecture diagram for dynamic collaborative material delivery control of torpedo tanks in a smart beam yard; Figure 10 The diagram shows the structure of the PPO algorithm model based on the masking mechanism. Figure 11 This is a schematic diagram of a smart beam yard multi-torpedo tank dynamic collaborative material distribution control system provided in one embodiment of the present invention. Detailed Implementation

[0008] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention. In addition, the technical features of the various embodiments or individual embodiments provided by the present invention can be arbitrarily combined with each other to form feasible technical solutions. Such combinations are not constrained by the order of steps and / or structural composition patterns, but must be based on the ability of those skilled in the art to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection claimed by the present invention.

[0009] Before describing the intelligent beam yard multi-torpedo tank collaborative material delivery method provided by the present invention, we will first introduce the design and implementation method of the intelligent beam yard mixing plant full-process automated batching control closed loop.

[0010] The automatic control process of the mixing plant starts with the "production instruction trigger" and forms a complete closed loop covering "raw material preparation - precise batching - orderly conveying - efficient mixing - standardized unloading". Each link realizes data linkage and action coordination through the automated control system, ensuring that every operation meets the preset production standards and that the whole process is monitorable and traceable. Figure 1 This is a block diagram of the automatic control system for a batching plant.

[0011] See Figure 1 The entire process is initiated by a filling command or batching signal, and proceeds sequentially through four stages: intermediate silo material selection (after intermediate silo material selection is completed, the silo outlet valve and intermediate silo inlet valve are triggered to open); intermediate silo loading (after intermediate silo loading is completed, the batching end signal, intermediate silo outlet valve, mixing plant inlet valve, and intermediate silo inlet valve are triggered to close); mixing plant loading (after mixing plant loading is completed, the intermediate silo is emptied, the intermediate silo outlet valve is closed, and the mixing motor is activated); and the mixing motor stops. After this, the mixing tank outlet valve is triggered to open, and after the mixing tank is unloaded, the mixing tank is emptied, thus completing the entire unloading process. After unloading is completed, the mixing tank outlet valve is closed and the batching signal is triggered, which then restarts the filling command, allowing the process to enter the next cycle.

[0012] The PLC-based method for zoned operation control and multi-tank collaborative scheduling of torpedo tanks is as follows: The torpedo canister's operating trajectory is divided into 5 zones, each corresponding to a specific control program. The control flow is as follows: R1: Fully loaded torpedo canister proceeds to unloading point #1; R2: Fully loaded torpedo canister proceeds to unloading point #2; M1: Empty torpedo canister is loaded; L1: Empty torpedo canister proceeds to the loading area; L2: Empty torpedo canister proceeds to the standby area. The operating cycle for torpedo canister #1 is: R1-L2-L1-M1-R2-L2-L1-M1-R1; the operating cycle for torpedo canister #2 is: M1-R2-L2-L1-M1-R1-L2-L1-M1; the operating cycle for torpedo canister #3 is: L1-M1-R1-L2-L1-M1-R2-L2-L1. Figure 2 This is a diagram showing the trajectory of a torpedo cannon. Figure 3 This is a cycle diagram of the torpedo tank operation process. Figure 4 This is the control block diagram for torpedo can #1. Figure 5 This is the control block diagram for torpedo can #2. Figure 6 This is the control block diagram for torpedo tank #3.

[0013] The torpedo tank's movement control uses a PLC as the core control unit, paired with a GA700 series walking frequency converter and an NPN type metal proximity switch, to achieve orderly switching and precise execution of programs in each area, and all control command transmissions follow the source logic of the electrical system.

[0014] For process R1 (fully loaded torpedo canister heading to unloading point #1), the triggering conditions are an empty tank signal from the mixing tank and a full-load signal from the torpedo canister's weight sensor. When both conditions are met simultaneously, the torpedo canister motor starts, causing the torpedo canister to reach a speed of V0. When the torpedo canister reaches a designated position, a photoelectric switch receives a signal, which in turn triggers the torpedo canister motor to brake, stopping the torpedo canister at unloading point #1. Simultaneously, when the photoelectric switch at unloading point #1 receives a signal (unloading point #1 is connected to the torpedo canister) and the photoelectric switch at the concrete placing boom receives a signal (concrete placing boom is connected to unloading point #1), the torpedo canister receives an unloading signal. Simultaneously, when the torpedo canister receives both the unloading signal and the empty-load signal, it receives a reset signal. When the photoelectric switch at unloading point #1 receives a signal (unloading point #1 is connected to the torpedo canister), the torpedo canister's weight sensor receives an empty-load signal, and the torpedo canister receives a reset signal, the motor starts. At this point, the torpedo canister completes the unloading process, and process R1 ends.

[0015] For the R2 process (fully loaded torpedo canister heading to unloading point #2), the triggering conditions are an empty tank signal from the mixing tank and a full-load signal from the torpedo canister's weight sensor. When both conditions are met simultaneously, the torpedo canister motor starts, causing the torpedo canister to reach a speed of V0. When the torpedo canister reaches a designated position, a photoelectric switch receives a signal, which in turn triggers the torpedo canister motor to brake, stopping the torpedo canister at unloading point #2. Simultaneously, when the photoelectric switch at unloading point #2 receives a signal (unloading point #2 is connected to the torpedo canister), and the photoelectric switch at the concrete placing boom receives a signal (concrete placing boom is connected to unloading point #2), the torpedo canister receives an unloading signal. Simultaneously, when the torpedo canister receives both the unloading signal and the empty-load signal, it receives a reset signal. When the photoelectric switch at unloading point #2 receives a signal (unloading point #2 is connected to the torpedo canister), the torpedo canister's weight sensor receives an empty-load signal, and the torpedo canister receives a reset signal, the motor starts. At this point, the torpedo canister completes the unloading process, and the R2 process ends.

[0016] For the M1 process (loading empty torpedo canisters), the trigger condition is that the sensor in the loading area receives a signal. When the sensor in the loading area receives a signal, the discharge valve of the mixing tank is triggered to open, causing the mixing tank to start unloading. When the mixing tank receives an empty tank signal, the discharge valve of the mixing tank is closed. When the torpedo canister weight sensor receives a full load signal, the M1 process ends.

[0017] For the L1 process (empty torpedo canister moving to the loading area), the triggering condition is that the sensor in the loading area does not detect a signal, and the sensor in the standby area receives a signal. When both of the above conditions are met, the motor is triggered to start, and the motor speed reaches V2. When the torpedo canister reaches the limit position, the photoelectric switch is triggered to receive a signal, which in turn triggers the torpedo canister motor to brake, and the torpedo canister stops at the loading area. When the sensor in the loading area receives a signal, the L1 process ends.

[0018] For the L2 process (unloaded torpedo canister moving to the standby area), the triggering conditions are that the torpedo canister's weight sensor receives an unloaded signal and the torpedo canister receives a reset signal. When both conditions are met simultaneously, the torpedo canister is triggered to start, causing its travel speed to reach V1. When the torpedo canister reaches the designated position, the photoelectric switch receives a signal, which in turn triggers the torpedo canister's motor braking, causing the torpedo canister to stop in the standby area. When the standby area photoelectric switch receives a signal (the torpedo canister has stopped in the standby area), the L2 process ends.

[0019] Torpedo canisters #1, #2, and #3 operate on differentiated cycle schedules. The PLC uses a combination of sequential program execution and staggered scheduling logic to prevent trajectory conflicts. (Next) Figure 3The system presents a cycle diagram of the torpedo canister operation. For example, when torpedo canister #1 executes process R1, torpedo canisters #2 and #3 execute processes M1 and L1 respectively. Since each process has the same execution time, each torpedo canister moves in a different area, ensuring orderly feeding and guaranteeing equipment safety. Simultaneously, the operating status of all torpedo canisters is displayed in real-time on the control panel. Operators can intervene via emergency stop key switches or remote commands to ensure the safety and continuity of multi-torpedo canister coordinated operation.

[0020] To overcome the complexity bottleneck of traditional PLC logic when handling more than three torpedo canisters and to address the real-time challenges of multi-vehicle path planning and obstacle avoidance in dynamic environments, this invention further proposes a smart beam yard torpedo canister dynamic collaborative material delivery control method. This method adopts a lightweight PPO (Proximal Policy Optimization) algorithm architecture to achieve dynamic collaborative navigation from "multi-source to single-purpose" or "multi-source multi-purpose".

[0021] See Figure 7 This paper illustrates a dynamic collaborative material delivery control method for torpedo containers in a smart beam yard, according to an embodiment of the present invention. The control method includes the following steps: Step 1: Abstract the operating area of ​​the smart beam yard into a graph structure, and abstract the discrete torpedo canisters into dynamic nodes in the graph structure.

[0022] Understandably, the operating area of ​​the smart beam yard is first abstracted as a two-dimensional gridded environment or graph structure network G, and its location is defined as... In this environment, the torpedo can intelligent agent set is defined as follows: Where N is the total number of torpedo canisters, and N can be dynamically adjusted according to production needs. The environment also includes a set of static obstacles. (Such as torpedo canisters and baffles that are being unloaded) and a collection of dynamic obstacles. (e.g., pedestrians, vehicles). The system's control objective is to achieve the following for each torpedo canister at time t: Planning the best action This allows it to reach the target location in the shortest possible time while avoiding collisions with obstacles or other torpedo canisters.

[0023] In the design of the state space, to reduce computational cost and improve generalization ability, this embodiment of the invention does not directly input the entire map image, but instead uses relative coordinate features, combined with an innovative heterogeneous feature fusion network with multi-head graph attention. This network abstracts discrete torpedo canisters into a graph structure. The dynamic nodes in the network break the limitation of fixed input dimensions in traditional fully connected networks.

[0024] Step 2: Extract the fused spatiotemporal feature vector of each torpedo tank in the graph structure based on the heterogeneous feature fusion network. The fused spatiotemporal feature vector is formed by concatenating the vehicle feature vector and the context feature vector aggregated by the multi-head attention mechanism.

[0025] Understandably, based on the graph structure, the feature vector of each node (torpedo canister) in the graph structure is extracted. Specifically, for a system containing N torpedo canisters, the node feature matrix is ​​represented as follows: This enables the system to handle the dynamic addition and subtraction of multiple torpedo tank arrays in real time, significantly improving the scheduling flexibility of the smart beam yard under complex working conditions.

[0026] In this embodiment of the invention, a dual-stream sensing model is constructed in the initial stage of feature extraction to handle the absolute attributes of the vehicle itself and the relative relationships between multiple vehicles. For each torpedo canister... , where i is the torpedo canister number, and its vehicle feature vector. Includes the current normalized position Instantaneous velocity and load condition Mapped to a high-dimensional latent feature space via a linear embedding layer: .

[0027] At the same time, in order to characterize the spatial dependence in multi-machine collaboration, taking current vehicles as an example... Centered on this, other torpedo canisters within its sensing range (j is the number of the torpedo can adjacent to torpedo can i) are considered as adjacent nodes, and their relative state characteristics are calculated. .

[0028] The feature vector of this vehicle and the relative state characteristics between them The input is fed into the multi-head image attention layer, and an attention score is calculated. From the master learning weight:

[0029] in, Representing relative state characteristics The attention score of the k-th attention head. For activation function, Let be the weight matrix of the k-th attention head. Let T represent the learnable parameter vector of the k-th attention head, and let T denote the transpose.

[0030] After normalizing the weights using the Softmax function, we obtain the first... k Note the coefficient below the head:

[0031] This mechanism can prioritize vehicles with overlapping travel paths. It automatically ignores vehicles that are physically close but traveling in the opposite direction. .

[0032] In the deep feature fusion stage, through K Contextual features obtained by aggregating attention heads Calculate according to the following formula:

[0033] Then, Features of this vehicle Nonlinear splicing is performed, and the data is fed into a gated loop unit (GRU) with time memory function to obtain each torpedo canister. The fusion of spatiotemporal feature vectors. GRU update gate. and reset door The formula is as follows:

[0034]

[0035] Formulas (5) and (6) together define the core gating mechanism of the GRU unit, where and These represent the output coefficients of the update gate (which determines how much historical state to retain) and the reset gate (which determines how much past state to ignore), respectively. Both are passed through... Activation function normalized to Interval; in the formula , and , These are the weight matrix and bias vector, respectively, automatically optimized through training in the corresponding gating structure. They correspond to the hidden state from the previous time step. (Historical trajectory trend of the torpedo canister) and current moment feature input vectors formed by concatenation A linear transformation is performed to give the neural network the ability to capture dynamic temporal features and suppress sensor noise.

[0036] This design allows the neural network to capture the inertial characteristics and historical trajectory trends of the torpedo canister during its orbital operation, thus making it more robust when dealing with sensor noise.

[0037] Step 3: Input the fused spatiotemporal feature vector of each torpedo canister, the current observation state of each torpedo canister, the environmental state and action set within the operating area of ​​the smart beam yard into the Actor-Critic neural network. Based on the Actor-Critic neural network, introduce an action mask mechanism to output the action with the highest safety and reward function from the action set as the current best action for each node.

[0038] Understandably, each torpedo canister The fused spatiotemporal feature vectors are fed into the Actor-Critic architecture, which outputs the best action for each torpedo canister at the current moment. Figure 8 This is a diagram of the Actor-Critic neural network architecture. Figure 9 The overall control network architecture diagram of the present invention is shown.

[0039] See Figure 8 and Figure 9 The Actor-Critic neural network architecture mainly consists of a policy network (Actor) and a value network (Critic). After being corrected by an "invalid action mask" mechanism, the policy network outputs the optimal control command while ensuring physical safety, while the Critic network outputs the state value estimate. To conduct a comprehensive assessment.

[0040] In the design of the state space, to reduce computational load and improve generalization ability, the entire map image is not directly input; instead, relative coordinate features are used. For each torpedo canister... Its observation status Defined as:

[0041] in torpedo canister Current normalized coordinate position; This indicates the coordinates of the current task target point. In a collaborative delivery scenario, this target can also be the "convergence centroid" dynamically calculated based on the positions of multiple vehicles. Indicates the current torpedo canister The sum of path distances to all other torpedo canisters is used to perceive global congestion through a neural network.

[0042] In terms of the motion space, considering the characteristics of rail transport, the operation of the torpedo canister is discretized. The motion set is defined as follows: , This represents the k-th job in the action set. The specific actions in the action set usually include forward, backward, waiting, and turning instructions at the switch.

[0043] The environmental status within the operating area of ​​the smart beam yard is located as follows: , among which, among which, This represents the set of static obstacles in the smart beam yard at time t. This represents the set of dynamic obstacles in the smart beam yard at time t.

[0044] In one embodiment of the present invention, step 3 specifically includes: Step 31, the Actor policy network, based on each torpedo canister... Given the current observation state and action mask vector, the algorithm outputs the probability distribution of each action in the action set. The action mask vector is determined based on the environmental state within the operating area of ​​the smart beam yard and the fused spatiotemporal feature vector of each torpedo canister. Generate the current road network map structure.

[0045] Understandably, in this invention, the policy network (Actor) introduces an action masking mechanism to output the probability distribution of each action. Specifically, to ensure the absolute safety of the torpedo canister operation, this invention innovatively introduces invalid action masking technology into the PPO algorithm. Figure 10 This is a structural diagram of the PPO algorithm model based on the mask mechanism.

[0046] Before the policy network outputs the action probability, the environmental state within the operating area of ​​the smart beam yard and the fused spatiotemporal feature vector of each torpedo canister are considered. Generate action mask vectors based on the current road network structure. Mask vector The generation is based on real-time topology verification and kinematics pre-simulation. For torpedo canisters... Each candidate action First, combine the current road network structure Fusion of spatiotemporal feature vectors Using the vehicle dynamics model, calculate the predicted position after performing the action. If the action If any of the following physical constraints are triggered, the action is determined to be high-risk and the mask value is set to 1; otherwise, it is determined to be a safe action and its mask value is set to 0. The physical constraints are as follows:

[0047] (1) Risk of going off track: There is no valid adjacent edge (i.e., adjacency matrix) between the target node to which the action is directed and the current node. (or the turnout ahead is not locked to the corresponding direction of travel); (2) Static Collision: Predicted coordinates Falling into a known set of static obstacles Inside the physical enclosure (such as the vehicle stop, maintenance area); (3) Dynamic Conflict: Predicted coordinates and other torpedo canisters within the sensing range Real-time location The Euclidean distance between them is less than the rigid body safety threshold. .

[0048] The above discrimination logic is formalized into the following mask generation function:

[0049] If a certain action This will cause the torpedo canister If it hits an obstacle or derails, =1 The Logit value corresponding to this action is set to negative infinity, that is... .

[0050] The system then calculates the action probability using the Softmax function to ensure that the sampled actions are always physically feasible. Following this, the system will... enter The function (as described in formula (9)) utilizes The mathematical properties of the algorithm force the sampling probability of dangerous actions to zero, while renormalizing the probabilities of the remaining legal actions. This mechanism significantly reduces the trial-and-error time for obstacle avoidance rules during training, enabling the model to converge quickly to a safe path.

[0051]

[0052] in, Indicates candidate actions After introducing the action mask mechanism, in the current observation state The probability distribution, This indicates candidate actions before the introduction of the action mask mechanism. In the current observation state The probability distribution, Candidate actions In the current observation state The mask value.

[0053] The PPO cooperative control algorithm architecture based on action masks corrects the neural network strategy in real time through physical layer constraints. Figure 10 This paper demonstrates the PPO algorithm model structure based on a masking mechanism, clearly depicting the closed-loop process of the agent from environmental perception to safety decision-making. The policy network (Actor Network) on the left side of the figure innovatively embeds an invalid action masking module after feature extraction. This module utilizes real-time sensor and map data to mask the Logits values ​​of dangerous actions, ensuring the accuracy of the action probability distribution generated by the Softmax layer and the final sampled actions. Physical security constraints are always satisfied. The Critic Network works in parallel, providing a global value assessment of the current state through feature extraction and value mapping layer outputs. Both processes store the experiential data (state, action, reward) generated through interaction with the environment into a replay pool. Finally, the PPO algorithm is used to calculate the advantage function and loss, and the strategy and value parameters are updated collaboratively, thereby achieving efficient path planning and control while ensuring operational safety.

[0054] Step 32: Based on the probability distribution of each action, sample the actions in the action set and use the Critic value network to calculate the current torpedo canister. The reward function for executing this sampling action will take the action with the largest reward function as the current torpedo canister. The best move.

[0055] In one embodiment of the present invention, the current torpedo canister... Actions with non-zero probabilities from the action set are sampled, and the current torpedo can is calculated based on the observed state of the current torpedo can. Execute the reward function value for each sampling action; select the sampling action with the highest reward function value as the current torpedo tank. The best move.

[0056] Understandably, step 31 calculates the current torpedo canister. The probability distribution for each action is calculated. Actions with a probability of 0 are considered dangerous and are not collected. Actions with a probability of less than 0 are considered legal. Actions with non-zero probabilities are collected, and the torpedo canister is calculated based on the reward function. Execute the reward function value for each sampling action, and select the action with the largest reward function value as the best action for the torpedo canister.

[0057] While the invalid action masking mechanism successfully addresses the physical safety constraints of torpedo canister operation, achieving efficient material delivery still requires guiding the agent to quickly find the optimal path within a vast state space. To address the slow convergence issue caused by sparse rewards in long-sequence decision-making in reinforcement learning, this invention designs a potential-based dense reward function. This function not only serves as a tool for evaluating the state value of the Critic network. The benchmark also directly participates in the calculation of the advantage function, continuously guiding the torpedo cannon towards the target. Reward function It consists of two parts: the sparse final value reward for completing the task and the potential energy-guided reward for each step, defined as follows:

[0058] in, It is a positive reward for completing a delivery task. It is an instant reward based on the potential energy difference. This represents the observed state of the torpedo canister at time t-1. This represents the observed state of the torpedo canister at time t. It is an instant reward based on potential energy difference, and its calculation formula is as follows:

[0059] In this formula, the potential energy function Defined as the shortest unobstructed path distance from the current position to the target point. Positive feedback is given as the torpedo canister approaches the target (potential energy decreases). Additional negative reward is given when the torpedo canister moves away from the target. To prevent unnecessary detours; a slight negative reward is given when the torpedo canisters remain stationary. To encourage completing tasks quickly, but allow necessary waiting during congestion.

[0060] For the current set of actions of the torpedo canister, firstly, based on the fused spatiotemporal feature vector of the current torpedo canister (which represents the position, velocity, etc. of the torpedo canister), the state of the surrounding obstacle environment, and the observation state (including the position of the vehicle itself, the position of the target, and the positions of other torpedo canisters), it is determined whether the execution of each action in the current set of actions of the torpedo canister is safe, and safe actions are selected. For each safe action, its reward function is calculated, and the action with the largest reward function is selected as the best action for the current torpedo canister at the current moment.

[0061] Step 4: Control each torpedo canister to operate according to the current optimal action.

[0062] Understandably, the optimal action of each torpedo canister in the smart beam yard is determined through the method in step S3, and each torpedo canister is controlled to perform the optimal action, so as to realize the collaborative work of multiple torpedo canisters in the smart beam yard, and thus realize the transportation of materials in the smart beam yard.

[0063] In this invention, a centralized training and distributed execution (CTDE) architecture is adopted, in which all torpedo tanks share the same Actor-Critic neural network model parameters.

[0064] The entire control process is divided into two stages: centralized training and distributed inference. First, the track layout of the intelligent beam yard is simulated in a virtual simulation environment, and random numbers are initialized. The torpedo can intelligent agents are trained by inputting the fused spatiotemporal feature vectors of all torpedo can intelligent agents, the current observation state, the environmental state and action set within the operating area of ​​the smart beam yard into the Actor-Critic neural network.

[0065] After training, the Actor-Critic neural network model is deployed to the edge computing unit of each torpedo tank. During actual operation, it calculates dynamic targets in real time (such as the geometric center calculated based on vehicles that have not completed their missions), and each torpedo tank uses its own sensors to acquire local observations. By using a shared Actor-Critic neural network model, the system can output next action commands in milliseconds. Compared to the traditional enumeration method, this approach significantly improves computational speed and can adapt to large-scale collaborative operations, scaling up from 3 to more than 10 torpedo tanks.

[0066] See Figure 11 This paper illustrates a smart beam yard torpedo canister dynamic collaborative material delivery control system according to an embodiment of the present invention. The system includes: The construction module 1101 is used to construct the operating area of ​​the smart beam yard as a graph structure, and to abstract discrete torpedo canisters as dynamic nodes in the graph structure. Extraction module 1102 is used to extract the fusion spatiotemporal feature vector of each node in the graph structure based on the heterogeneous feature fusion network. The fusion spatiotemporal feature vector is formed by concatenating the feature vector of the node and the context feature vector aggregated by the multi-head attention mechanism. The output module 1103 is used to input the fused spatiotemporal feature vector of each node, the current observation state of each torpedo canister, the environmental state and action set within the operating area of ​​the smart beam yard into the Actor-Critic neural network. Based on the Actor-Critic neural network, an action mask mechanism is introduced to output the action with the highest safety and the largest reward function from the action set as the current best action of each node. Control module 1104 is used to control each torpedo canister to operate according to the current optimal action.

[0067] It is understood that the intelligent beam yard torpedo tank dynamic collaborative material distribution control system provided by the present invention corresponds to the intelligent beam yard torpedo tank dynamic collaborative material distribution control method provided in the foregoing embodiments. The relevant technical features of the intelligent beam yard torpedo tank dynamic collaborative material distribution control system can be referred to the relevant technical features of the intelligent beam yard torpedo tank dynamic collaborative material distribution control method, and will not be repeated here.

[0068] The present invention provides a method and system for dynamic collaborative material delivery control of torpedo containers in a smart beam yard, which has the following advantages compared with existing control methods: (1) The scheduling decision is more global: Compared with the traditional scheduling mode based on experience rules or single operation logic, the method of this invention uses deep reinforcement learning (PPO algorithm) to construct a global material distribution strategy, which can perceive the location of multiple torpedo canisters, the urgency of tasks and track occupancy in real time, and realize the dynamic optimal allocation of cross-regional and multi-task, which significantly improves the overall logistics turnover efficiency of the smart beam plant.

[0069] (2) More mandatory safety obstacle avoidance: Unlike traditional methods that rely solely on penalty functions to avoid collisions, the method of this invention judges the safety of each action at the underlying logic level and blocks actions that are not safe. This ensures that the path selection of the torpedo can always meet physical safety constraints in complex dynamic environments, and completely eliminates safety accidents that may be caused by algorithm trial and error.

[0070] (3) More real-time environmental response: The real-time inference mechanism based on the Actor-Critic architecture is adopted, which can provide millisecond-level feedback on the rapidly changing state information in the beam yard (such as equipment failure, temporary shutdown, etc.). Compared with heuristic search algorithms that require long-term recalculation, the method of this invention achieves real-time scheduling through a pre-trained policy network, ensuring seamless connection between material supply and production demand and reducing system waiting time.

[0071] (4) More efficient model convergence: The advantage function estimation is designed, which greatly reduces the search space of reinforcement learning and filters out a large number of invalid samples. This enables the algorithm to achieve higher convergence accuracy with fewer training iterations when dealing with large-scale track networks and multi-agent collaborative problems, effectively solving the problems of difficult training and placement of traditional deep learning methods in complex industrial scenarios.

[0072] It should be noted that the descriptions of each embodiment in the above embodiments have different focuses. For parts that are not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0073] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0074] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0075] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0076] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0077] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention.

[0078] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A method for dynamic collaborative material delivery control of multiple torpedo-shaped containers in a smart beam yard, characterized in that, include: The operating area of ​​the smart beam yard is abstracted as a graph structure, and the discrete torpedo canisters are abstracted as dynamic nodes in the graph structure. Based on the heterogeneous feature fusion network, the fused spatiotemporal feature vector of each torpedo tank in the graph structure is extracted. The fused spatiotemporal feature vector is formed by concatenating the vehicle feature vector and the context feature vector aggregated by the multi-head attention mechanism. The fused spatiotemporal feature vector of each torpedo canister, the current observation state of each torpedo canister, the environmental state and action set within the operating area of ​​the smart beam yard are input into the Actor-Critic neural network. Based on the Actor-Critic neural network, an action mask mechanism is introduced to output the action with the highest safety and reward function from the action set, which is taken as the current best action for each torpedo canister. Control each torpedo canister to operate according to the current optimal action.

2. The intelligent beam yard multi-torpedo tank dynamic collaborative material delivery control method according to claim 1, characterized in that, The heterogeneous feature fusion network includes a feature extraction layer, a linear embedding layer, a multi-head graph attention layer, and a feature fusion layer; The extraction of the fused spatiotemporal feature vector for each torpedo canister in the graph structure based on the heterogeneous feature fusion network includes: Each torpedo canister is extracted using the feature extraction layer. The vehicle's feature vector ,in, torpedo canister The current normalized position, torpedo canister The current instantaneous velocity , torpedo canister The current load status; The feature vector of this vehicle The linear embedding layer maps the data to high-dimensional latent features. ; With current torpedo canisters Centered on this, other torpedo canisters within its sensing range Treating them as adjacent nodes, calculate their relative state characteristics. ; Current torpedo can High-dimensional latent features The relative state characteristics of the nodes and their neighbors The input is fed into a multi-head graph attention layer, and the output is the current torpedo tank. Contextual features aggregated with adjacent nodes ; The current torpedo can is obtained through the feature fusion layer. High-dimensional latent features and aggregated context features Nonlinear splicing is performed, and the data is processed by a gated recurrent unit (GRU) to obtain the current torpedo canister. Fusion spatiotemporal feature vectors .

3. The intelligent beam yard multi-torpedo can dynamic collaborative material delivery control method according to claim 2, characterized in that, The high-dimensional latent features Represented as: ； in, The activation function of the linear embedding layer The linear coefficients of the linear embedding layer are, is the bias coefficient of the linear embedding layer.

4. The intelligent beam yard multi-torpedo tank dynamic collaborative material delivery control method according to claim 2, characterized in that, The current torpedo can High-dimensional latent features The relative state characteristics of the nodes and their neighbors The input is fed into a multi-head graph attention layer, and the output is the current torpedo tank. Contextual features aggregated with adjacent nodes ,include: Calculate relative state characteristics Attention score for each attentional head : ； in, Representing relative state characteristics The attention score of the k-th attention head. For activation function, Let be the weight matrix of the k-th attention head. Let T represent the learnable parameter vector of the k-th attention head, and T denote the transpose. Attention score using the Softmax function After normalization, the coefficients under the k-th attention head are obtained. : ； in, Indicates the current torpedo canister With the mth adjacent torpedo canister Relative state characteristics Attention score for the k-th attention head; Based on relative state characteristics The coefficient under each attention head Aggregate to obtain the current torpedo canister Contextual features aggregated with adjacent nodes : ； Current torpedo can High-dimensional latent features and current torpedo canisters Contextual features aggregated with adjacent nodes Nonlinear splicing is performed, and the data is processed by a gated loop unit (GRU) with time memory function to obtain the current torpedo canister. Fusion spatiotemporal feature vectors .

5. The intelligent beam yard multi-torpedo tank dynamic collaborative material delivery control method according to claim 1, characterized in that, The process involves inputting the fused spatiotemporal feature vector of each node, the current observation state of each torpedo canister, the environmental state within the operating area of ​​the smart beam yard, and the action set into the Actor-Critic neural network. Based on the Actor-Critic neural network, an action mask mechanism is introduced to output the action with the highest safety and reward function from the action set as the current optimal action for each torpedo canister, including: The Actor strategy network is based on each torpedo canister. Given the current observation state and action mask vector, the algorithm outputs the probability distribution of each action in the action set. The action mask vector is determined based on the environmental state within the operating area of ​​the smart beam yard and the fused spatiotemporal feature vector of each torpedo canister. And the current road network map structure is generated; Based on the probability distribution of each action, actions in the action set are sampled, and the current torpedo can is calculated using the Critic value network. The reward function for executing this sampling action will take the action with the largest reward function as the current torpedo canister. The best move.

6. The intelligent beam yard multi-torpedo tank dynamic collaborative material delivery control method according to claim 5, characterized in that, Based on the environmental conditions within the operating area of ​​the smart beam yard and the fused spatiotemporal feature vector of each torpedo canister... Generate action mask vectors based on the current road network structure, including: The environmental state within the operating area of ​​the smart beam yard is defined as follows: ,in, This represents the set of static obstacles in the smart beam yard at time t. Represents the set of dynamic obstacles in the intelligent beam yard at time t; For torpedo cans Each candidate action Combined with the current road network structure The fused spatiotemporal feature vector of each torpedo canister And vehicle dynamics model, calculate torpedo canister Execute the candidate action Predicted position after ; Based on predicted location Determine candidate actions If any of the following physical constraints are triggered, the action is classified as high-risk and a mask value of 1 is set. These physical constraints include derailment risk, static collision, and action conflict. Otherwise, a candidate action is set. The mask value is 0; The mask value for each action in the action set is set to obtain the action mask vector.

7. The intelligent beam yard multi-torpedo tank dynamic collaborative material delivery control method according to claim 6, characterized in that, The Actor strategy network is based on each torpedo can. Given the current observation state and action mask vector, output the probability distribution of each action in the action set, including: in, Indicates candidate actions After introducing the action mask mechanism, in the current observation state The probability distribution, This indicates candidate actions before the introduction of the action mask mechanism. In the current observation state The probability distribution, Candidate actions In the current observation state The mask value.

8. The intelligent beam yard multi-torpedo tank dynamic collaborative material delivery control method according to claim 7, characterized in that, Based on the probability distribution of each action, actions in the action set are sampled, and the current torpedo can is calculated using the Critic value network. The reward function value of the sampling action is used to determine the current torpedo tank. The best moves include: For the current torpedo canister Actions with non-zero probabilities from the action set are sampled, and the current torpedo can is calculated based on the observed state of the current torpedo can. The reward function value for each sampling action; The sampling action with the largest reward function value is taken as the current torpedo container. The best move; Among them, the calculation of the current torpedo canister The reward function value for each sampling action includes: The reward function includes a sparse reward for completing the task and a potential-guided reward for each step, defined as follows: ； in, It is a positive reward for completing a delivery task. It is an instant reward based on the potential energy difference. This represents the observed state of the torpedo canister at time t-1. The observed state of the torpedo canister at time t is represented by the following formula: ； In the formula, the potential energy function Defined as the shortest unobstructed path distance from the current position to the target point, positive feedback is given as the torpedo canister approaches the target position. Additional negative reward is given when the torpedo canister moves away from the target location. A slight negative reward is given when the torpedo canister remains stationary. .

9. The intelligent beam yard multi-torpedo tank dynamic collaborative material delivery control method according to claim 1, characterized in that, Also includes: Simulate the track layout of the smart beam yard in a virtual simulation environment and initialize a random number of torpedo canister smart agents; The shared Actor-Critic neural network is deployed on a central server. The fused spatiotemporal feature vector of each torpedo tank agent, the current observation state, the environmental state and action set within the operating area of ​​the smart beam yard are input into the Actor-Critic neural network to train it. The trained Actor-Critic neural network is deployed to the edge computing unit of each torpedo tank; In actual operation, dynamic targets are calculated in real time. The current observation state of the torpedo canister is input into the Actor-Critic neural network, which outputs the best action of the torpedo canister at the current moment.

10. A smart beam yard multi-torpedo can dynamic collaborative material delivery control system, characterized in that, include: The construction module is used to construct the operating area of ​​the smart beam yard as a graph structure, and to abstract discrete torpedo canisters as dynamic nodes in the graph structure. The extraction module is used to extract the fused spatiotemporal feature vector of each torpedo tank in the graph structure based on the heterogeneous feature fusion network. The fused spatiotemporal feature vector is formed by concatenating the vehicle feature vector and the context feature vector aggregated by the multi-head attention mechanism. The output module is used to input the fused spatiotemporal feature vector of each torpedo can, the current observation state of each torpedo can, the environmental state and action set within the operating area of ​​the smart beam yard into the Actor-Critic neural network. Based on the Actor-Critic neural network, an action mask mechanism is introduced to output the action with the highest safety and the highest reward function from the action set, which is the current best action for each torpedo can. The control module is used to control each torpedo canister to operate according to the current optimal action.

Images