A port internal cargo distribution monitoring system and method

Abstract

The application discloses a kind of port interior goods transport monitoring method and method, this method constructs port digital twin base, and abstracts physical entity as multi-agent network;Based on the initial plan of global resource collaborative transport generated by deep reinforcement learning model;Real-time monitoring actual transport process, identify the sudden interference event between twin base and physical space;When monitoring interference event, trigger multi-agent collaborative negotiation mechanism, and the affected agent carries out local re-bidding based on contract net protocol, dynamically allocates tasks and issues local adjustment instructions.The application decouples the complex global rearrangement into local negotiation, effectively avoids the combination explosion and recalculation delay problem easily caused by traditional centralized scheduling when facing sudden abnormality, realizes the second-level self-healing of transport exception, greatly improves the robustness and real-time execution efficiency of port scheduling monitoring system.

Description

Technical Field

[0001] This invention relates to the field of smart port and logistics monitoring technology, specifically to a cargo transfer monitoring system and method within a port area. Background Technology

[0002] Currently, it is an important hub in the modern international logistics chain. With the in-depth advancement of smart port construction, the automated monitoring and scheduling system for cargo transportation within the port area (covering berths, storage yards, horizontal transport vehicles, and flow equipment) has become a core component for improving the overall operational efficiency of the port.

[0003] Currently, most patented technologies for port cargo handling monitoring and scheduling on the market adopt a centralized computing architecture. Specifically, a central dispatch server issues collection instructions to acquire real-time location and status data of all equipment and vehicles within the port area. Then, traditional operations research algorithms (such as mixed integer programming and genetic algorithms) are used to solve the problem centrally, ultimately generating global handling instructions and planned routes to achieve on-site monitoring and control.

[0004] However, traditional centralized technology solutions have inherent limitations in the complex real-world application of port facilities:

[0005] The computational logic is rigid and prone to "combinatorial explosion": Port operations operate in a high-frequency, dynamic environment, constantly facing unexpected disruptions such as sudden equipment failures, temporary road closures, and vehicle delays. When these anomalies occur, the existing central dispatch server must recalculate all variables across the entire port area using a massive mathematical model. As the scale of port equipment increases, this interconnected global recalculation easily triggers a "combinatorial explosion," causing excessively long times (usually several minutes or even longer) for the system to regenerate dispatch instructions. This fails to meet the millisecond or second-level real-time anomaly response requirements of port operations, resulting in prolonged equipment downtime and significantly reducing the robustness and efficiency of the monitoring and dispatch system.

[0006] Therefore, how to break through the limitations of centralized scheduling while maintaining overall resource planning, and achieve efficient local reordering and second-level self-healing in the face of sudden interference, is a technical problem that urgently needs to be solved in this field. Summary of the Invention

[0007] In order to overcome the shortcomings and deficiencies of the existing technology, the present invention aims to provide a cargo transfer monitoring system and method within a port area. This solution combines the distributed collaborative capabilities of a multi-agent architecture with the global planning capabilities of deep reinforcement learning, thereby achieving extremely high rapid readjustment efficiency and system robustness in the event of sudden anomalies.

[0008] This invention is achieved through the following technical solution: In a first aspect, the present invention discloses a method for monitoring cargo transfer within a port area, the method comprising the following steps: S1. Construct a digital twin foundation for the port area and abstract the physical entities within the port area into a multi-agent network with independent perception and decision-making capabilities; S2. Obtain real-time cargo dispatch tasks and generate an initial plan for global resource collaborative dispatch based on a pre-trained deep reinforcement learning model; S3. Real-time monitoring of the actual transportation process in the port area to identify sudden interference events between the digital twin base and the physical space; S4. When the aforementioned sudden interference event is detected, a multi-agent collaborative negotiation mechanism is triggered. Each affected agent re-bids based on the Contract Network protocol, dynamically allocates tasks to generate local adjustment instructions, and issues them for execution.

[0009] In conjunction with the first aspect, further, step S1 specifically includes: S11. Real-time multi-dimensional physical status data, including berths, storage yards, flow equipment and transport vehicles, are acquired through multi-source IoT sensors deployed in the port area; S12. Based on the multidimensional physical state data, a real-time synchronously updated digital twin base of the port area is generated by mapping in the virtual space; S13. In the digital twin base, each berth, flow machine equipment and transport vehicle is encapsulated as an independent intelligent agent, and an observation state space, behavior action space and local reward function are defined for each intelligent agent; S14. Connect all defined agents through topological relationships and output an initialized multi-agent network for subsequent dispatch planning and monitoring.

[0010] In conjunction with the first aspect, further, step S2 specifically includes: S21. Extract the feature vector of the real-time cargo dispatching task to be executed, and the global state matrix of all agents in the current multi-agent network; S22. Input the feature vector and global state matrix into a pre-trained deep reinforcement learning decision network; S23. The deep reinforcement learning decision network is used to predict the Q value of each agent taking different scheduling actions in the current state, and a greedy strategy is used to select the action combination that maximizes the global expected cumulative reward. S24. Decode the action combination into specific task timing and running path for each agent, and output the timing and path as the initial plan.

[0011] In conjunction with the first aspect, further, step S3 specifically includes: S31. Extract the expected execution trajectory and expected operation progress from the digital twin base according to the initial plan according to the preset sampling period; S32. Synchronously acquire the actual operating trajectory and actual work progress of the on-site physical entities; S33. Calculate the spatial deviation between the expected execution trajectory and the actual running trajectory, and the time deviation between the expected work progress and the actual work progress; S34. Determine whether the spatial deviation value or the time deviation value exceeds the preset dynamic adaptive tolerance threshold; S35. If the threshold is exceeded, the abnormal source entity that caused the deviation and the affected range are extracted based on the coordinate location and timestamp of the deviation, and an interference event alarm report containing the ID of the affected intelligent agent is generated and output.

[0012] In conjunction with the first aspect, further, step S4 specifically includes: S41. When the interference event alarm report is received, the intelligent agent that is directly affected by the sudden interference event and is unable to continue to perform the original task shall act as the bidding party and broadcast the task bidding information to idle or low-load intelligent agents within its communication range. S42. The intelligent agent that receives the task bidding information acts as the bidder, calculates the execution cost of the task based on its current geographical location, remaining rated load and energy consumption status, and returns a bid containing the execution cost to the bidding party. S43. The tendering party collects all received bids, selects the optimal bidder agent based on the principle of minimizing execution cost, and sends the winning bid confirmation information to the optimal bidder agent. S44. After receiving the confirmation information, the winning agent updates its local task queue and generates a local adjustment instruction based on the updated task queue, which is then output to the corresponding physical entity for execution, thereby completing the local rearrangement.

[0013] In conjunction with the first aspect, further, after step S4, the method also includes step S5: S51. The generated local adjustment command is sent to the corresponding physical entity for execution, and the local state nodes in the digital twin base are updated in real time according to the execution feedback; S52. After the sudden interference event is handled, extract the process data of this multi-agent collaborative negotiation and the actual time taken to finally eliminate the anomaly; S53. Combine the process data and actual time consumption into a state-action-reward tuple, and store it in the model experience pool as experience replay data for reinforcement learning. S54. When the data in the model's experience pool reaches a preset batch size, the network parameters of the deep reinforcement learning model are updated offline using the policy gradient descent algorithm, and the evolved deep reinforcement learning model is output.

[0014] Secondly, the present invention discloses a monitoring system for cargo transportation within a port area, the monitoring system being used to execute the method described above, the monitoring system comprising: The digital twin and multi-agent initialization module is used to build the digital twin foundation of the port area and abstract the physical entities in the port area into a multi-agent network with independent perception and decision-making capabilities. The global plan generation module is used to obtain real-time cargo dispatching tasks and generate an initial plan for global resource collaborative dispatching based on a pre-trained deep reinforcement learning model. The real-time monitoring and anomaly identification module is used to monitor the port area's transportation process in real time and identify sudden interference events between the digital twin base and the physical space. The multi-agent collaborative negotiation module is used to trigger the multi-agent collaborative negotiation mechanism when a sudden interference event is detected. Each affected agent re-bids based on the contract network protocol, dynamically allocates tasks to generate local adjustment instructions and issues them for execution.

[0015] In conjunction with the second aspect, the multi-agent collaborative negotiation module further includes: The bidding broadcast subunit is configured so that the intelligent agent whose task is interrupted due to a sudden interference event acts as the bidding party, generates task bidding information and multicasts it to surrounding intelligent agents; The cost calculation and bidding subunit is configured such that after the bidding agent receives the bidding information, it calculates the execution cost based on its own state and submits a bid containing the execution cost. The evaluation and bid confirmation subunit is configured so that the bidding party selects the optimal bidder based on the principle of minimizing execution cost and issues bid confirmation information, triggering the winning agent to generate the local adjustment instruction.

[0016] Thirdly, the present invention discloses an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, the processor running the computer program to implement the steps of the monitoring method as described in any of the preceding claims.

[0017] Fourthly, the present invention discloses a computer-readable storage medium having a computer program stored thereon, the program being executed by a processor to implement the steps of the monitoring method as described in any of the preceding claims.

[0018] The beneficial effects of this invention are: This invention discloses a port area cargo transportation monitoring system and method. By constructing a digital twin foundation containing multiple agents and triggering a collaborative negotiation mechanism based on the contract network protocol when a sudden interference event is detected, it completely solves the problems of combinatorial explosion and excessively long recalculation time in traditional centralized monitoring and scheduling systems under sudden anomalies. It reduces and decouples the complex global rescheduling problem into local re-bidding among affected agents, generating local adjustment instructions without global recalculation. It achieves rapid rescheduling and self-healing in response to sudden interference, greatly improving the real-time performance, robustness, and execution efficiency of the port area cargo transportation monitoring system. Attached Figure Description

[0019] The present invention will be further described with reference to the accompanying drawings, but the embodiments in the drawings do not constitute any limitation on the present invention. For those skilled in the art, other drawings can be obtained based on the following drawings without creative effort.

[0020] Figure 1 This is a flowchart illustrating a method for monitoring cargo transportation within a port area, as provided in an embodiment of the present invention.

[0021] Figure 2 This is a block diagram of a monitoring system for cargo transportation within a port area, provided in an embodiment of the present invention.

[0022] Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention.

[0023] Figure 4 This is a schematic diagram of a storage medium provided in an embodiment of the present invention. Detailed Implementation

[0024] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Many specific details are set forth in the following description to provide a thorough understanding of the present invention. However, the present invention can be practiced in many other ways different from those described herein, and those skilled in the art can make similar modifications without departing from the spirit of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0025] Example 1 like Figure 1 As shown in the figure, this embodiment discloses a method for monitoring cargo transportation within a port area, which includes the following steps: S1. Construct a digital twin foundation for the port area and abstract the physical entities within the port area into a multi-agent network with independent perception and decision-making capabilities; S2. Obtain real-time cargo dispatch tasks and generate an initial plan for global resource collaborative dispatch based on a pre-trained deep reinforcement learning model; S3. Real-time monitoring of the actual transportation process in the port area to identify sudden interference events between the digital twin base and the physical space; S4. When the aforementioned sudden interference event is detected, a multi-agent collaborative negotiation mechanism is triggered. Each affected agent re-bids based on the Contract Network protocol, dynamically allocates tasks to generate local adjustment instructions, and issues them for execution.

[0026] Specifically, in step S1, high-precision 3D modeling technology (such as oblique photogrammetry and Unity3D engine parametric modeling) is used to construct a static digital base for the port area's shoreline, storage yard, and silos. Simultaneously, the coordinates, operating trajectories, and working parameters of physical equipment are collected in real time through a 5G private network deployed within the port area and multi-source IoT sensors (including BeiDou high-precision positioning terminals, LiDAR, PLC status acquisition devices, etc.), achieving millisecond-level mapping from physical space to digital space. Secondly, multi-agent abstraction configuration is performed, encapsulating physical entities within the port area, such as gantry cranes, horizontal transport trucks, and storage yard forklifts, into independent agents at the algorithm's underlying layer. Each agent is configured with three core modules: The status awareness module is responsible for receiving the real-time location coordinates of the current device. Load condition, remaining oil / electricity level, and equipment health status, etc. The communication module, based on a publish-subscribe model, interacts with other intelligent agents and the central control platform. The decision-making module, with its built-in lightweight inference engine, supports autonomous decision-making based on rules or fine-tuning models. Through the above encapsulation and topological connections, a distributed multi-agent network is formed.

[0027] In step S2, this embodiment employs the Multi-Agent Deep Deterministic Policy Gradient (MADDPG) algorithm as the deep reinforcement learning model. The system's global state space... It is defined as a splicing matrix of all task features (start point, end point, cargo attributes) to be assigned within the port area and the state of each agent. The action space A is defined as the set of action vectors of each agent choosing to accept a specific task or move to a specific area within the current time step.

[0028] To guide the model in generating an optimized scheduling plan, an immediate reward function is used. Taking into account a balance between operation time, energy cost, and equipment utilization, its mathematical expression is specifically defined as:

[0029] in, The total time taken to complete the task. This is the predicted total energy consumption value. The variance of equipment utilization (the smaller the variance, the higher the reward, which promotes a more balanced task allocation). These are the preset weighting coefficients.

[0030] After the cargo transportation task is input into the model, the pre-trained Actor network outputs a combination of actions based on the global state, decodes it into a specific operation sequence and physical operation path, and issues it as an initial plan.

[0031] In step S3, during operation, the system monitoring engine extracts the expected trajectory sequence and expected progress benchmark derived from the digital twin base according to the initial plan at a set sampling frequency (e.g., 10 times per second), and simultaneously acquires the actual physical space trajectory sequence and actual progress transmitted back by the IoT sensors. The system introduces a spatial tolerance threshold. and time tolerance threshold Calculate the deviation between the expected parameters and the actual parameters. When the actual position deviation of a certain device... Or the work time is delayed. If the duration exceeds a preset period (e.g., 30 seconds), the system's underlying layer will trigger an interrupt check.

[0032] Typical sudden interference events include: machine downtime caused by mechanical failure of conveyor equipment, sudden path congestion in loading and unloading areas, or local speed restrictions caused by severe weather. Once identified, the system immediately locks onto the abnormal source agent causing the deviation and generates an interference event alarm message.

[0033] In step S4, to avoid combinatorial explosion caused by a full recalculation, this step utilizes the Contract Net Protocol (CNP) to dynamically reallocate resources within the affected local area, specifically in four stages: Call for Proposals (CFP): A faulty or obstructed agent (the procurer) voluntarily suspends its current task and sends a notification to its communication radius. Idle or low-load agents (bidders) within the system broadcast bidding messages containing task type, target coordinates, and deadline.

[0034] Bidding: Upon receiving the CFP, the agent invokes the built-in local cost evaluation function to calculate the cost of undertaking the task. The cost calculation formula is as follows:

[0035] in, For intelligent agents The cost of the bid, This represents the distance the agent travels from its current position to the task breakpoint. This represents the current load level of the agent. and This is the penalty factor. After calculation, it will include the cost. The tender documents are returned to the tendering party.

[0036] Bid evaluation and award: After waiting for a preset time window (e.g., 200 milliseconds), the bidding party summarizes all bids, selects the optimal agent based on the principle of minimizing local costs, and sends the award confirmation information to it.

[0037] Command issuance and execution: The winning intelligent agent inserts the new task into its own high-priority task queue, and then the system's control interface (such as the API service that interfaces with the BTOS system) issues the generated local adjustment command to the winning physical entity (such as the PLC of the idle truck) for execution.

[0038] It should be noted that this embodiment confirms the effectiveness of the policy network by observing the cumulative reward convergence curve of the deep reinforcement learning model during the training phase. After 10,000 training rounds, the loss function stably converges to below 0.01, and the reward value tends to plateau, qualitatively confirming that the model structure can effectively find the optimal scheduling solution.

[0039] In a digital twin simulation environment, 15% random device fault interference was artificially injected. Test data shows that the traditional centralized integer programming algorithm takes an average of 135 seconds to resolve after interference; while the multi-agent CNP collaborative negotiation mechanism described in this embodiment of the invention takes only 85 milliseconds on average from interference identification to the generation and issuance of local adjustment instructions. Qualitatively, this demonstrates that the proposed solution can effectively avoid combinatorial explosion, and quantitatively, it demonstrates that the system can achieve rapid self-healing in the event of sudden anomalies.

[0040] Furthermore, step S1 in this embodiment specifically includes: S11. Real-time multi-dimensional physical status data, including berths, storage yards, flow equipment and transport vehicles, are acquired through multi-source IoT sensors deployed in the port area; S12. Based on the multidimensional physical state data, a real-time synchronously updated digital twin base of the port area is generated by mapping in the virtual space; S13. In the digital twin base, each berth, flow machine equipment and transport vehicle is encapsulated as an independent intelligent agent, and an observation state space, behavior action space and local reward function are defined for each intelligent agent; S14. Connect all defined agents through topological relationships and output an initialized multi-agent network for subsequent dispatch planning and monitoring.

[0041] Specifically, step S11 realizes the acquisition of multi-source IoT spatiotemporal data: Within the port area's physical space, multi-source IoT sensors deployed at various operational nodes collect physical status data in real time. Specifically, transport vehicles and conveyor equipment are equipped with BeiDou RTK high-precision differential positioning terminals (which acquire centimeter-level three-dimensional coordinates). It includes a heading angle sensor, a load / lifting digital sensor (to acquire instantaneous load), and a PLC status acquisition module (to acquire operating parameters such as instantaneous motor current and torque). Data is transmitted via the port area's 5G NSA private network. The high-frequency sampling rate is uploaded to the edge computing node in real time.

[0042] Step S12 implements the mapping of the port area's digital twin base: Based on the aforementioned multidimensional physical state data, the physical entity state is mapped to the virtual space in real time using a 3D rendering engine and coordinate transformation matrix on the server side. The system's underlying layer employs kinematic Kalman filtering to smooth and interpolate dropped frame data caused by network jitter, ensuring the temporal continuity and spatial fidelity of the trajectory sequences of each device in the digital twin base.

[0043] Step S13 implements the mathematical encapsulation of the agent's state space, action space, and local reward function: In the digital twin foundation, each berth, gantry crane, and transport vehicle is encapsulated as an independent intelligent agent. To address the technical pain points of slow model convergence and high equipment wear caused by the even distribution of rewards in multi-device collaboration scenarios (such as gantry crane grabbing + truck transportation) in deep reinforcement learning, this embodiment defines a local reward function. At that time, a dynamic contribution attribution algorithm based on Shapley value and a nonlinear kinematic penalty function were introduced.

[0044] intelligent agent The local reward function at time t Specifically defined as:

[0045] Where N represents the entire set of cooperative agents currently participating in the same work process (such as the unloading task of a specific cargo ship), and the parameters are... For set The total number of agents in the system, typically ranging from 1 to 2. ; For set Remove intelligent agents Any subset thereof; For subset collaboration alliance The task efficiency characteristic function represents the actual throughput tonnage of the alliance per unit time. This data is obtained by calling the real-time cargo counting interface of the BTOS system. For intelligent agents In micro-element time The instantaneous motor output power is obtained through the inverter parameter interface of the PLC at the bottom of the equipment; For intelligent agents The instantaneous rate of change of acceleration (Jerk) characterizes the degree of mechanical shock and wear during equipment operation. This value is obtained by analyzing the velocity components acquired from RTK-GPS or IMU. Obtained by performing second-order difference calculations; The weighting coefficient for the penalty term. Used to constrain energy consumption Used to restrain mechanical impact, its specific value is determined by the port area equipment operation and maintenance department based on historical overhaul cost data.

[0046] It should be noted that the first part of the above formula comes from the Shapley Value model in cooperative game theory, which has been rigorously proven mathematically to guarantee the uniqueness and fairness of the distribution of benefits in a multi-participant alliance. Introducing it into reward shaping in deep reinforcement learning allows for the accurate calculation of the marginal contribution of a single device to the overall throughput improvement of the production line. The second part of the formula is a nonlinear kinematic penalty term, where the integral of the square of the jerk is widely recognized in modern mechanical control theory as being directly positively correlated with the fatigue wear of the equipment's drive shaft and the impact of the gearbox.

[0047] Simulation verification using historical port data shows that, compared to the traditional equal-reward strategy, the convergence speed of the reinforcement learning model using this higher-order reward algorithm is significantly improved. Meanwhile, because the penalty is directly related to mechanical wear, the model automatically avoids short-sighted behaviors such as sudden starts and stops when outputting work instructions. Field measurements show that this extends the lifespan of the wire rope and transmission components of the conveyor equipment by approximately [percentage missing]. .

[0048] In addition, this local reward function The calculations fall under the category of online real-time control. This system not only relies on a central server, but also equips large-scale operating equipment in the port area (such as gantry cranes and intelligent container trucks) with industrial-grade edge computing gateways (such as the NVIDIA Jetson series) with independent NPU computing power. Each intelligent agent conducts low-latency peer-to-peer communication at the microsecond level through a 5G private network, and uses its own mounted edge computing gateway to perform distributed real-time calculations of the above integral and factorial operations in each control cycle, thereby generating local reward feedback to meet the requirements of high computing power and real-time performance.

[0049] After completing the above configuration, the system executes step S14, multi-agent network topology initialization, constructing a dynamic directed graph structure based on the spatial Euclidean distance and task coupling degree between agents. All defined agent nodes are connected through dynamic topology relationships, outputting the initialized multi-agent network, which is then handed over to the subsequent deep reinforcement learning decision network for unified global scheduling and local re-bidding.

[0050] Furthermore, step S2 in this embodiment specifically includes: S21. Extract the feature vector of the real-time cargo dispatching task to be executed, and the global state matrix of all agents in the current multi-agent network; S22. Input the feature vector and global state matrix into a pre-trained deep reinforcement learning decision network; S23. The deep reinforcement learning decision network is used to predict the Q value of each agent taking different scheduling actions in the current state, and a greedy strategy is used to select the action combination that maximizes the global expected cumulative reward. S24. Decode the action combination into specific task timing and running path for each agent, and output the timing and path as the initial plan.

[0051] The above steps introduce a monotonic value function decomposition algorithm based on hypernetworks and a dynamic exploratory annealing strategy. In the path time sequence decoding stage, a high-order fifth-order polynomial motion student function considering lateral centrifugal force constraints is designed, which effectively solves the global exploration and anti-tipping safety problems of high center of gravity bulk cargo transportation equipment in complex port dynamic coordination.

[0052] Specifically, steps S21 to S24 of this embodiment are mainly calculated and executed in real time or near real time by the concurrent computing server cluster of the port central dispatch cloud control center.

[0053] Step S21 involves extracting features of the transportation task and constructing the global state matrix.

[0054] When the central dispatch system receives a real-time task pool pushed by a port area business system (such as cloud-based BTOS), the core scheduling engine first performs feature vectorization processing. For the first... Extract the spatiotemporal feature vectors of the following tasks to be executed:

[0055] in, Quantify and code bulk goods categories. For a single shipment weight, parameters such as location, rated load, and current work group status from all participating agents in the current multi-agent network are extracted in real time, normalized, and concatenated. The output dimension is... global state matrix .

[0056] Steps S22 and S23 involve global joint Q-value calculation and greedy decision-making based on a nonlinear monotonic hybrid network.

[0057] In multi-agent collaborative operations (such as quay crane unloading - flow machine stacking - truck delivery), if a single agent only pursues the shortest possible local operation time, deadlocks can easily occur at port intersections. To achieve global collaboration, the pre-trained deep reinforcement learning decision network in this embodiment adopts a monotonic hybrid network architecture.

[0058] Specifically, the global state matrix The value of individual actions output by each agent's local network Input into a hybrid network, global joint expected cumulative reward value The solution formula is defined as follows:

[0059] The system obtains the solution After the value is determined, a dynamic annealing greedy strategy based on Boltzmann distribution is used to select the action combination. Its selection probability function is defined as:

[0060] Where N is the total number of agents currently participating in the collaborative task, and its value is usually between 5 and 50; For the first Each agent is based on its own local observations Take scheduling action independence value; On the grounds of The two-layer nonlinear weight matrix generated for the input hypernetwork is given by taking its absolute value in the formula. To strictly satisfy the monotonicity constraint ; For the first The adaptive decay formula for the exploration temperature parameter at each global scheduling decision step is: initial temperature The value ranges from 1.0 to 5.0, and the attenuation rate is... It is usually calibrated to 0.001 to 0.01.

[0061] It should be noted that the absolute value monotonicity constraint is mathematically rigorously guaranteed. With each The equivalence of these parameters allows the central control console to perform globally optimal combinations of actions by distributively retrieving the actions with the highest Q values ​​from each individual agent during online decision-making, thus avoiding the need to navigate through a vast joint action space. An exhaustive search is then performed.

[0062] In actual production at bulk cargo terminals, when the gantry crane grabs and unloads material extremely quickly (in some areas) (Extremely high), but when there is congestion due to stacking at the rear conveyor belt or aisle storage yard, traditional operations research algorithms will force the gantry crane to continue feeding at high speed, leading to severe blockage. This hybrid network, however, uses weights generated by a hypernetwork... It integrates the overall system status and automatically reduces the gate operator's latency under high-concurrency congestion conditions. The value weight drives the gantry crane to actively reduce speed or put the equipment on standby, achieving a balance between green and low-carbon development and overall line flow.

[0063] Through simulation and comparison of high-frequency operations in the port area, the annealed Boltzmann QMIX decision network, compared with the standard... The greedy strategy reduced the rate of vehicle deadlocks caused by conflicting transport orders. .

[0064] Step S24 is path timing decoding based on a high-order fifth-degree polynomial and centrifugal force constraints.

[0065] Decoding Action Combination After obtaining the task sequence, it needs to be converted into a continuous temporal trajectory that can be physically executed by the transportation equipment. Bulk cargo (such as grain, fertilizer, and iron ore) is fluidized in the truck bed, and the vehicle is highly susceptible to overturning due to lateral centrifugal force disturbance when turning at a high center of gravity. Therefore, this embodiment designs a high-order fifth-degree polynomial temporal decoding function with lateral dynamic constraints.

[0066] intelligent agent In the allocated time window Inside, its two-dimensional physical control path The generating function is defined as:

[0067] Furthermore, the solution process of the above formula is subject to strict safety constraints on the vehicle's lateral centrifugal acceleration:

[0068] in, The coefficient vector of the two-dimensional polynomial trajectory is solved by substituting the boundary conditions (the starting point position, velocity, and acceleration are the actual starting state; the ending point position is the mission target point, and the velocity and acceleration are set to zero) and then solving the simultaneous derivatives. path coordinates to time The first derivative (instantaneous velocity component) and the second derivative (instantaneous acceleration component); The maximum permissible lateral centrifugal acceleration threshold for transport equipment is typically calibrated to be 1.2 to 2.5 m / s² for cargo trailers in port areas. This data is obtained from static rollover critical angle experiments conducted under full vehicle load conditions.

[0069] Specifically, the first term on the left side of the above inequality is the instantaneous curvature formula in calculus geometry. The latter term is the square of the instantaneous linear velocity. The product of the two is the centrifugal acceleration in classical mechanics. The standard definition is as follows. The use of a fifth-order polynomial ensures continuous acceleration of the trajectory (i.e., no abrupt change in acceleration), directly corresponding to and protecting the hydraulic lifting cylinders and steering linkages of heavy mobile machinery on site, avoiding instantaneous stress impact; centrifugal force constraint forcibly reduces the instantaneous speed of vehicles at sharp bends in blind spots of the yard intersection, sealing off the operational safety loopholes caused by the offset of the bulk cargo center of gravity.

[0070] It should be noted that the algorithm described in this embodiment belongs to a computing architecture that combines offline training with online real-time control.

[0071] Offline computing: Parameter training for deep reinforcement learning supernetworks and policy networks is completed offline in a high-performance data center in the cloud.

[0072] Online real-time execution terminal: In actual port area operation, the central cloud control center is configured with a dual-path server cluster. When the initial plan generation step is triggered, it completes the forward inference of the hybrid network trained offline and the multinomial coefficient matrix. The computational power requirement for the reverse operation is lower than that for the analysis. The computation time for a single task is consistently within 15 ms. The calculated coefficient matrix... Through the port area The low-latency network slicing channel sends data to the PLC controller of the vehicle on site. The conventional computing power built into the vehicle PLC performs real-time differentiation of the fifth-order polynomial within the control cycle, directly driving the vehicle's steer-by-wire and electronic throttle actuators. The hardware chain is clear and complete, and it is fully feasible.

[0073] Furthermore, step S3 in this embodiment specifically includes: S31. Extract the expected execution trajectory and expected operation progress from the digital twin base according to the initial plan according to the preset sampling period; S32. Synchronously acquire the actual operating trajectory and actual work progress of the on-site physical entities; S33. Calculate the spatial deviation between the expected execution trajectory and the actual running trajectory, and the time deviation between the expected work progress and the actual work progress; S34. Determine whether the spatial deviation value or the time deviation value exceeds the preset dynamic adaptive tolerance threshold; S35. If the threshold is exceeded, the abnormal source entity that caused the deviation and the affected range are extracted based on the coordinate location and timestamp of the deviation, and an interference event alarm report containing the ID of the affected intelligent agent is generated and output.

[0074] In the above steps, for the real-time digital twin monitoring and anomaly identification workflow, a two-dimensional spatial convolution algorithm based on artificial potential field theory was introduced to calculate the dynamic adaptive tolerance threshold. This algorithm effectively solves the safety technical pain point of false and frequent alarms or missed collisions in congested areas that are easily generated by traditional fixed monitoring thresholds in the complex and ever-changing scenarios of port areas.

[0075] This embodiment describes steps S31 to S35 in detail. These steps are performed by the edge cluster of the port central dispatch cloud control center for millisecond-level uninterrupted polling.

[0076] Specifically, steps S31 and S32 are the real-time synchronization of the expected state deduction and the actual physical state.

[0077] During the deployment mission, the monitoring system first extracts prior data from the digital twin base: along the time axis at a preset sampling period of 100Hz. Extract the coordinates of the expected execution trajectory derived from the initial plan. And expected work progress milestones (such as the timestamp of expected arrival at the hoisting position).

[0078] Simultaneously, the system receives real-time feedback from on-site physical entities (such as trucks, reach stackers, and gantry cranes) via IoT middleware. Positioning coordinates and Real-time progress tracking ensures absolute correspondence between twin space data and physical space data within the same microsecond-level time slice through precise timestamp alignment protocols (such as PTP Precision Time Protocol).

[0079] Steps S33 and S34 are the dynamic adaptive tolerance threshold calculation and deviation determination based on two-dimensional convolution of repulsive potential field.

[0080] After obtaining the expected and actual parameters, the system calculates the spatial absolute deviation value. and time schedule deviation value .

[0081] Because the port area includes both open main roads and highly congested storage yard blind spots, using a fixed threshold (e.g., a uniform rule that an alarm is triggered if the deviation is 2 meters) would lead to unnecessary re-bidding by multiple agents in open road sections, while in congested storage yards, it might have already caused a scraping accident. Therefore, this embodiment introduces a dynamic adaptive tolerance threshold generation function based on the repulsive potential field distribution.

[0082] Spatial dynamic adaptive tolerance threshold The mathematical model is defined as follows:

[0083] in, As a basic safety tolerance margin, this value is typically specified as 1.5 to 3.0 meters for regular container trucks at bulk cargo terminals; Using the current physical coordinates of the intelligent agent A local observation window centered on a point with a radius of R; The entity density distribution function in the local space is generated by analyzing the real-time bounding boxes of other flow machinery, obstacles and personnel in the digital twin base, and is represented as the weight values ​​of a two-dimensional grid matrix. This is the potential field attenuation coefficient, which controls the sensitivity of environmental crowding to threshold shrinkage, and its value is usually between 0.05 and 0.2. It is a very small constant to prevent singularity collapse caused by a zero denominator; M represents the real-time total mass of the current intelligent agent, which is read by the device's weighing sensor. The instantaneous scalar velocity of the current agent; The momentum compensation factor is determined by the friction coefficient of the brake disc of the equipment. It should be noted that the first part of the above formula is an integral extension of the artificial repulsive potential field method in classical robot navigation, and the integral term... Density of all entities within a local space The reciprocal of the distance in two-dimensional convolution characterizes the feeling of congestion and oppression in this area. In actual port operations, when trucks enter dense container / bulk cargo stacking areas or multi-vehicle intersections (where the integral value increases sharply), the exponential decay mechanism forces the allowable deviation threshold to be lowered. The speed was reduced from 3 meters to within 0.3 meters, and strict control was implemented. The latter part was based on the momentum theorem for reverse compensation. Because the greater the load and the faster the speed, the longer the braking inertia distance of the vehicle, and the allowable fault tolerance space must be compressed accordingly to ensure absolute safety.

[0084] Through on-site physics engine simulation testing, the introduction of this adaptive threshold function reduced the false frequent reordering alarm rate by 78% in open road sections, while increasing the effective collision interception rate to 99.4% in blind spots and dense intersection areas, thus balancing operational flexibility with ultimate safety.

[0085] Step S35 involves locking the anomaly source entity and analyzing the cascade wave of the explosion radius. When it is determined that the actual deviation value exceeds the threshold calculated in real time above ( When this occurs, the monitoring engine immediately triggers an interrupt.

[0086] The system traces the source entity of the anomaly back to its origin based on the coordinates and timestamp of the violation. Then, it invokes a directed acyclic graph task topology tree and employs a breadth-first search algorithm to calculate the blast radius of the anomaly along the time and space axes. All agents in the same pipeline as the source entity and in its subsequent processes are marked as affected.

[0087] Finally, the system generates and outputs an alarm report containing the IDs of all affected agents, the type of the anomaly, and the coordinates of the occurrence, and sends interruption and reorganization signals to the next process.

[0088] The monitoring and threshold calculation algorithm described in this embodiment falls under the category of online real-time control. The two-dimensional continuous spatial integral involved in the formula is discretized into matrix convolution operations at the computer level. Since the large-screen control terminal of the port area digital twin system is equipped with a high-performance GPU array, it is extremely adept at handling parallel two-dimensional matrix convolution and tensor operations. When performing rasterization calculations on a local 50x50 meter area, the threshold calculation cycle of a single device does not exceed 2 milliseconds. The high-frequency polling calculation of 500 devices in the entire field is completely within the performance margin of the GPU computing power, which can support real-time online millisecond-level interception and monitoring without generating system-level latency.

[0089] Furthermore, step S4 specifically includes: S41. When the interference event alarm report is received, the intelligent agent that is directly affected by the sudden interference event and is unable to continue to perform the original task shall act as the bidding party and broadcast the task bidding information to idle or low-load intelligent agents within its communication range. S42. The intelligent agent that receives the task bidding information acts as the bidder, calculates the execution cost of the task based on its current geographical location, remaining rated load and energy consumption status, and returns a bid containing the execution cost to the bidding party. S43. The tendering party collects all received bids, selects the optimal bidder agent based on the principle of minimizing execution cost, and sends the winning bid confirmation information to the optimal bidder agent. S44. After receiving the confirmation information, the winning agent updates its local task queue and generates a local adjustment instruction based on the updated task queue, which is then output to the corresponding physical entity for execution, thereby completing the local rearrangement.

[0090] The above steps introduce a high-order dynamic execution cost evaluation algorithm based on functional variational path integral and Gaussian risk complementary error function. This algorithm abandons the crude mode of simply weighting straight distance and equipment load in traditional logistics scheduling, and accurately quantifies the energy consumption and physical shutdown risk in complex port environment.

[0091] Specifically, step S4, as the self-healing core of the monitoring system in this embodiment, is triggered when the real-time monitoring and anomaly identification module throws an alarm report for an interference event containing the ID of the obstructed intelligent agent. Specifically, the following distributed collaborative control logic is executed: Step S41 is the abnormal interruption and task bidding broadcast.

[0092] When a physical entity (such as a truck transporting fertilizer) is unable to continue its journey due to a sudden mechanical failure or a landslide at the storage yard ahead, the intelligent agent mapping that entity (i.e., the bidding party) immediately suspends the current task pipeline. This bidding party intelligent agent sends a task bidding data packet to all idle or low-load intelligent agents (i.e., bidders) within its physical communication radius R via the V2X side link of the port area's 5G NSA private network. The bidding data packet encapsulates: the three-dimensional coordinates of the original task's breakpoint, the coordinates of the target point, the remaining tonnage of the cargo, and the delivery deadline timestamp.

[0093] Step S42 is the bid response based on the higher-order dynamic cost evaluation function.

[0094] Upon receiving the tender information, the bidding agent must not blindly accept the order but must assess the overall execution cost of undertaking the task. To accurately characterize the complex physical environment obstacles in the port area and the nonlinear energy consumption decay characteristics of heavy mobile machinery, this embodiment deploys a high-order dynamic execution cost evaluation function on each edge computing node.

[0095] No. The mathematical model for the computational execution cost C_i of each bidding agent is defined as follows:

[0096] in, For intelligent agents The overall execution cost of undertaking a new task (scalar, dimensionless, the smaller the value, the more suitable it is to undertake). To predict the total expected travel time required to reach the target point; This is the two-dimensional instantaneous velocity vector of the intelligent agent; It is a diagonal positive definite weight matrix used to penalize kinetic energy consumption and sharp turns during motion; Based on the current coordinates The gradient vector of the artificial potential field in the port area is the independent variable, representing the repulsive force of physical obstacles or traffic congestion resistance around the coordinate point. The complementary error function is defined as follows: This is used here to characterize nonlinear probability decay; For intelligent agents The current remaining effective energy (such as the SOC value of the power battery or the diesel tank level). The minimum safe energy consumption threshold set for completing this relay mission; The standard deviation of energy consumption fluctuations when the device performs similar tasks in the past; These are the weighting coefficients calibrated for the system. and Through the wheel encoder of the equipment chassis and High-precision positioning terminal fusion acquisition; Potential gradient Sobel operator difference is obtained by using a global grid map (with a resolution of 0.5 meters) distributed in real time by the digital twin base. The fuel level is obtained directly through the battery management system or ECU fuel level interface at the device's underlying level via the CAN bus 202; Range of weighting coefficients , This is used to impose a very high penalty barrier for energy depletion, and the weight is updated by the port area cloud server based on the daily production load.

[0097] It should be noted that the integral term on the left side of the formula is a functional variational path integral from analytical mechanics. In actual operations at bulk cargo terminals, heavy trucks may encounter potholes or congested road sections (i.e.,...). In extremely large regions, frequent braking and acceleration generate enormous mechanical work losses. This integral term directly reflects the total physical mechanical energy consumed to overcome environmental resistance; a complementary error function is introduced on the right side of the formula. It has a profound statistical basis and is based on the assumption of normal distribution. Depicting the remaining energy The risk of the vehicle breaking down halfway due to its inability to complete the task.

[0098] Through actual testing, it was found that traditional systems using Euclidean distance addition and subtraction for evaluation often resulted in trucks running out of power en route when accepting orders with only 15% of their remaining battery power. This was because the high energy consumption fluctuations during heavy-load starts were often overlooked. Introducing a complementary error function, due to the steep nonlinear penalty characteristics of the function in the extreme region, successfully reduced the mid-journey failure rate from 6.4% to below 0.02%, significantly improving the physical safety of the fleet accepting orders.

[0099] Furthermore, this function is an online real-time control algorithm, meaning it doesn't run on a central server but is instead deployed on industrial-grade edge gateways (such as industrial PCs equipped with NXP i.MX 8 or NVIDIA Jetson Nano) of each bidder's equipment. Because edge devices support hardware-level floating-point parallel computation and Taylor expansion approximation instructions, this complex expression, containing integrals and Gaussian functions, can be algebraically solved within a 20-millisecond single-step control cycle. After each bidder completes its calculation, the scalar... It is packaged into a tender document and returned to the tendering party.

[0100] Step S43 involves comparing execution costs and confirming the winning bid.

[0101] The bidding agent maintains a timer (e.g., a time window set to 100 milliseconds). After the time window expires, the bidding agent aggregates all collected bids, calls the array sorting interface, and sorts them according to execution cost. Extracting the optimal agent using the minimization principle Subsequently, the bidding party sends an encrypted winning bid confirmation frame based on the TCP / IP protocol to the optimal intelligent agent, and at the same time sends a release signal to other unsuccessful devices.

[0102] Step S44 generates instructions for updating the local task queue and smoothing the transition.

[0103] Optimal intelligent agent Upon receiving confirmation of winning the bid, the task is placed in its local high-priority task queue. To prevent abrupt changes in the current motion state of the equipment due to the instructions for the newly acquired task (e.g., requiring a vehicle traveling straight to make a 90-degree turn to take over the task, which would cause the vehicle to lose control and overturn), the system uses a quintic B-spline curve interpolation algorithm to generate local adjustment instructions.

[0104] Specifically, using the current device's coordinates, velocity, and acceleration as the initial constraint vector, and the breakpoint coordinates and tangent direction as the endpoint constraint vector, a continuous and bounded smooth transition control sequence with jerk is calculated through matrix inversion. This sequence is ultimately translated into a low-level executable steer-by-wire angle and motor torque control frame, which is then sent to the servo driver of the actuator for execution, thus completing a closed-loop, seamless local rearrangement of this sudden anomaly.

[0105] Furthermore, after step S4, the method further includes step S5: S51. The generated local adjustment command is sent to the corresponding physical entity for execution, and the local state nodes in the digital twin base are updated in real time according to the execution feedback; S52. After the sudden interference event is handled, extract the process data of this multi-agent collaborative negotiation and the actual time taken to finally eliminate the anomaly; S53. Combine the process data and actual time consumption into a state-action-reward tuple, and store it in the model experience pool as experience replay data for reinforcement learning. S54. When the data in the model's experience pool reaches a preset batch size, the network parameters of the deep reinforcement learning model are updated offline using the policy gradient descent algorithm, and the evolved deep reinforcement learning model is output.

[0106] In the above steps, this embodiment introduces a trust domain policy optimization algorithm based on a smoothing penalty term with the Jacobian matrix Frobenius norm. This model iterative algorithm effectively solves the security technical pain point of high-frequency control jitter at the physical execution end due to the mutation of neural network parameters after the traditional AI scheduling model evolves.

[0107] Specifically, step S51 involves issuing local instructions and updating the digital twin closed loop.

[0108] After completing the local rearrangement (step S44), the system's multi-agent collaborative negotiation module sends the generated local adjustment instructions to the corresponding physical entities (e.g., assigning a backup flow machine to take over the task of the malfunctioning vehicle). Upon receiving the instructions, the physical entities execute drive-by-wire actions, and their underlying IoT sensors synchronously feed back the actual movement trajectory and work progress to the digital twin base. The digital twin base, based on real-time telemetry data, synchronously refreshes the local state nodes, achieving state alignment between the physical world and the virtual space.

[0109] Steps S52 and S53 are for process data extraction and experience replay pool construction.

[0110] After the sudden interference event was completely resolved (i.e. the affected goods successfully arrived at the target point), the data cleaning pipeline in the system background was started to extract the global state sequence, the negotiation actions taken by each agent, and the actual total time and energy consumed in eliminating the anomaly during the multi-agent negotiation process.

[0111] The above data is encapsulated into state-action-reward tuples. This data, as a valuable and effective piece of experience replay data, is stored in the model experience pool on the cloud server. middle.

[0112] Step S54 is an iteration of a higher-order offline policy based on Jacobian matrix smoothing penalty.

[0113] When the model experience pool When the amount of new data accumulated within the system reaches the preset batch size, the system suspends the background training thread to perform offline evolution updates on the parameters of the deep reinforcement learning decision model.

[0114] To ensure the smoothness of the scheduling actions output by the model after evolution and to prevent physical damage to heavy machinery, this embodiment performs a high-order mathematical reconstruction of the objective function of deep reinforcement learning. Network parameters The iterative update aims to maximize the following objective functional :

[0115] in, For the weight parameters of the deep reinforcement learning model, These are the old parameters before the iteration; Given a state under the new strategy Take action at the time The probability distribution density; The advantage function represents the additional gain of the local rearrangement action relative to the average baseline level, and is calculated in the experience replay pool by generalized advantage estimation. The Kolbec-Leibler divergence between the probability distributions of the old and new policies is used to measure the degree of deviation between the two policy updates. The deterministic action mean vector output by the policy network Relative to the input state vector Jacobian matrix; The square of the Frobenius norm of a matrix is ​​equal to the sum of the squares of the absolute values ​​of all its elements. Let be the KL divergence penalty coefficient and the Jacobi smoothing regularization penalty coefficient, respectively.

[0116] In this embodiment, the size of the experience pool The threshold for triggering an offline iteration is typically set to 64,000 to 128,000 time-step transition tuples. Penalty coefficient The value range is usually set to The system monitors the divergence of KL divergence and automatically and dynamically adjusts it. Smoothing coefficient The range of values ​​is strictly defined as follows: The upper limit of this parameter is limited by the response bandwidth of the underlying hydraulic control system of the port flow equipment.

[0117] It should be noted that the first two terms of the above formula are derived from information geometry and trust domain policy optimization theory. By applying KL divergence constraints to the policy update step size, it has been rigorously proven in mathematics that it can guarantee that each evolution of the model will bring a monotonically improved policy performance. The third term of the formula is the Sobolev regularization constraint in calculus and functional analysis, which is essentially a constraint on the Lipschitz constant of the neural network mapping surface.

[0118] In bulk cargo handling at ports, mobile machinery such as heavy-duty reach stackers and 50-ton dump trucks are extremely vulnerable to hydraulic steering and electronic throttle systems. If the AI ​​model is updated, it may react to minute changes in state (such as those caused by sensor noise). A perturbation (or drastic fluctuation in the output of control commands) can cause hydraulic lines to rupture or trigger a severe pendulum effect. Jacobi smoothing penalty term. The physical significance lies in suppressing the high-frequency gain of the neural network, forcing the control sequence output by the updated AI model to be extremely smooth and continuous in the mathematical space.

[0119] After 5,000 training and comparison cycles using the physics engine on the port testing platform, it was proven that the updated AI model, employing an evolutionary algorithm that includes Jacobian matrix norm penalties, reduced the high-frequency glitch rate of the truck's steer-by-wire commands when deployed to the actual machine. This directly extends the mechanical fatigue life of the equipment steering tie rod assembly by approximately [amount missing]. .

[0120] Furthermore, the high-order functional solution is a purely offline computation. The formula involves a massive amount of probability density integrals, automatic differentiation of matrix partial derivatives, and norm calculations. These extremely computationally intensive tasks are not run on on-site equipment, but rather on a high-performance GPU computing cluster in the port's central computer room. The training process utilizes the off-peak hours at night for asynchronous background computation. Converge and compute the next generation of weights Subsequently, the system uses a lossless hot update protocol to directly inject the new weights into the online scheduling inference gateway, achieving model self-evolution with zero interference to production operations.

[0121] Example 2 like Figure 2 As shown, this embodiment discloses a monitoring system for cargo transportation within a port area. The system's hardware topology consists of a cloud-based central dispatch cluster, a 5G private network communication layer, and industrial edge computing gateways deployed on various flow control equipment and transport vehicles. Specifically, the system's module structure is as follows: I. Digital Twin and Multi-Agent Initialization Module This module is deployed in the access layer and rendering layer servers of the port area's cloud data center. Structurally, it includes a southbound physical data access gateway, a 3D topology rendering engine (such as a custom rendering engine based on WebGL), and a memory object instantiation unit.

[0122] Specifically, the southbound access gateway receives real-time status from the port area hardware via the MQTT protocol. To address packet loss caused by signal obstruction, the access gateway is equipped with a bidirectional Kalman filter smoother at its underlying layer, which can perform trajectory dead reckoning even after GPS signal loss of up to 500 milliseconds, ensuring the continuity of uploaded coordinates.

[0123] The memory object instantiation unit allocates an independent memory sandbox for each network-connected work device in the server's memory, constructing it as an agent object. By allocating independent multi-threaded handles, each agent object runs independently in the virtual space and maintains its own state vector space, thereby achieving a precise logical mapping from physical entities to the multi-agent network.

[0124] II. Global Plan Generation Module This module primarily relies on high-performance GPU computing clusters in cloud data centers (such as parallel arrays composed of multiple NVIDIA A100 tensor core graphics cards) to run.

[0125] Specifically, the module's front end is configured with a high-concurrency message queue, which is used to connect to the production work orders of the port area's cloud-based BTOS system, and to unpack discrete logistics orders in real time and transform them into feature vector matrices containing start and end coordinates and cargo attributes.

[0126] Using the feature vector matrix as input to the deep reinforcement learning model, the GPU cluster generates a global initial plan after completing joint Q-value calculation and fifth-order polynomial time-series decoding. This plan is not sent directly to the field; instead, it is packaged into control command frames conforming to JSON or Protobuf format via a control conversion API and distributed to the corresponding edge computing gateway through the core network.

[0127] III. Real-time Monitoring and Anomaly Detection Module This module, which serves as the system's monitoring hub, also runs on a cloud server, but is configured in the system architecture as a polling daemon process with the highest execution priority (RTOS-level preemptive priority).

[0128] Specifically, to achieve real-time calculation of the two-dimensional convolution tolerance threshold of the repulsive potential field as described in the claims, this module allocates a dedicated tensor calculation area in the video memory. The static road network and dynamic equipment entities of the entire port area are rasterized into a 500×500 resolution two-dimensional state matrix.

[0129] The daemon process uses the parallel multiply-accumulate instructions of the GPU hardware at a frequency of 100Hz to synchronously calculate the spatial error and threshold of all agents in the field. Once the deviation value of an entity exceeds the limit, the trigger inside the module immediately throws a hardware-level interrupt to the system bus 202, locks the abnormal source entity, and sends an interference event alarm message to subsequent modules.

[0130] IV. Multi-agent Cooperative Negotiation Module Unlike the aforementioned cloud-based modules, the multi-agent collaborative negotiation module employs a fully distributed, decentralized architecture. The code for each sub-unit is embedded within industrial-grade edge computing gateways (such as rugged vehicle-mounted industrial PCs powered by NXP i.MX 8 cores) deployed on physical entities like flow machines and modular equipment. When an interference event alarm is issued from the cloud, the field devices disconnect from the cloud and engage in point-to-point local self-negotiation. Specific technical details of the sub-units are as follows: 1. The tender broadcast subunit resides in the vehicle gateway of the abnormal source entity. Upon receiving an alarm interruption from the cloud, this subunit immediately cuts off the normal driving CAN messages from the vehicle chassis, forcing the device into a safe parking state. Subsequently, this subunit activates the PC5 direct-connect side link of the vehicle V2X communication module, and broadcasts the tender data packet containing the breakpoint coordinates and remaining task volume to intelligent agents within a set physical radius (e.g., 800 meters) in the form of UDP multicast with low latency, without needing to be relayed through a 5G base station.

[0131] 2. Cost Calculation and Bidding Subunit: This unit resides in the edge gateway of surrounding normally operating equipment that receives multicast signals. The gateway has a built-in floating-point unit for calculating the integral and Gaussian complementary error function. This subunit reads the current SOC value of the vehicle's battery management system (BMS) in real time through the onboard OBD interface and reads the chassis speed. Cost calculation is completed within a 20-millisecond time window. The calculation is then performed, and a bid message containing the bid value and vehicle MAC address is encapsulated and sent back to the bidding party's vehicle gateway via TCP / IP protocol unicast.

[0132] 3. Evaluation and Bid Confirmation Subunit: This subunit resides on the bidding party's vehicle-mounted gateway. It is equipped with a hardware timer that starts counting from the moment a multicast signal is sent. When the time window (e.g., set to 100 milliseconds) arrives, the timer interrupts the current receiving process. The sorting module quickly sorts all received bids by value, selecting the winning device with the lowest value. A bid confirmation ACK frame is sent to this winning device. Upon receiving the ACK, the edge gateway of the winning device uses its internal trajectory generator to generate a smooth local adjustment path using a quintic B-spline algorithm. This path is then translated into control pulses (PWM or CANOpen protocol) for the chassis drive motor and steer-by-wire, directly driving the physical actuators. This allows for independent replacement and self-healing of the physical entity on-site without relying on cloud recalculation.

[0133] Example 3 Please refer to Figure 3 The diagram illustrates a schematic of an electronic device provided by some embodiments of this application. The electronic device 20 includes a processor 200, a memory 201, a bus 202, and a communication interface 203. The processor 200, the communication interface 203, and the memory 201 are connected via the bus 202. The memory 201 stores a computer program that can run on the processor 200. The processor 200 is used to perform operations according to the instructions to execute the steps of the method described in Embodiment 1.

[0134] In actual port applications, the electronic device 20 can be a scheduling server node centrally deployed in the port's cloud data center, or it can be an industrial-grade edge computing gateway distributed and deployed on various flow equipment (such as gantry cranes and reach stackers) and transport vehicles (such as container trucks). In terms of hardware physical architecture, the electronic device 20 mainly includes: a communication bus 202, a communication interface 203, a memory 201, and a processor 200.

[0135] The communication bus 202 is used to realize the internal connection and bus 202 addressing between the processor 200, memory 201 and communication interface 203. Depending on the device form, the bus 202 may include a peripheral component interconnect standard (PCIe) bus 202, an integrated circuit built-in (I2C) bus 202 or a high-speed serial computer expansion bus 202, etc. It ensures the bandwidth throughput when multi-source IoT data is transmitted to the processor 200.

[0136] Communication interface 203 is used to enable network interaction between electronic device 20 and IoT sensors (such as RTK high-precision positioning terminals, PLC status acquisition devices, and vehicle-mounted electronic weighing modules) and other intelligent agent nodes in the port area's physical space. Its hardware includes a 5G NSA baseband communication module, an industrial Ethernet card, and a V2X (Vehicle-to-Everything) sidelink RF antenna module configured to support local negotiation between multiple intelligent agents.

[0137] The memory 201 is used to store the computer program executing the method of the present invention and the static data (such as high-precision 3D map data of the port area) and dynamic data (such as experience replay pool data of reinforcement learning) required for runtime. Specifically, the memory 201 includes a volatile high-speed random access memory 201 (RAM, such as DDR4 / DDR5 memory chips) for caching the frequently refreshed digital twin coordinate array; and a non-volatile memory 201 (NVM), such as an industrial-grade solid-state drive (SSD) or an embedded multimedia memory card (eMMC), for persistently storing the network weight files and system logs of the deep reinforcement learning model.

[0138] The processor 200, as the control and computing core of the electronic device, is responsible for calling the computer instruction set stored in the memory 201 and parsing and executing it.

[0139] In cloud application scenarios, the processor 200 includes a multi-core central processing unit 200 (CPU) and a tensor graphics processing unit 200 (GPU) array, which is specifically used to handle offline model iteration or global inference tasks that require extremely high concurrent floating-point computing power, such as global joint Q-value calculation and Jacobian matrix smoothing penalty operation.

[0140] In edge application scenarios, if the electronic device is an in-vehicle edge computing gateway, the processor 200 can be a system-on-a-chip (SoC) integrating a neural network processing unit (NPU) and an ARM architecture microprocessor 200. This architecture ensures that the electronic device can independently complete the online real-time calculation of "high-order dynamic execution cost evaluation" and "five-order polynomial timing decoding" within an extremely short control cycle of 20 milliseconds in harsh industrial vehicle environments.

[0141] When the electronic device is powered on and the system is initialized, the processor 200 reads the corresponding computer program from the memory 201 through the communication bus 202. When executing the program, all the steps of the port area cargo transfer monitoring method based on multi-agent collaboration and deep reinforcement learning described in any embodiment of the present invention are implemented.

[0142] Example 4 This embodiment also provides a computer-readable storage medium corresponding to the FPC component injection molding and packaging method provided in the foregoing embodiments. Please refer to... Figure 4 The computer-readable storage medium shown is an optical disc, on which a computer program (i.e., program product 30) is stored. When the computer program is run by the processor 200, it executes the monitoring method provided in any of the aforementioned embodiments.

[0143] The term "computer-readable storage medium" as used herein includes any tangible and non-transitory medium capable of storing program code. Specific physical forms include, but are not limited to, physical storage devices such as solid-state drives (SSDs), USB flash drives, external hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks (such as CD-ROMs).

[0144] Data Structure Configuration: Within this storage medium, the program code is encapsulated using a modular, highly cohesive, and loosely coupled structure. Its logical division strictly corresponds to the method flow, packaged into digital twin mapping module code, deep reinforcement learning inference and evolution engine code, real-time threshold monitoring code based on the repulsive potential field algorithm, and multi-agent negotiation and control code based on the Contract Network Protocol (CNP). During execution, each code package directly calls the APIs of the device's underlying operating system (such as Linux kernel drivers) to rationally allocate thread pool and memory stack resources at the hardware layer.

[0145] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit the scope of protection of the present invention. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the essence and scope of the technical solutions of the present invention.

Claims

1. A method for monitoring cargo transfer within a port area, characterized in that, The method includes the following steps: S1. Construct a digital twin foundation for the port area and abstract the physical entities within the port area into a multi-agent network with independent perception and decision-making capabilities; S2. Obtain real-time cargo dispatch tasks and generate an initial plan for global resource collaborative dispatch based on a pre-trained deep reinforcement learning model; S3. Real-time monitoring of the actual transportation process in the port area to identify sudden interference events between the digital twin base and the physical space; S4. When the aforementioned sudden interference event is detected, a multi-agent collaborative negotiation mechanism is triggered. Each affected agent re-bids based on the Contract Network protocol, dynamically allocates tasks to generate local adjustment instructions, and issues them for execution.

2. The monitoring method according to claim 1, characterized in that, Step S1 specifically includes: S11. Acquire real-time multi-dimensional physical status data, including berths, storage yards, flow equipment and transport vehicles, through multi-source IoT sensors deployed in the port area; S12. Based on the multidimensional physical state data, a port area digital twin base that is updated in real time is mapped and generated in the virtual space; S13. In the digital twin base, each berth, flow machine equipment and transport vehicle is encapsulated as an independent intelligent agent, and an observation state space, behavior action space and local reward function are defined for each intelligent agent; S14. Connect all defined agents through topological relationships and output an initialized multi-agent network for subsequent dispatch planning and monitoring.

3. The monitoring method according to claim 1, characterized in that, Step S2 specifically includes: S21. Extract the feature vector of the real-time cargo dispatching task to be executed, and the global state matrix of all agents in the current multi-agent network; S22. Input the feature vector and global state matrix into a pre-trained deep reinforcement learning decision network; S23. The deep reinforcement learning decision network is used to predict the Q value of each agent taking different scheduling actions in the current state, and a greedy strategy is used to select the action combination that maximizes the global expected cumulative reward. S24. Decode the action combination into specific task timing and running path for each agent, and output the timing and path as the initial plan.

4. The monitoring method according to claim 1, characterized in that, Step S3 specifically includes: S31. Extract the expected execution trajectory and expected operation progress from the digital twin base according to the initial plan according to the preset sampling period; S32. Synchronously acquire the actual operating trajectory and actual work progress of the on-site physical entities; S33. Calculate the spatial deviation between the expected execution trajectory and the actual running trajectory, and the time deviation between the expected work progress and the actual work progress; S34. Determine whether the spatial deviation value or the time deviation value exceeds the preset dynamic adaptive tolerance threshold; S35. If the threshold is exceeded, the abnormal source entity that caused the deviation and the affected range are extracted based on the coordinate location and timestamp of the deviation, and an interference event alarm report containing the ID of the affected intelligent agent is generated and output.

5. The monitoring method according to claim 1, characterized in that, Step S4 specifically includes: S41. When the interference event alarm report is received, the intelligent agent that is directly affected by the sudden interference event and is unable to continue to perform the original task shall act as the bidding party and broadcast the task bidding information to idle or low-load intelligent agents within its communication range. S42. The intelligent agent that receives the task bidding information acts as the bidder, calculates the execution cost of the task based on its current geographical location, remaining rated load and energy consumption status, and returns a bid containing the execution cost to the bidding party. S43. The tendering party collects all received bids, selects the optimal bidder agent based on the principle of minimizing execution cost, and sends the winning bid confirmation information to the optimal bidder agent. S44. After receiving the confirmation information, the winning agent updates its local task queue and generates a local adjustment instruction based on the updated task queue, which is then output to the corresponding physical entity for execution, thereby completing the local rearrangement.

6. The monitoring method according to claim 1, characterized in that, Following step S4, the method further includes step S5: S51. The generated local adjustment command is sent to the corresponding physical entity for execution, and the local state nodes in the digital twin base are updated in real time according to the execution feedback; S52. After the sudden interference event is handled, extract the process data of this multi-agent collaborative negotiation and the actual time taken to finally eliminate the anomaly; S53. Combine the process data and actual time consumption into a state-action-reward tuple, and store it in the model experience pool as experience replay data for reinforcement learning. S54. When the data in the model's experience pool reaches a preset batch size, the network parameters of the deep reinforcement learning model are updated offline using the policy gradient descent algorithm, and the evolved deep reinforcement learning model is output.

7. A monitoring system for cargo transfer within a port area, characterized in that, The system for performing the method as described in any one of claims 1-6 includes: The digital twin and multi-agent initialization module is used to build the digital twin foundation of the port area and abstract the physical entities in the port area into a multi-agent network with independent perception and decision-making capabilities. The global plan generation module is used to obtain real-time cargo transportation tasks and generate an initial plan for global resource collaborative transportation based on a pre-trained deep reinforcement learning model. The real-time monitoring and anomaly identification module is used to monitor the port area's transportation process in real time and identify sudden interference events between the digital twin base and the physical space. The multi-agent collaborative negotiation module is used to trigger the multi-agent collaborative negotiation mechanism when a sudden interference event is detected. Each affected agent re-bids based on the contract network protocol, dynamically allocates tasks to generate local adjustment instructions and issues them for execution.

8. The monitoring system according to claim 7, characterized in that, The multi-agent collaborative negotiation module includes: The bidding broadcast subunit is configured so that the intelligent agent whose task is interrupted due to a sudden interference event acts as the bidding party, generates task bidding information and multicasts it to surrounding intelligent agents; The cost calculation and bidding subunit is configured such that after the bidding agent receives the bidding information, it calculates the execution cost based on its own state and submits a bid containing the execution cost. The evaluation and bid confirmation subunit is configured so that the bidding party selects the optimal bidder based on the principle of minimizing execution cost and issues bid confirmation information, triggering the winning agent to generate the local adjustment instruction.

9. An electronic device, characterized in that, include: It includes a memory, a processor, and a computer program stored in the memory and executable on the processor, the processor running the computer program to implement the steps of the monitoring method as described in any one of claims 1-6.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, The program is executed by the processor to implement the steps of the monitoring method as described in any one of claims 1-6.

Images