Physical heterogeneous formation centralized training of distributed execution control methods, equipment and media
By generating capability envelopes and structured test scenarios, and performing targeted information ablation and physical counterfactual testing, the problems of indivisible control results for physically heterogeneous vehicle formations and unprovable CTDE advantages in existing technologies are solved, and the diagnosis of failure mechanisms and the quantification of deployment boundaries are realized.
Patent Information
- Application Number
- CN202611413643.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-09-11
- Publication Date
- 2026-11-17
AI Technical Summary
Existing CTDE control schemes for mixed traffic flow lack controlled diagnosis of differences in the capability envelopes of physically heterogeneous vehicles, making it difficult to further convert outcome indicators into failure mechanism labels and deployment recommendations. The source of CTDE's advantages is unprovable, and the deployment boundaries of communication degradation cannot be quantified.
By initializing the physical heterogeneous formation mission environment, the capability envelope of each formation object is generated, candidate control policies are trained under unified training conditions, structured controlled test scenarios are generated based on the target physical heterogeneity level and communication quality, targeted information ablation and physical counterfactual testing are performed, the final policy is diagnosed, and a deployment policy is generated.
It achieves effective control of physically heterogeneous vehicle formations, identifies the failure mechanisms and deployment boundaries of learning-based formation control strategies, and solves the problems of visible control results but indivisible failure root causes, unprovable sources of CTDE advantages, and unquantifiable deployment boundaries of communication degradation.
Smart Images

Figure CN123113555A_ABST
Abstract
Description
Technical Field
[0001] The embodiments of the present invention relate to the field of intelligent transportation technology, and in particular to a method, device and medium for centralized training and distributed execution control of physical heterogeneous formations. Background Technology
[0002] Vehicle platooning improves traffic safety, efficiency, and energy consumption by coordinating spacing, speed, and maneuverability among adjacent vehicles. Existing solutions mainly include Adaptive Cruise Control (ACC), Cooperative Adaptive Cruise Control (CACC), Model Predictive Control (MPC), and Centralized Training with Decentralized Execution (CTDE) based on multi-agent reinforcement learning. CACC and MPC typically rely on modelable and compensable differences in vehicle dynamics; CTDE methods such as MAPPO (Multi-Agent Proximal Policy Optimization) utilize centralized information during the training phase, while each vehicle executes based on local observations during the deployment phase, making them suitable for vehicle-side distributed control. For example, related technologies disclose a multi-dimensional state matrix for mixed traffic flow, a dynamic spatiotemporal directed graph, graph attention feature extraction, a multi-agent proximal policy optimization network based on a CTDE architecture, a joint reward function, and a one-step kinematic safety supervision mechanism, enabling cooperative control of mixed vehicle platoons. Its focus is on generating coordinated control actions and improving the effectiveness of mixed traffic flow control through safety monitoring.
[0003] However, existing technologies have the following drawbacks: (1) Existing mixed traffic flow CTDE control schemes can output control actions, but lack controlled diagnosis of the differences in the capability envelope of physically heterogeneous vehicles.
[0004] (2) Existing solutions typically evaluate control effectiveness using outcome indicators such as rewards, spacing errors, or collision rates. They lack the technical means to further convert outcome indicators into failure mechanism labels and deployment recommendations, making it difficult to guide the vehicle-road cloud simulation platform to conduct risk screening before going online.
[0005] (3) The existing CTDE training process usually compares the average performance of different algorithms, which can easily misjudge the small sample random advantage as the control strategy advantage. It cannot verify whether the advantage comes from the physical heterogeneous capability information through paired seeds, final strategy unified evaluation and targeted ablation.
[0006] (4) Existing communication disturbance tests mostly remain at the evaluation level, and do not map the impact of packet loss, delay and observation noise on reward retention rate and collision risk inflation rate into specific robust training or policy replacement rules.
[0007] Therefore, there is an urgent need in this field to address the problems that control results are visible but failure causes are indivisible, the sources of CTDE advantages are unprovable, and the deployment boundaries of communication degradation are unquantifiable. Summary of the Invention
[0008] The purpose of this invention is to provide at least one method, device and medium for centralized training and distributed execution control of physical heterogeneous formations, which can at least solve the technical problems of existing methods where the control results are visible but the root causes of failure are indivisible, the source of CTDE advantages is unprovable, and the deployment boundary of communication degradation is unquantifiable.
[0009] To address the aforementioned technical problems, at least one embodiment of this application provides a method for centralized training and distributed execution control of physically heterogeneous formations, comprising: Initialize the physical heterogeneous formation task environment and generate the capability envelope for each formation object; Candidate control policies are trained under uniform training conditions, including decentralized baseline policies and CTDE candidate policies. A structured, controlled test scenario is generated based on the target physical heterogeneity level, formation size, communication quality, predefined shared discrete action space, and formation object arrangement order. The final policy obtained from training is evaluated in each controlled test scenario; While keeping the capability envelope, state transition and controlled test scenario of the formation objects unchanged, perform directional information ablation, and perform physical counterfactual tests of equilibrium dynamics while changing the capability envelope and physical heterogeneity level of the formation objects; Based on the results of targeted information ablation and physical counterfactual testing, the final strategy is diagnosed and diagnostic labels are generated. The deployment strategy for the final strategy is generated based on the diagnostic labels.
[0010] At least one embodiment of this application also provides a distributed execution control device for centralized training of physical heterogeneous formations, comprising: The heterogeneous formation mission environment module is used to initialize the physical heterogeneous formation mission environment and generate the capability envelope of each formation object. A unified training module for candidate control strategies is used to train candidate control strategies under unified training conditions. The candidate control strategies include decentralized baseline strategies and CTDE candidate strategies. The multidimensional controlled test scenario generation module is used to generate structured controlled test scenarios based on the target physical heterogeneity level, formation size, communication quality, predefined shared discrete action space, and formation object arrangement order. The final policy evaluation module is used to evaluate the trained final policy in various controlled test scenarios. While keeping the capability envelope, state transition and controlled test scenario of the formation objects unchanged, perform directional information ablation, and perform physical counterfactual tests of equilibrium dynamics while changing the capability envelope and physical heterogeneity level of the formation objects; The mechanism diagnosis and strategy selection module is used to diagnose the final strategy and generate diagnostic labels based on the results of targeted information ablation and physical counterfactual testing, and to generate the deployment strategy of the final strategy based on the diagnostic labels.
[0011] At least one embodiment of this application also provides an electronic device, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to execute the above-described physical heterogeneous formation centralized training distributed execution control method.
[0012] At least one embodiment of this application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described method for centralized training and distributed execution control of physical heterogeneous formations.
[0013] The method provided in this application generates a capability envelope for each formation object; trains a strategy under uniform training conditions; generates structured controlled test scenarios based on the target physical heterogeneity level, formation size, communication quality, predefined shared discrete action space, and formation object arrangement order; and evaluates the trained final strategy in each controlled test scenario. While keeping the formation object's capability envelope, state transitions, and controlled test scenarios unchanged, it performs directional information ablation; and performs physical counterfactual testing of equilibrium dynamics while changing the formation object's capability envelope and physical heterogeneity level. This diagnoses the final strategy, generates diagnostic labels, and generates a deployment strategy for the final strategy. This method can solve the technical problems of visible control results but indivisible failure root causes, unprovable sources of CTDE advantages, and unquantifiable deployment boundaries of communication degradation.
[0014] In some optional embodiments, the physical heterogeneous formation centralized training distributed execution control method further includes: calculating capability offset and internal capability difference, wherein the capability envelope includes the maximum positive acceleration amplitude, the maximum deceleration amplitude and the maximum velocity, the capability offset represents the maximum capability difference of the formation object relative to the reference capability envelope, and the internal capability difference represents the maximum capability difference between the formation objects.
[0015] In some optional embodiments, the physical heterogeneous formation centralized training distributed execution control method further includes: selecting a target physical heterogeneous level from candidate physical heterogeneous levels based on the internal capability difference. Selecting a target physical heterogeneity level from candidate physical heterogeneity levels based on the internal capability difference includes: if the internal capability difference does not exceed a first threshold, then selecting a first physical heterogeneity level as the target physical heterogeneity level; if the internal capability difference is greater than the first threshold but does not exceed a second threshold, then selecting a second physical heterogeneity level as the target physical heterogeneity level; if the internal capability difference exceeds the second threshold, then selecting a third physical heterogeneity level as the target physical heterogeneity level, wherein the first threshold is ≥0 and less than the second threshold.
[0016] In some optional embodiments, the physical heterogeneous formation mission environment includes formation size, target spacing, target speed, set of formation object types, and formation object arrangement order.
[0017] In some optional embodiments, the physical heterogeneous formation centralized training distributed execution control method further includes: constructing a capability-aware local observation vector, wherein the local observation vector includes: its own position or velocity, relative distance and relative velocity with adjacent formation objects, maximum speed proxy and action feasibility flag, wherein the action feasibility flag is determined based on the current velocity, predicted vehicle spacing and capability envelope, and is used to enable candidate control strategies to identify or block currently unexecutable actions based on the flag when outputting actions.
[0018] In some optional embodiments, the physical heterogeneous array centralized training distributed execution control method further includes: A predefined shared discrete action space is defined, which includes at least one of a three-valued symbolic action set and a five-valued symbolic action set. The same symbolic action is mapped to different accelerations according to the capability envelope, and the velocity and position of the formation objects are updated based on the mapping results.
[0019] In some optional embodiments, candidate control policies are trained under uniform training conditions, including: Candidate control policies are trained using the same observation fields, action interfaces, reward functions, training steps, random seed sets, final policy evaluation rules, and invalid run judgment rules. Attached Figure Description
[0020] One or more embodiments are illustrated by way of example with reference to the accompanying drawings, and these illustrative descriptions do not constitute a limitation on the embodiments.
[0021] Figure 1 This is a flowchart of a method for centralized training and distributed execution control of physical heterogeneous formations provided in an embodiment of this application; Figure 2 This is a schematic diagram of the closed-loop process of the heterogeneous stress diagnosis protocol provided in an embodiment of this application; Figure 3 This is a schematic diagram of a distributed execution control device for centralized training of physical heterogeneous formations provided in an embodiment of this application. Detailed Implementation
[0022] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the various embodiments of this application will be described in detail below with reference to the accompanying drawings. However, those skilled in the art will understand that many technical details have been provided in the various embodiments of this application to help readers better understand this application. However, the technical solutions claimed in this application can be implemented even without these technical details and various changes and modifications based on the following embodiments. The division of the various embodiments below is for the convenience of description and should not constitute any limitation on the specific implementation of this application. The various embodiments can be combined with and referenced by each other without contradiction.
[0023] To facilitate understanding of the embodiments of this application, relevant content regarding the prior art will be introduced first.
[0024] Vehicle platooning improves traffic safety, efficiency, and energy consumption by coordinating distance, speed, and maneuverability among adjacent vehicles. Existing solutions mainly include Adaptive Cruise Control (ACC), Cooperative Adaptive Cruise Control (CACC), Model Predictive Control (MPC), and Centralized Training Distributed Execution (CTDE) methods based on multi-agent reinforcement learning. Related technologies include MAPPO-like CTDE algorithms and CACC-like traditional platooning control methods. MAPPO itself is a general cooperative MARL algorithm; CACC is geared towards cooperative following control and typically assumes that vehicle dynamics differences can be modeled or compensated. CACC and MPC usually rely on modelable and compensable vehicle dynamics differences and do not provide heterogeneous stress diagnosis and mechanism attribution processes for learning-based strategies; CTDE methods such as MAPPO utilize centralized information during the training phase, and each vehicle executes based on only local observations during the deployment phase, making them suitable for vehicle-side distributed control but not limiting the diagnosis of vehicle dynamics heterogeneity, communication degradation, and safety indicators. For example, related technologies disclose multi-dimensional state matrices for mixed traffic flows, dynamic spatiotemporal directed graphs, graph attention feature extraction, multi-agent proximal policy optimization networks based on CTDE architecture, joint reward functions, and one-step kinematic safety supervision mechanisms, enabling collaborative control of mixed vehicle fleets. The focus is on generating collaborative control actions and improving the control effect of mixed traffic flows through safety supervision, without addressing the "heterogeneous stress diagnostic protocol" for differences in physical capability envelopes. Specifically, this existing technology does not define physical heterogeneous stress levels such as H0 / H1 / H2, nor does it diagnose differences in vehicle capabilities such as maximum acceleration, maximum deceleration, and maximum speed as controlled test dimensions. It does not attribute performance changes to physical heterogeneity, scale stress, communication degradation, or capability information dependence through unified final policy evaluation, paired random seeds, directional ablation, and diagnostic metrics. Furthermore, it does not convert diagnostic results into deployable decision rules that require robust training, policy replacement, or model structure adjustment.
[0025] However, existing technologies have the following drawbacks: (1) Existing mixed traffic flow CTDE control schemes can output control actions, but lack controlled diagnosis of the differences in the capability envelope of physically heterogeneous vehicles.
[0026] (2) Existing solutions typically evaluate control effectiveness using outcome indicators such as rewards, spacing errors, or collision rates. They lack the technical means to further convert outcome indicators into failure mechanism labels and deployment recommendations, making it difficult to guide the vehicle-road cloud simulation platform to conduct risk screening before going online.
[0027] (3) The existing CTDE training process usually compares the average performance of different algorithms, which can easily misjudge the small sample random advantage as the control strategy advantage. It cannot verify whether the advantage comes from the physical heterogeneous capability information through paired seeds, final strategy unified evaluation and targeted ablation.
[0028] (4) Existing communication disturbance tests mostly remain at the evaluation level, and do not map the impact of packet loss, delay and observation noise on reward retention rate and collision risk inflation rate into specific robust training or policy replacement rules.
[0029] Therefore, there is an urgent need in this field to address the problems that control results are visible but failure causes are indivisible, the sources of CTDE advantages are unprovable, and the deployment boundaries of communication degradation are unquantifiable.
[0030] To address the technical problems of existing methods, this invention proposes a physically heterogeneous formation centralized training distributed execution control method. This method goes beyond simply applying CTDE or MAPPO to vehicle formation control itself, or utilizing reward functions or safety supervision. Instead, it establishes a HSDP heterogeneous stress diagnostic protocol around physically heterogeneous vehicle formations. This protocol connects capability-aware observation, controlled test scenarios, a unified training interface, final policy evaluation, targeted ablation, diagnostic metrics, and deployment policy output into a closed loop. This loop is used to identify the failure mechanisms and deployment boundaries of learning-based formation control strategies under physically heterogeneous conditions. The implementation details of this physically heterogeneous formation centralized training distributed execution control method are described below. These details are provided for ease of understanding and are not essential for implementing this solution.
[0031] Example 1: The physical heterogeneous formation centralized training distributed execution control method of this embodiment can be applied to electronic devices with communication, computing, and data storage capabilities. Its specific process can be as follows: Figure 1 As shown, it includes: Step 110: Initialize the physical heterogeneous formation task environment and generate the capability envelope for each formation object.
[0032] Specifically, the physical heterogeneous formation mission environment includes formation size, target spacing, target velocity, set of formation object types, and formation object arrangement order.
[0033] In practical implementation, the formation object (diagnostic object) can include vehicle formation objects (vehicles), or it can be extended to include drone formations (drones), robot swarms (robots), or other multi-agent collaborative control systems with different physical capabilities. Taking vehicles as an example, the capability envelope includes the maximum positive acceleration amplitude, the maximum deceleration amplitude, and the maximum speed. If the formation object is extended from vehicles to drone formations and robot swarms, the maximum positive acceleration amplitude, the maximum deceleration amplitude, and the maximum speed are replaced with capability envelope parameters such as drone thrust, payload, battery power, turning radius, or robot maximum speed and load capacity. The subsequent diagnosis is still performed according to the HSDP controlled test dimensions, final strategy evaluation, and targeted ablation.
[0034] Step 120: Train the candidate control policy under uniform training conditions.
[0035] Specifically, candidate control policies are trained under uniform training conditions, including: Train candidate control policies using the same observation fields, action interfaces, reward functions, training steps, random seed sets, final policy evaluation rules, and invalid run judgment rules to ensure consistent execution interfaces.
[0036] Specifically, candidate control strategies include decentralized baseline strategies and CTDE candidate strategies.
[0037] In one example, decentralized and CTDE policies are trained under a unified interface, with IPPO as the decentralized baseline and MAPPO as a CTDE candidate policy; after training, the final policy is retained and evaluated uniformly, and similar algorithms can be replaced.
[0038] In other examples, CTDE candidate policies can be replaced by MAPPO with QMIX, MADDPG, VDN, CTDE methods employing graph-structured centralized value networks or Transformer centralized value networks, and other centralized training distributed execution methods. The replacement algorithms do not change the unified training interface, final policy evaluation, controlled diagnostics, and attribution rules.
[0039] Step 130: Generate a structured, controlled test scenario based on the target physical heterogeneity level, formation size, communication quality, predefined shared discrete action space, and formation object arrangement order.
[0040] In practice, a controlled test matrix can be generated based on H, N, C, A, O and a random seed index Z, and the specific parameter values of each dimension can be saved to form a multi-dimensional controlled test scenario.
[0041] In one example, HSDP sets six controlled test dimensions: physical heterogeneity level H, formation size N, communication quality C, action space A, vehicle arrangement order O, and random seed index S. These six dimensions together constitute the controlled test space Ω = Ω_H × Ω_N × Ω_C × Ω_A × Ω_O × Ω_seed. H is determined by D_H = D_pair and thresholds τ0 and τ2; N can be 4, 6, 10, or other preset sizes; each value of C includes the communication type and its numerical parameters, such as packet loss rate, fixed or random delay distribution, and observation noise intensity; A includes three-valued discrete action sets and five-valued discrete action sets; O represents the arrangement order of vehicles with different capabilities when they are in front, middle, or rear; S represents the pairwise random seed index used for training and evaluation, which can also be represented by Z. The above dimensions are combined according to preset values to form a structured controlled test scenario and a controlled test matrix. When performing univariate diagnostics, only one dimension is changed while keeping the others constant. When performing multi-factor interaction diagnostics, two or more dimensions are changed simultaneously to identify the coupled effects of factors such as physical heterogeneity, scale, communication, and vehicle sequence. Each candidate strategy must use the same observation fields, action interface, reward function, training budget, random seed set, final strategy evaluation rules, and invalid run judgment rules. Deployment must not be based solely on the best checkpoint or single-run average reward.
[0042] Step 140: Evaluate the final policy obtained from training in each controlled test scenario.
[0043] In the specific implementation, the final strategy after training is uniformly evaluated, and the outputs are: mean reward, absolute error of average distance between adjacent vehicle pairs, round collision ratio, speed variance, training cost, running time, invalid run flag, random seed sensitivity, and normalized distance deviation (NSD). The round collision ratio is uniformly defined as the percentage of rounds in a fixed evaluation round where at least one vehicle collision occurs.
[0044] Step 150: While keeping the capability envelope, state transition and controlled test scenario of the formation objects unchanged, perform directional information ablation, and perform physical counterfactual tests of equilibrium dynamics while changing the capability envelope and physical heterogeneity level of the formation objects.
[0045] Step 160: Based on the results of targeted information ablation and physical counterfactual testing, diagnose the final strategy and generate diagnostic labels.
[0046] Step 170: Generate the final deployment strategy based on the diagnostic tags.
[0047] The method in this embodiment solves the technical problems in the prior art, such as visible control results but indivisible failure root causes, unprovable sources of CTDE advantages, and unquantifiable deployment boundaries of communication degradation, by defining physical heterogeneity levels, constructing capability-aware local observations, unifying training and final strategy evaluation interfaces, applying controlled test conditions such as physical heterogeneity / scale / communication / action space / vehicle sequence, performing targeted ablation, calculating diagnostic metrics, and outputting deployment strategy recommendations.
[0048] In some embodiments, the above-described physical heterogeneous formation centralized training distributed execution control method further includes: selecting a target physical heterogeneous level from candidate physical heterogeneous levels based on the internal capability difference.
[0049] Specifically, selecting a target physical heterogeneity level from candidate physical heterogeneity levels based on the amount of internal capability difference includes: if the amount of internal capability difference does not exceed a first threshold, then the first physical heterogeneity level is selected as the target physical heterogeneity level; if the amount of internal capability difference is greater than the first threshold but does not exceed a second threshold, then the second physical heterogeneity level is selected as the target physical heterogeneity level; if the amount of internal capability difference exceeds the second threshold, then the third physical heterogeneity level is selected as the target physical heterogeneity level, wherein the first threshold is ≥0 and less than the second threshold.
[0050] In the specific implementation, the capability envelope of each vehicle is defined as follows: Equation (1) in, For vehicles i The maximum positive acceleration amplitude, The values represent the maximum deceleration amplitude, all in m / s². This represents the maximum speed, expressed in m / s. All capability parameters are positive numbers.
[0051] Given the capability envelope B0=(a0,b0,v0) of a reference vehicle, where a0, b0, v0>0, the normalized capability bias is defined as: Equation (2) Where α, β, and γ are positive weights and satisfy α + β + γ = 1, ensuring that the hierarchical scale remains interpretable under different weight settings; D i It is a dimensionless quantity.
[0052] In one example, α=0.4, β=0.4, γ=0.2.
[0053] The difference in the capability envelope between the two vehicles is defined as: Equation (3) In some embodiments, the above-mentioned centralized training and distributed execution control method for physically heterogeneous formations further comprises: calculating a capability offset and an internal capability difference. The capability offset represents the maximum capability difference of the formation objects relative to the reference capability envelope, and the internal capability difference represents the maximum capability difference between the formation objects.
[0054] In specific implementation, on the basis of initializing the physically heterogeneous formation task environment and generating the capability envelope of each formation object, D_ref, D_pair and D_H=D_pair are further calculated, the physical heterogeneity level H is determined accordingly, and a partially observable cooperative control environment is established.
[0055] For a formation containing N vehicles, D_ref=max_i D_i is defined to represent the maximum offset of the formation vehicles relative to the reference capability envelope B0; D_pair=max_{i<j}D(B_i,B_j) is defined to represent the maximum capability difference between the vehicles inside the formation, and the physical heterogeneity classification quantity D_H=D_pair is set. D_ref is recorded as an independent capability offset diagnostic quantity and does not participate in the physical heterogeneity classification of H0 / H1 / H2, so as to avoid misjudging a homogeneous formation with identical capabilities but entirely deviating from the reference vehicle type as strongly heterogeneous. The thresholds satisfy 0≤τ0<τ2, and in an example, the first threshold τ0=0.05 and the second threshold τ2=0.50. The physical heterogeneity classification is as follows: H0: D_H≤τ0, which is the homogeneous capability baseline.
[0056] H1: τ0<D_H≤τ2, which is weak to moderate heterogeneity.
[0057] H2: D_H>τ2, which is strong heterogeneity.
[0058] In an example, the capability envelope of the reference vehicle is taken as B0=(7.0,7.0,28), the capability envelope of the truck is (3.0,4.0,20), and the capability envelope of the sports car is (8.0,9.0,35), of the sports car and the truck is approximately 0.679, which is greater than 0.50, so the sequence of "sports car, truck, sedan, sports car" belongs to H2.
[0059] In other examples, the above classification thresholds τ0 and τ2 can be adjusted according to the vehicle type library and the safety level, for example, τ0 is set to 0.03~0.10, and τ2 is set to 0.40~0.60; as long as D_ref and D_pair are still calculated from the capability envelopes, the internal heterogeneity of the formation is classified with D_H=D_pair, and D_ref is recorded as an independent capability offset.
[0060] In other examples, the vehicle capability envelope can be generated using different parameter generation methods: one method is to set a template based on discrete vehicle types such as cars, trucks, and sports cars; another method is to continuously sample capability parameters such as maximum acceleration, maximum deceleration, and maximum speed within a preset parameter range or probability distribution. Both methods can generate a specific capability envelope B for each vehicle. i This capability envelope is then used in state transitions, observation encoding, and action feasibility assessments. When continuous sampling is used, the sampling range, distribution parameters, and random seed are recorded.
[0061] In some embodiments, the above-described method for centralized training of distributed execution control of physical heterogeneous formations further includes: constructing a capability-aware local observation vector.
[0062] Specifically, the local observation vector includes: its own position or velocity, relative distance and relative velocity with adjacent formation objects, maximum speed proxy and action feasibility flag. The action feasibility flag is determined based on the current velocity, predicted vehicle spacing and capability envelope, and is used to enable candidate control strategies to identify or block currently unexecutable actions when outputting actions.
[0063] In one example, for the first i The vehicle generates a local observation vector, including its own position and speed, relative distance and relative speed with neighboring vehicles, normalized maximum speed features, and action feasibility indicators determined based on current speed, predicted vehicle spacing, and vehicle capability envelope. For each candidate action, the candidate actual acceleration and the predicted state at the next moment are first calculated; when the candidate acceleration is located at [ , Predicted velocity is located in [0, If the predicted net spacing is not less than the minimum safe spacing d_min, the corresponding action feasibility flag is set to 1; otherwise, it is set to 0. The candidate control strategy identifies or blocks currently unexecutable actions based on this flag.
[0064] In other examples, the local observation vector can also be a 12-dimensional vector, including the vehicle's position, velocity, and acceleration; distance from the vehicle in front, relative velocity, and relative acceleration; distance from the vehicle behind, and relative velocity; maximum acceleration, maximum deceleration, and maximum velocity; and local control feasibility indicators.
[0065] In some embodiments, the above-described physical heterogeneous formation centralized training distributed execution control method further includes: pre-defining a shared discrete action space: the shared discrete action space includes at least one of a three-valued symbolic action set and a five-valued symbolic action set, the same symbolic action is mapped to different accelerations according to the capability envelope, and the velocity and position of the formation objects are updated based on the mapping results.
[0066] In one example, the mapping rule between symbolic actions and actual state transitions is as follows: ternary symbolic action set 3 = {deceleration, maintenance, acceleration} are respectively mapped to { ,0, }; Five-value symbol action set 5 = {strong deceleration, weak deceleration, maintenance, weak acceleration, strong acceleration} are respectively mapped to { , ρ_b ,0,ρ_a , }, where 0 < ρ_a, ρ_b < 1, for example, take ρ_a = ρ_b = 0.5. Let the actual acceleration after mapping be a. i t Candidate speed is i t+ ¹=v i t +a i t Δt; when the action is feasible, v i t+ ¹= i t+ ¹, x i t+ ¹=x i t +(v i t +v i t+ ¹)Δt / 2. Predicted net spacing according to Calculation, where The length of the vehicle in front. For example, take Δt=0.1s and d_min=5 m. Inactive actions are replaced by actions with the highest policy probability in the set of active actions by hard action masking; if the set of active actions is empty, the maximum permissible deceleration is executed and the safety takeover flag is triggered.
[0067] Example 2: Based on the above embodiments, this embodiment provides an application example of a distributed execution control method for centralized training of physical heterogeneous formations.
[0068] In this example, HSDP (Heterogeneity-Stress Diagnostic Protocol) is used as the core technology. HSDP is not a single CTDE or MAPPO control algorithm, but a set of diagnostic, attribution, and deployment decision-making protocols for learning-based control strategies in physically heterogeneous vehicle formations. System inputs include vehicle capability tables, formation task parameters, candidate control strategies, training configurations, and controlled test scenario configurations; system outputs include the final strategy evaluation matrix, information ablation matrix, physical counterfactual matrix, failure mechanism attribution, deployment risk boundary, and strategy adjustment suggestions.
[0069] The protocol parameter set is defined as P=(T_task,G,Ω,M,J,Φ). Here, T_task is the target task and its parameter set, G is the candidate strategy and baseline strategy set, Ω is the controlled test space, M is the indicator set, J is the intervention operation set, and Φ is the diagnostic mapping. J includes at least three types of operations: communication disturbance application, model information ablation, and vehicle dynamics physical counterfactualization; Φ maps the evaluation matrix, information ablation matrix, physical counterfactualization matrix, and preset thresholds to diagnostic labels and deployment suggestions.
[0070] Figure 2 This demonstrates the closed-loop flow of the HSDP heterogeneous stress diagnostic protocol, from input, controlled test matrix, unified training interface, evaluation matrix, ablation matrix, diagnostic mapping, diagnostic labels to deployment recommendations. Each output of this flow retains the original metrics, threshold gating, diagnostic labels, and auditable logs. Deployment feedback can write back the training configuration and controlled test matrix.
[0071] The parameterized capability envelope and H0 / H1 / H2 are defined as follows: The capability envelope of each vehicle is defined as: Equation (1) in, For vehicles i The maximum positive acceleration amplitude, The values represent the maximum deceleration amplitude, all in m / s². The maximum speed is given in m / s. All capability parameters are positive. Given a reference vehicle capability envelope B0 = (a0, b0, v0), where a0, b0, v0 > 0, the normalized capability deviation is defined as: Equation (2) Where α, β, and γ are positive weights and satisfy α + β + γ = 1, ensuring that the hierarchical scale remains interpretable under different weight settings; D i These are dimensionless quantities. For example, α = 0.4, β = 0.4, γ = 0.2. The difference in the capacity envelope between the two vehicles is defined as: Equation (3) For a platoon containing N vehicles, define D_ref=max_i D_i, which represents the maximum offset of platoon vehicles relative to the reference capability envelope B0; define D_pair=max_{i<j}D(B_i,B_j), which represents the maximum capability difference between vehicles inside the platoon, and let the physical heterogeneity classification quantity D_H=D_pair. D_ref is recorded as an independent capability offset diagnostic quantity and does not participate in the H0 / H1 / H2 heterogeneity classification, so as to avoid misclassifying homogeneous platoons with identical capabilities that deviate entirely from the reference vehicle type as strongly heterogeneous. The thresholds satisfy 0≤τ0<τ2, for example, τ0=0.05 and τ2=0.50 are adopted.
[0072] H0: D_H≤τ0, which is the homogeneous capability baseline.
[0073] H1: τ0<D_H≤τ2, which is weak to moderate heterogeneity.
[0074] H2: D_H>τ2, which is strong heterogeneity.
[0075] Taking B0=(7.0,7.0,28), the capability envelope of the truck as (3.0,4.0,20), and the capability envelope of the sports car as (8.0,9.0,35), D(B_i,B_j) between the sports car and the truck is approximately 0.679, which is greater than 0.50. Therefore, the sequence "sports car, truck, sedan, sports car" falls into H2.
[0076] The heterogeneity classifications of H0 / H1 / H2 can be shown in the following table:
[0077] The multi-dimensional controlled test dimensions and implementation principles of HSDP are as follows: HSDP sets six controlled test dimensions: physical heterogeneity level H, platoon scale N, communication quality C, action space A, vehicle arrangement order O, and random seed index S. The above six dimensions together constitute the controlled test space Ω=Ω_H×Ω_N×Ω_C×Ω_A×Ω_O×Ω_seed. H is determined by D_H=D_pair and the thresholds τ0 and τ2; N can take 4, 6, 10 or other preset scales; each value of C includes the communication type and its numerical parameters, such as packet loss rate, fixed or random delay distribution, and observation noise intensity; A includes a ternary discrete action set and a quinary discrete action set; O represents the arrangement order of vehicles with different capabilities when they are in the position of leading vehicle, middle vehicle or following vehicle; S represents the paired random seed indices used for training and evaluation.
[0078] The aforementioned dimensions are combined according to preset values to form a structured controlled test scenario and a controlled test matrix. For univariate diagnostics, only one dimension is changed while keeping the others constant; for multi-factor interactive diagnostics, two or more dimensions are changed simultaneously to identify the coupled effects of factors such as physical heterogeneity, scale, communication, and vehicle sequence. Each candidate strategy must use the same observation fields, action interfaces, reward functions, training budgets, random seed sets, final strategy evaluation rules, and invalid execution rules; deployment must not be based solely on the best checkpoint or single-run average reward.
[0079] This example can be implemented as the following system modules and interaction logic: (1) Heterogeneous Formation Task Environment Module: Receives formation size N, target spacing, target speed, vehicle type set and vehicle arrangement order O, and generates B i Calculate D_ref, D_pair, and D_H=D_pair, determine H accordingly, and establish a partially observable cooperative control environment.
[0080] (2) Capability perception observation coding module: for the first i The vehicle generates a local observation vector, including its own position and speed, relative distance and relative speed with neighboring vehicles, normalized maximum speed features, and action feasibility indicators determined based on current speed, predicted vehicle spacing, and vehicle capability envelope. For each candidate action, the candidate actual acceleration and the predicted state at the next moment are first calculated; when the candidate acceleration is located at [ , Predicted velocity is located in [0, If the predicted net spacing is not less than the minimum safe spacing d_min, the corresponding flag is set to 1; otherwise, it is set to 0. The candidate control policy identifies or blocks currently unexecutable actions based on this flag.
[0081] The mapping rules between symbolic actions and actual state transitions are as follows: ternary action set 3 = {deceleration, maintenance, acceleration} are respectively mapped to { ,0, }; Five-value action set 5 = {strong deceleration, weak deceleration, maintenance, weak acceleration, strong acceleration} are respectively mapped to { , ρ_b ,0,ρ_a , }, where 0 < ρ_a, ρ_b < 1, for example, take ρ_a = ρ_b = 0.5. Let the actual acceleration after mapping be a. i t Candidate speed is i t+ ¹=v i t +a i t Δt; when the action is feasible, v i t+ ¹= i t+ ¹, x i t+ ¹=x i t +(v i t +v i t+ ¹)Δt / 2. Predicted net spacing according to Calculate, where, The length of the vehicle in front. For example, take Δt=0.1s and d_min=5m. Inoperable actions are replaced by the action with the highest policy probability in the set of operable actions, instead of being masked by hard actions; if the set of operable actions is empty, the maximum permissible deceleration is executed and the safety takeover flag is triggered.
[0082] In other examples, the local observation encoder can be replaced by a sensor fusion encoder, graph neural network encoder, or Transformer encoder, as long as the physical constraints of vehicle capability information are still retained and incorporated into the diagnostic protocol.
[0083] (3) Unified training module for candidate control strategies: Decentralized and CTDE strategies are trained under a unified interface. For example, IPPO is the decentralized baseline and MAPPO is the CTDE candidate strategy; the final strategy is retained after training and evaluated uniformly, and similar algorithms can be replaced.
[0084] (4) Multidimensional controlled test scenario generation module: Generates a controlled test matrix based on H, N, C, A, O and random seed index S, and saves the specific parameter values of each dimension.
[0085] (5) Final Strategy Evaluation Module: This module performs a unified evaluation of the final strategy after training, outputting the mean reward, the absolute error of the average distance between adjacent vehicle pairs, the round collision ratio, the speed variance, the training cost, the running time, the invalid run flag, the random seed sensitivity, and the normalized distance deviation (NSD). The round collision ratio is uniformly defined as the percentage of rounds in which at least one vehicle collision occurs in a fixed evaluation round.
[0086] (6) Mechanism Diagnosis and Strategy Selection Module: Reads the evaluation matrix and ablation matrix, calculates diagnostic metrics, performs threshold gating, and outputs failure mechanism labels and deployment recommendations. The diagnostic labels and deployment recommendations are shown in the table below:
[0087] Based on the above modules, the method steps include: S1: Initialize the physical heterogeneous formation mission environment and generate B i Calculate the capability offset D_ref and the internal capability difference D_pair, set D_H=D_pair, and determine H0 / H1 / H2 based on τ0 and τ2; D_ref is recorded as an independent diagnostic field.
[0088] S2: Construct a capability-aware local observation vector, write the normalized maximum velocity feature and the action feasibility flag calculated according to the candidate next time step state into the observation, and enable the candidate control strategy to identify or block currently unexecutable actions based on the flag when outputting actions.
[0089] S3: Define the shared discrete action variable u i t The three-value action set is {deceleration, maintenance, acceleration}, and the five-value action set is {strong deceleration, weak deceleration, maintenance, weak acceleration, strong acceleration}. According to the mapping rules given by the capability-aware observation coding module, actions with the same symbol are assigned based on B. i The data is mapped to actual accelerations of different magnitudes, and the vehicle speed and position are updated accordingly.
[0090] S4: Train at least one decentralized baseline policy and at least one candidate CTDE policy under a unified reward function, training steps, random seed and final policy evaluation protocol.
[0091] S5: Select H0 / H1 / H2 according to D_H=D_pair, and generate a structured controlled test scenario by combining N=4 / 6 / 10 or other scales, communication packet loss rate / delay parameters / noise intensity, action space and vehicle arrangement order; record complete B1:N, D_ref, D_pair, D_H, task parameters, communication parameters and thresholds for each scenario.
[0092] S6: Use the final strategy to evaluate in all controlled test scenarios, record the reward, absolute error of the average distance between adjacent vehicle pairs, round collision ratio, speed variance, training cost, runtime, NSD and random seed sensitivity, and avoid misjudgment caused by only reporting the best checkpoint.
[0093] S7: Perform targeted information ablation, including removing capability features from the policy network (actors), removing capability inputs from the centralized value network (critic), and shuffling capability labels only at the model input; all information ablation preserves the true B. i The vehicle state transition and controlled scenario remain unchanged. A separate counterfactual physical test of the equilibrium vehicle dynamics is performed, which alters B. iAnd H, not included in the information ablation amount A_abl^m. By performing targeted ablation, it is possible to identify whether the CTDE benefit comes from heterogeneous information with physical meaning.
[0094] S8: Calculate diagnostic metrics and form mechanistic attributions.
[0095] S9: Output deployment strategies based on preset thresholds, providing suggestions for strategy selection, training robustness, or model structure adjustment.
[0096] The reward function, diagnostic metrics, and threshold rules for this example are as follows: 1. Definition of reward function and basic quantity To avoid the natural amplification of reward values due to changes in formation size, the spacing error, relative speed, and collision event items in the reward are calculated based on the number of adjacent vehicle pairs N. 1. Normalization: Non-action request items are normalized by the number of vehicles N. Single-step reward can be set as follows: Equation (4) Equation (5) Where, d i t Let d_des be the actual net distance between the i-th vehicle and the vehicle in front at time t, and Δv be the target net distance. i t For the corresponding relative velocity; e_d t and e_v t These represent the absolute error of the average distance between adjacent train pairs and the absolute value of the average relative speed, I_form. t For a formation to satisfy the criterion: when max_{i=2,…,N}|d i t d_des|≤ε_d and max_{i=2,…,N}|Δv i t | is set to 1 if | ≤ ε_v, otherwise it is set to 0. For example, let ε_d = 2 m and ε_v = 0.5 m / s.
[0097] Δx_lead t =x_lead t+ ¹ x_lead t L_ref represents the distance the navigator moves forward in a single step, and n_col represents the preset normalized distance. t Define the normalized collision quantity c_col as the number of adjacent vehicle pairs that collide at this time step. t =n_col t / (N 1); n_inv tTo determine the number of vehicles that cannot act before hard blocking, define the normalized invalid request quantity c_inv. t =n_inv t / N. Boundary violations include candidate velocities exceeding [0, ... Candidate acceleration exceeds [ , Or the predicted net spacing is less than d_min. During deployment, no actions are allowed to enter a state transition; during training, previously blocked requests are recorded. For example, take λd=0.05, λv=0.02, b_form=5.0, λp=10.0, L_ref=500 m, λc=10.0, λinv=20.0.
[0098] 2. Standardize evaluation variables and signified utility Let the policy set be G. Besides the random seed, a reproducible controlled scenario configuration is denoted as q = (B1:N,N,C(θ_C),A,O,θ_task)∈Ω_q, where B1:N is the complete capability envelope arranged in vehicle order, H is calculated from B1:N; θ_C includes communication numerical parameters such as packet loss rate, latency or distribution parameters, and noise intensity; θ_task includes target spacing, target speed, initial state, and disturbance recovery task parameters. The complete controlled test space is Ω = Ω_q × Ω_seed; the set of paired random seed indices is... ={1,…,K} Ω_seed, the set of metrics is M. x_g,q,m^(s) represents the mean of metrics m obtained by the final policy of policy g in the complete scene q and seed index s after a fixed number of evaluation rounds. The seed index s simultaneously fixes the training initialization, the randomized environment process, and the evaluation scene sequence; the same s and evaluation rounds are used when comparing policy g with the baseline policy g0.
[0099] For any index m, define a signed utility u_m(x): when the index is larger and better, u_m(x) = x; when the index is smaller and better, u_m(x) = ... x. This transformation is only used to unify the direction of improvement and should not be used to directly add across indicators. Q_m>0 is a normalized scale determined before testing based on the acceptable range of the project, and should not be temporarily derived from the mean of this sample. For example, for the main indicators, take Q_reward=1000 reward units, Q_gap=50 m, Q_col=0.02, and Q_NSD=10 percentage points; the Q_m for other indicators should be specified in advance in the test configuration.
[0100] 3. Paired diagnostic measures (both strategies use the same random seed) Equation (6) In equation (6), δ_s^m represents the utility difference between the candidate strategy and the baseline strategy on a single pair of seeds, and Δ_ctrl^m represents the average control payoff of the K pair of seeds. g and g0 can be MAPPO and IPPO, respectively, or other CTDE strategies and the decentralized baseline.
[0101] Equation (7) In Equation (7), q_H0 and q_H2 differ only in the capability envelope configuration B1:N and the H derived from it, and q_Nmin and q_Nmax differ only in the formation size N and the vehicle replication rules determined to maintain the same vehicle composition ratio; the mission parameters, communication parameters, action set, evaluation rounds, and pair seeds remain consistent. A positive S_het^m or S_scale^m indicates performance degradation from H0 to H2 or from a smaller scale to a larger scale. The reward used for scale comparison must be normalized to all vehicle or vehicle pair related terms in Equation (4).
[0102] Equation (8) In equation (8), x_perfect^(s) and x_stress^(s) are the reward metrics for the same policy g, the same non-communication scenario configuration, and the same seed index s under perfect communication and specified disturbed communication parameters θ_C, respectively. D_comm^reward is the normalized reward degradation, with a positive value indicating degradation; Q_reward is a pre-defined reward engineering scale. For each seed, E evaluation rounds are fixed, and p_perfect^(s) and p_stress^(s) represent the proportion of rounds in which at least one collision occurs under that seed, respectively. Let p_perfect and p_stress be the average of the corresponding proportions for K seeds; therefore, Δp_comm = p_stress p_perfect, the confidence interval is based on the seed level difference p_stress^(s). p_perfect^(s) resampling. Simultaneously, in the same scene q, the safety degradation of the candidate policy relative to the baseline is defined as Δp_ctrl = p_g,q. p_g0,q, where p_g,q and p_g0,q are the mean collision ratios of the K seeds in each round, and the confidence interval is defined by the seed level difference p_g,q^(s). p_g0,q^(s) resampling.
[0103] Equation (9) In Equation (9), x_full and x_ablation are the metrics of the full capability information model and the specified information ablation model, respectively, under the same training budget, full scenario q, evaluation rounds, and paired seeds. Information ablation only changes the capability information received by the policy network or centralized value network, and does not change the true B. i State transition, action consequences, and external safety takeover rules. When shuffling capability labels, only the correspondence between the model input labels and the vehicle is shuffled. A_abl^m is calculated for each type of information ablation; a positive A_abl^m indicates performance degradation after removing or shuffling the corresponding information. Equipping vehicle dynamics will change B. i H, as a physical counterfactual, is reported via S_het^m or the independent counterfactual difference and is not included in A_abl^m.
[0104] Equation (10) In equation (10), This is the set of indices for all valid adjacent vehicle pairs, where each element is (e, t, i), and e is the evaluation round, t is the valid time step for that round, and i = 2, ..., N; N_samp = | | represents the total number of valid samples. NSD(%) directly outputs the percentage; for example, NSD=10 means that the absolute error of the average distance between adjacent vehicles is 10% of the target distance. This is consistently written as NSD=10% in both the main text and tables. This definition is naturally compatible with different round lengths.
[0105] Equation (11) In equation (11), the operator SD_{s∈ }[·] represents the sample standard deviation across K seed indices, with the denominator using K. 1; S_seed^m is the normalized result of the standard deviation relative to the preset scale Q_m. Ind(·) is the indicator function, which takes the value 1 if the condition is met, and 0 otherwise; η_m is the tack tolerance, for example, η_m = 0.01Q_m. w^m is the winning ratio of the candidate strategy on paired seeds, and is counted as 0.5 wins if |δ_s^m|≤η_m. The above two quantities are calculated separately for the pre-specified main indicators, and indicators with different dimensions are not mixed.
[0106] 4. Paired bootstrap (sampling seed) and threshold gating When calculating confidence intervals for Δ_ctrl^m, S_het^m, S_scale^m, D_comm^reward, Δp_comm, Δp_ctrl, and A_abl^m, the resampling unit is the seed-level pairwise difference: K indices are drawn with replacement from K seed indices, the mean is calculated once, and this process is repeated at least 10,000 times. The 2.5% and 97.5% quantiles are used as the 95% pairwise bootstrap confidence intervals. Resampling candidate strategies with baseline strategies, perfect communication with perturbed communication, or complete models with ablation models is not permitted.
[0107] In one example, the thresholds can be set as follows: upper limit for normalized reward degradation τr = 0.20, upper limit for collision ratio increment in communication rounds τΔcomm = 0.01, upper limit for collision ratio increment in candidate policies relative to the baseline rounds τΔctrl = 0.005, upper limit for absolute collision ratio in perturbed conditions τc = 0.02, upper limit for win ratio τw = 0.50, upper limit for NSD τNSD = 10%, and upper limit for normalized seed sensitivity τs^m = 0.20. The thresholds for heterogeneity degradation, scale degradation, and information ablation are denoted as τhet^m, τscale^m, and τabl^m, respectively, and are all taken as 0.10Q_m in the example; all Q_m and thresholds must be written into the test configuration before training.
[0108] The invalid run criteria are: failure to generate a loadable final policy file; NaN / Inf errors during training or evaluation; abnormal exit of the simulation process; completion of less than 90% of the predetermined evaluation rounds; or the safety takeover flag lasting longer than 5% of the total evaluation time steps. Invalid runs must not be silently deleted from the mean; they should be retained in the evaluation matrix and directly trigger retraining or deployment blocking. The final policy is the last policy after reaching the preset number of training steps and is not replaced by a post-selected best checkpoint.
[0109] The conclusion "Candidate CTDE strategy is more stable than decentralized baseline" is output only when the pre-specified main performance indicators (e.g., reward and spacing error) have Δ_ctrl^m > 0, their 95% pairwise bootstrap confidence intervals are all greater than 0, w^m > τw, and no invalid runs and all security gates are triggered. In communication disruption scenarios, if any of the following conditions are met: D_comm^reward > τr, Δp_comm > τΔcomm, or p_stress > τc, then "Communication fragile" is output, and deployment is blocked or restricted under this communication condition; in any scenario, if Δp_ctrl > τΔctrl, NSD > τNSD, or an invalid run occurs, then "Deployment boundary exceeded" is output, and deployment is blocked or restricted. "OR" logic is used. For any pre-specified primary metric, if S_het^m > τhet^m, output "Heterogeneity Sensitive"; if S_scale^m > τscale^m, output "Scale Sensitive"; if A_abl^m of any information ablation > τabl^m, output "Effective Dependence on Capability Information"; if S_het^m > τhet^m and A_abl^m of all information ablation ≤ τabl^m, further output "Insufficient Utilization of Capability Information". If S_seed^m > τs^m, output "Seed Sensitive", and re-evaluate after increasing the number of paired seeds or adjusting the model. When T_task is a perturbation recovery task, if the upper bound of the 95% confidence interval of the difference between the paired signed utility of any primary metric of the candidate policy relative to the task-specific baseline is less than 0, output "Negative Transfer".
[0110] In some cases, diagnostic thresholds can be pre-configured based on security level, road type, or product strategy. High-security scenarios can reduce the normalized reward degradation cap τr, the communication round collision ratio increment cap τΔcomm, the candidate strategy round collision ratio increment cap τΔctrl, the absolute round collision ratio cap τc, the NSD cap τNSD, the heterogeneous degradation threshold τhet^m, the scale degradation threshold τscale^m, the information ablation threshold τabl^m, or the seed sensitivity cap τs^m to more strictly block deployments or require reassessment.
[0111] The method in this embodiment achieves a closed-loop protocol instead of a single CTDE algorithm. It introduces the vehicle capability envelope into the state transition execution mechanism, making heterogeneity reflected in action consequences rather than vehicle type labels. A unified training and final policy evaluation interface is adopted, with different algorithms using the same observation fields, action interfaces, reward functions, training steps, random seed sets, final policy evaluation rules, and invalid run judgment rules, avoiding misjudgments based on optimal checkpoints or single-run average rewards. It eliminates the dependence on capability information by removing policy network capability features, removing centralized value network capability inputs, and shuffling capability labels only at the model input; balancing vehicle dynamics is used to change B...i Physical counterfactual information (PfD) is reported separately from information ablation, forming an information ablation and physical counterfactual attribution matrix. Labels such as vulnerable CTDE gains, communication vulnerability, deployment boundary exceeding limits, heterogeneity sensitivity, scale sensitivity, effective reliance on capability information, insufficient utilization of capability information, seed sensitivity, and negative transfer are transformed into recommendations such as deployable, requiring robust training, requiring strategy change, requiring model adjustment, requiring the addition of random seeds, or prohibiting deployment. This achieves the transformation from diagnostic labels to deployment strategies. HSDP can identify risks masked by traditional average reward assessments. For example, when using 50 seed indices to form 50 pairs of MAPPO / IPPO results, although the average MAPPO reward increases by 216.41, the confidence interval crosses 0, the win rate is below 50%, and NSD exceeds limits, thus triggering vulnerable gains and deployment boundary exceeding limits labels.
[0112] Example 3: Based on the above embodiments, this embodiment provides another application example of the distributed execution control method for centralized training of physical heterogeneous formations.
[0113] Under the conditions of N=4, H2 strong heterogeneity, and perfect communication, 50 seed indices were used; each index was used for training and unified evaluation of MAPPO and IPPO respectively, resulting in 50 pairs of results. MAPPO improved the average reward by 216.41 and reduced the average interval error by 8.55 m compared to IPPO; however, the 95% pairwise bootstrap confidence interval for the reward was […]. [323.16,784.53], the confidence interval for the signed improvement in the spacing error is [ [61.41, 42.24], the pairwise win rates of MAPPO in terms of reward, spacing error, and round collision ratio are 46.0%, 40.0%, and 44.0%, respectively. The round collision ratio increases from 0.0161 for IPPO to 0.0176 for MAPPO, hence Δp_ctrl = 0.0015; the average NSD of MAPPO is 17.56%, exceeding τNSD = 10%. Although Δp_ctrl does not exceed the threshold of 0.005, the confidence interval crosses 0, the win rate does not exceed 0.50, and the NSD exceeds the limit. HSDP outputs the labels "Vulnerable CTDE Gain" and "Deployment Boundary Exceeded".
[0114] Under communication degradation training conditions, the average reward of the packet loss-aware MAPPO relative to the nominally trained model in the same packet loss-controlled scenario is determined by... Improved from 919.49 687.93, corresponding to a robust training intervention gain of 231.56; the average reward of the delay-aware MAPPO model relative to the nominally trained model in the same delay-controlled scenario decreased by 566.88, with a 95% pairwise confidence interval of []. 928.27, [224.11], the average spacing error increased by 64.28 m. This set of comparisons is used to determine whether the robust training intervention is effective; during deployment, D_comm^reward, Δp_comm and p_stress of equation (8) must still be calculated for each final strategy under perfect communication and disturbed communication conditions.
[0115] When T_task performs a perturbation recovery task, the perturbation type, magnitude, application time, and recovery window are written into θ_task, and the maximum perturbation-induced spacing error and the spacing error integral AUC are added to M, where AUC = Σ_t e_d^tΔt. In the embodiment, MAPPO reduces the round collision ratio from 0.0268 to 0.0101, but the final reward is changed from... 658.03 dropped to The mean spacing error increased from 91.57 m to 173.67 m after the maximum perturbation. Both the spacing error and the spacing AUC deteriorated significantly after the maximum perturbation, so the label "negative migration" was output.
[0116] The task feasibility verification for local feedback control uses a separate feasibility reference threshold τfeas = 13%. Under the same B1:N, initial state, and task parameters, the average NSD of H0 / H1 / H2 is less than 13%, and the maximum spacing error is less than 4m, indicating that the task is not physically impossible. τfeas is not used as the deployment threshold for the learning strategy; the final deployment still follows the stricter τNSD = 10%. If local feedback control is used as a deployment alternative, it must also be readjusted and pass the 10% threshold.
[0117] The method / module of this embodiment can be integrated into vehicle-to-everything (V2X) or vehicle-road-cloud (V2X) simulation testing platforms as a module for training, evaluating, and diagnosing risks before deployment of heterogeneous platooning control strategies. It can also serve as a control strategy selection tool in autonomous driving simulation evaluation services, providing suggestions for different fleet compositions, communication quality, vehicle arrangement order, and platoon size, including those suggesting deployment is feasible, those requiring robust training, those requiring strategy replacement, or those prohibiting deployment. It can be used independently for V2X communication reliability assessment, mapping packet loss, latency, and noise to normalized reward degradation, communication round collision ratio increment, absolute round collision ratio under disturbed conditions, candidate strategy relative to baseline round collision ratio increment, NSD, and deployment threshold. When integrated with V2X simulation testing platform products, it can simultaneously record the platform version, code commit number, hardware asset number, random seed, final strategy evaluation report, and diagnostic tags. HSDP can identify deployment risks such as confidence intervals crossing 0, seed win rates below 50%, and NSD exceeding limits when the MAPPO average reward appears to improve. It can also distinguish between the positive intervention benefits of loss-robust training and the negative transfer from delayed training, requiring the final policy to be separately gated through perfect / disrupted communication pairs. Furthermore, it can be reused in multi-agent collaborative control platforms, providing cross-scenario physical heterogeneity diagnosis, robustness assessment, and deployment boundary identification capabilities for CTDE methods.
[0118] Example 4: Based on the above embodiments, this embodiment provides another application example of the distributed execution control method for centralized training of physical heterogeneous formations.
[0119] Vehicle types include sedans, trucks, and sports cars. Sedan has a maximum acceleration of 3.0 m / s², a maximum deceleration of 5.0 m / s², and a maximum speed of 30 m / s; truck has a maximum acceleration of 1.5 m / s², a maximum deceleration of 3.0 m / s², and a maximum speed of 25 m / s; sports cars have a maximum acceleration of 8.0 m / s², a maximum deceleration of 9.0 m / s², and a maximum speed of 40 m / s.
[0120] Under a four-vehicle strong mismatch configuration (sports car, truck, sedan, sports car), a unified final strategy evaluation can be performed on ACC, IPPO, QMIX, MADDPG, and MAPPO. Prototype simulation results show that MAPPO outperforms other baselines in terms of reward, spacing error, and collision rate; from homogeneous H0 to strong mismatch H2, all methods degenerate, indicating that physical heterogeneity poses a substantial control challenge. Before formal submission, it is recommended to archive the simulation platform, code version, hardware configuration, random seed, and statistical intervals.
[0121] By incorporating vehicle capability characteristics such as maximum acceleration, maximum deceleration, and maximum speed into the local observation of each vehicle, decentralized executors can select actions based on the vehicle's physical envelope, thus achieving decentralized observation of capability perception.
[0122] The simulation platform, code version, and hardware configuration are as follows: 1. Simulation Platform: The vehicle-road-cloud simulation platform is used. This platform provides a physically heterogeneous vehicle platooning task environment, supporting the configuration of capability parameters for car, truck, sports car and other vehicle types; the simulation step size Δt=0.1 s, and vehicle dynamics execute state transitions according to maximum acceleration, maximum deceleration and speed limit.
[0123] 2. Code version: The prototype code version is HSDP-CTDE-Prototype v0.1.0, which includes modules such as environment, train, evaluate, stress_tests, ablation, and diagnosis.
[0124] 3. Hardware configuration: The training device is an x86_64 / Linux GPU workstation. It is recommended to configure it with an 8-core or higher CPU, 32GB or more of memory, and an NVIDIA RTX 3090 / 4090 CUDA GPU. Small-scale evaluations with N=4 / 6 can be run in a CPU environment, while training with N=10 or more random seeds is recommended to use GPU acceleration.
[0125] 4. Operation Records: For each H0 / H1 / H2 group, formation size, and communication disturbance experiment, the random seed, training steps, model checkpoints, and final policy evaluation results are recorded for reproduction, auditing, and subsequent platform integration.
[0126] Example 5: Another embodiment of this application relates to a distributed execution control device for centralized training of physical heterogeneous formations. The implementation details of this embodiment's distributed execution control device for centralized training of physical heterogeneous formations are described below. The following implementation details are provided for ease of understanding and are not essential for implementing this solution. A schematic diagram of this embodiment's distributed execution control device for centralized training of physical heterogeneous formations can be seen as follows: Figure 3 As shown, it includes a heterogeneous formation task environment module 310, a candidate control strategy unified training module 320, a multi-dimensional controlled test scenario generation module 330, a final strategy evaluation module 340, and a mechanism diagnosis and strategy selection module 350.
[0127] Heterogeneous formation mission environment module 310 is used to initialize the physical heterogeneous formation mission environment and generate the capability envelope of each formation object; The candidate control policy unified training module 320 is used to train candidate control policies under unified training conditions. The candidate control policies include decentralized baseline policies and CTDE candidate policies. The multidimensional controlled test scenario generation module 330 is used to generate structured controlled test scenarios based on the target physical heterogeneity level, formation size, communication quality, predefined shared discrete action space, and formation object arrangement order. The final policy evaluation module 340 is used to evaluate the trained final policy in various controlled test scenarios. The mechanism diagnosis and strategy selection module 350 is used to diagnose the final strategy and generate diagnostic labels based on the results of directed information ablation and physical counterfactual testing, and to generate the deployment strategy of the final strategy based on the diagnostic labels. Directed information ablation is performed while keeping the capability envelope, state transitions, and controlled test scenarios of the formation objects unchanged. Physical counterfactual testing of equilibrium dynamics is performed while changing the capability envelope and physical heterogeneity level of the formation objects.
[0128] The specific implementation methods of the above modules can be found in the aforementioned embodiments, and will not be repeated here.
[0129] It is worth mentioning that all modules involved in this embodiment are logical modules. In practical applications, a logical unit can be a physical unit, a part of a physical unit, or a combination of multiple physical units. Furthermore, to highlight the innovative aspects of this application, this embodiment does not introduce units that are not closely related to solving the technical problems proposed in this application; however, this does not mean that other units are absent in this embodiment.
[0130] In some optional embodiments, the physical heterogeneous formation centralized training distributed execution control device can implement the physical heterogeneous formation centralized training distributed execution control method described in any of the above embodiments.
[0131] Example 6: Another embodiment of this application relates to an electronic device, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to execute the physical heterogeneous formation centralized training distributed execution control method in the above embodiments.
[0132] In this embodiment, the memory and processor are connected via a bus, which can include any number of interconnected buses and bridges, connecting various circuits of one or more processors and the memory together. The bus can also connect various other circuits, such as peripheral devices, voltage regulators, and power management circuits, which are well known in the art and therefore will not be further described in this embodiment. A bus interface provides an interface between the bus and the transceiver. The transceiver can be a single element or multiple elements, such as multiple receivers and transmitters, providing a unit for communicating with various other devices over a transmission medium. Data processed by the processor is transmitted over the wireless medium via an antenna, which further receives data and transmits it to the processor.
[0133] The processor manages the bus and general processing, and also provides various functions, including timing, peripheral interfaces, voltage regulation, power management, and other control functions. Memory is used to store data used by the processor during operation.
[0134] Example 7: Another embodiment of this application relates to a computer-readable storage medium storing a computer program. When executed by a processor, the computer program implements the above-described embodiment of the centralized training distributed execution control method for physically heterogeneous formations.
[0135] That is, those skilled in the art will understand that all or part of the steps in the methods of the above embodiments can be implemented by a program instructing related hardware. This program is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.
[0136] Those skilled in the art will understand that the above embodiments are specific embodiments for implementing this application, and in practical applications, various changes can be made to them in form and detail without departing from the spirit and scope of this application.
Claims
1. A physical heterogeneous platoon centralized training distributed execution control method, characterized in that, include: Initialize the physical heterogeneous formation task environment and generate the capability envelope for each formation object; Candidate control policies are trained under uniform training conditions, including decentralized baseline policies and CTDE candidate policies. A structured, controlled test scenario is generated based on the target physical heterogeneity level, formation size, communication quality, predefined shared discrete action space, and formation object arrangement order. The final policy obtained from training is evaluated in each controlled test scenario; While keeping the capability envelope, state transition and controlled test scenario of the formation objects unchanged, perform directional information ablation, and perform physical counterfactual tests of equilibrium dynamics while changing the capability envelope and physical heterogeneity level of the formation objects; Based on the results of targeted information ablation and physical counterfactual testing, the final strategy is diagnosed and diagnostic labels are generated. The deployment strategy for the final strategy is generated based on the diagnostic labels.
2. The physical heterogeneous platoon centralized training distributed execution control method of claim 1, wherein, Also includes: The capability offset and internal capability difference are calculated. The capability envelope includes the maximum positive acceleration amplitude, the maximum deceleration amplitude, and the maximum velocity. The capability offset represents the maximum capability difference of the formation object relative to the reference capability envelope, and the internal capability difference represents the maximum capability difference between formation objects.
3. The physical heterogeneous platoon centralized training distributed execution control method of claim 2, wherein, Also includes: Select the target physical heterogeneity level from the candidate physical heterogeneity levels based on the internal capability difference; Selecting a target physical heterogeneity level from candidate physical heterogeneity levels based on the internal capability difference includes: if the internal capability difference does not exceed a first threshold, then selecting a first physical heterogeneity level as the target physical heterogeneity level; if the internal capability difference is greater than the first threshold but does not exceed a second threshold, then selecting a second physical heterogeneity level as the target physical heterogeneity level; if the internal capability difference exceeds the second threshold, then selecting a third physical heterogeneity level as the target physical heterogeneity level, wherein the first threshold is ≥0 and less than the second threshold.
4. The physical heterogeneous platoon centralized training distributed execution control method of claim 1, wherein, The physical heterogeneous formation mission environment includes formation size, target spacing, target speed, set of formation object types, and formation object arrangement order.
5. The physical heterogeneous platoon centralized training distributed execution control method of claim 1, wherein, Also includes: A capability-aware local observation vector is constructed, which includes: its own position or speed, relative distance and relative speed with adjacent formation objects, maximum speed proxy, and action feasibility flag. The action feasibility flag is determined based on the current speed, predicted vehicle spacing, and capability envelope, and is used to enable candidate control strategies to identify or block currently unexecutable actions when outputting actions.
6. The physical heterogeneous platoon centralized training distributed execution control method of claim 1, wherein, Also includes: A predefined shared discrete action space is defined, which includes at least one of a three-valued symbolic action set and a five-valued symbolic action set. The same symbolic action is mapped to different accelerations according to the capability envelope, and the velocity and position of the formation objects are updated based on the mapping results.
7. The physical platooning centralized training distributed execution control method of claim 1, wherein, Train candidate control policies under uniform training conditions, including: Candidate control policies are trained using the same observation fields, action interfaces, reward functions, training steps, random seed sets, final policy evaluation rules, and invalid run judgment rules.
8. A physical heterogeneous platoon centralized training distributed execution control apparatus, characterized by, include: The heterogeneous formation mission environment module is used to initialize the physical heterogeneous formation mission environment and generate the capability envelope of each formation object. A unified training module for candidate control strategies is used to train candidate control strategies under unified training conditions. The candidate control strategies include decentralized baseline strategies and CTDE candidate strategies. The multidimensional controlled test scenario generation module is used to generate structured controlled test scenarios based on the target physical heterogeneity level, formation size, communication quality, predefined shared discrete action space, and formation object arrangement order. The final policy evaluation module is used to evaluate the trained final policy in various controlled test scenarios. While keeping the capability envelope, state transition and controlled test scenario of the formation objects unchanged, perform directional information ablation, and perform physical counterfactual tests of equilibrium dynamics while changing the capability envelope and physical heterogeneity level of the formation objects; The mechanism diagnosis and strategy selection module is used to diagnose the final strategy and generate diagnostic labels based on the results of targeted information ablation and physical counterfactual testing, and to generate the deployment strategy of the final strategy based on the diagnostic labels.
9. An electronic device, comprising: include: At least one processor; as well as, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, which, when executed by the at least one processor, enables the at least one processor to perform the physical heterogeneous formation centralized training distributed execution control method as described in any one of claims 1 to 7.
10. A computer readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the physical heterogeneous formation centralized training distributed execution control method as described in any one of claims 1 to 7.
