Air-ground collaborative sensing and communication integrated resource allocation method and system in mixed traffic scenario

Abstract

This invention proposes a resource allocation method and system for air-ground collaborative sensing and computing in mixed traffic scenarios, belonging to the field of wireless communication technology. The method constructs an ABS-assisted air-ground collaborative multi-layer offloading architecture, dynamically dividing the computing task into three parallel paths: local computing, direct RSU offloading, and relay to a low-load RSU via ABS. Simultaneously, the total vehicle power is dynamically allocated to three parts: communication, sensing, and computing, achieving deep integration of sensing and computing functions. A joint optimization problem is established with the objectives of maximizing QoE and minimizing task latency. A PPO-based reinforcement learning algorithm is employed, introducing generalized advantage estimation and attenuation entropy regularization mechanisms to jointly solve the offloading ratio and power allocation, obtaining the optimal task offloading ratio and power allocation results. Simulation results show that compared with various benchmark strategies, this invention achieves significant improvements in system latency and sensing accuracy, verifying the effectiveness of the proposed method.

Description

Technical Field

[0001] This invention belongs to the field of wireless communication technology, specifically relating to a method and system for resource allocation that integrates communication, sensing, and computing in a mixed traffic scenario. Background Technology

[0002] With the rapid development of autonomous driving technology and Intelligent Transportation Systems (ITS), the demand for real-time communication between vehicles and between vehicles and infrastructure is increasing, especially in applications such as cooperative perception and autonomous driving cooperative control, where vehicles need to handle massive amounts of computationally intensive and latency-sensitive tasks. However, limited by the computing power and energy constraints of onboard terminals, offloading tasks to roadside units (RSUs) for edge computing has become an effective solution. Traditional research on vehicle-to-everything (V2X) resource allocation mainly focuses on offloading single communication or computing tasks, which is insufficient to meet the future business needs of deep integration of sensing, communication, and computing.

[0003] In current practical applications, urban traffic is in a transitional phase where autonomous vehicles and non-connected vehicles coexist for a long time. Traditional ground networks face many challenges in dealing with this mixed traffic flow: on the one hand, traffic flow exhibits strong randomness and tidal effects, causing uneven load on roadside units during peak hours or congested periods, resulting in a surge in task processing latency; on the other hand, existing resource allocation mechanisms often fail to fully consider the deep coupling between communication, sensing, and computing resources, making it difficult to achieve optimal overall system task processing efficiency while ensuring sensing accuracy. Furthermore, the vehicle-to-everything (V2X) environment is highly time-varying, making it difficult for traditional optimization algorithms to quickly provide optimal resource allocation decisions.

[0004] To address these challenges, integrated sensing, communication, and computing (ISCC) technology and air-ground cooperative architectures assisted by aerial base stations (ABS) have gradually become research hotspots in recent years. By introducing aerial nodes to provide relay support and utilizing ISCC technology to integrate communication and sensing functions on the same hardware resources, the load pressure on ground-based RSUs can be alleviated to some extent and the service coverage can be expanded.

[0005] However, existing air-ground collaborative architectures still have significant limitations in practical applications: existing ABS-assisted solutions mostly focus on simple coverage enhancement and lack a multi-layered offloading framework that can dynamically relay tasks to remote low-load RSUs based on the real-time load status of ground RSUs and take advantage of the ABS line-of-sight link; at the same time, when dealing with the multi-dimensional continuous action space of synesthesia, existing reinforcement learning methods often struggle to balance perception accuracy and load balancing stability, and are prone to getting trapped in local optima.

[0006] Patent application CN119562356A proposes a resource allocation method in a sensor-computing integrated vehicle network system. It primarily addresses the conflict between latency-sensitive perception services and high-volume communication services. By jointly considering node association, power, and subcarrier block allocation, it constructs an optimization problem among perception latency, computational processing, network stability, and communication rate, transforming it into a multi-agent reinforcement learning problem to obtain optimal resource allocation decisions. These decisions include node selection, power allocation, V2I communication bandwidth allocation, V2V communication bandwidth allocation, and radar perception bandwidth allocation. Patent application CN121057007A provides a sensor-computing integrated resource allocation method for stochastic multi-target perception. It constructs a sensor-computing integrated system model for stochastic multi-target perception; based on the system model, it constructs perception task models, perception utility models, communication task models, task queue models, and computation task models; based on these models, it constructs a joint optimization problem with the objective of maximizing the overall energy efficiency of the model; and it uses a Lyapunov-guided multi-agent deep reinforcement learning algorithm with fused graph attention networks to solve the joint optimization problem, obtaining the resource allocation results. However, none of the aforementioned patents considered introducing ABS to construct a multi-layered unloading topology for air-ground collaboration, and they did not differentiate between the task priorities of autonomous vehicles, making it difficult to simultaneously ensure the latency constraints of high-priority tasks and the overall perception quality of the system in dynamic traffic scenarios.

[0007] How to achieve integrated resource allocation of sensing and computing in the context of mixed transportation, and how to achieve efficient joint scheduling of multi-dimensional resources for computing tasks of different priorities, is a key technical challenge that urgently needs to be solved in the field of vehicle networking. Summary of the Invention

[0008] To address the problems existing in current technologies, this invention provides a sensor-computing integrated resource allocation method for air-ground collaboration in hybrid traffic scenarios. Addressing the coexistence of autonomous and non-connected vehicles in current urban traffic scenarios, and the spatiotemporal load unevenness faced by RSUs (Resource Units), this invention constructs an ABS-assisted air-ground collaborative architecture and a sensor-computing integrated resource model to dynamically offload tasks. This invention employs a proximal policy optimization (PPO)-based algorithm, introducing generalized advantage estimation and entropy regularization mechanisms to jointly optimize the task offloading ratio and power allocation. Under the premise of satisfying the maximum tolerable latency, it minimizes task processing latency and maximizes user experience quality, thereby improving the system's resource allocation performance and service quality.

[0009] To achieve the above objectives, in a first aspect, the present invention provides a method for resource allocation integrating sensing and computation in a hybrid traffic scenario, comprising the following steps: Based on the integrated sensing, sensing, and computing (ISCC) model, a power-constrained sensing model, a millimeter-wave beamforming-based air-ground cooperative communication model, and a multi-layer offloading architecture-based computing model are constructed for the air-ground cooperative hybrid traffic scenario. Based on the constructed perception model, air-ground cooperative communication model, and computational model, a system utility maximization problem is established. By jointly optimizing the task offloading strategy and power allocation scheme, the overall system utility is maximized while satisfying the latency constraints of each task. The system utility maximization problem is modeled as a Markov decision process. The design of the reinforcement learning reward function is aligned with the goal of maximizing system utility. The reinforcement learning reward function is designed with weights based on task priority, perceived quality, and latency satisfaction, and penalizes task timeout behavior. By combining the reinforcement learning reward function, the Proximal Policy Optimization (PPO) algorithm is used to solve the Markov decision process. Through iterative training using the Actor-Critic dual-network architecture, the optimal task offloading ratio and power allocation results are obtained.

[0010] Furthermore, the air-ground cooperative hybrid transportation scenario includes autonomous vehicles (AVs) with integrated sensing and computing capabilities, non-commercial vehicles (NCVs), multiple roadside units (RSUs), and an airborne base station; the computing tasks generated by the AVs have a three-level cooperative processing path: local processing, direct offloading to the near-end RSU, and offloading to the far-end low-load RSU via ABS relay. Randomly distributed within the two-way lane area A number of autonomous vehicles are deployed, with one RSU fixed at each of the four corners of the area and a height of [missing information] at the center of the area. The airborne base station, with non-connected vehicles randomly distributed within the area as dynamic sensing targets for AV; The three-level collaborative processing path specifically includes: the local processing path, where the task is directly executed by the AV vehicle terminal; the direct offloading path, where the task is directly offloaded and sent to the RSU near the AV for edge computing; and the relay offloading path, where when the load of the near-end RSU exceeds a preset threshold, the task is relayed to other remote RSUs in the area with low load via the ABS relay.

[0011] Furthermore, in the power-constrained perception model, the perception accuracy of an autonomous vehicle is determined by the perception power, specifically:

[0012] in, For the first The perception accuracy of the AV unit. This is the sensor sensitivity coefficient. For the first The sensing power allocated to each AV unit.

[0013] Furthermore, the computational model based on the multi-layer offloading architecture is as follows: autonomous vehicles Real-time generated computational tasks consist of triples Define, where For task data volume, The required computation period for the task The maximum tolerable latency for the task; To reflect the different sensitivity of tasks to latency, the system introduces a task priority classification: safety-critical tasks with strict low-latency requirements and non-critical tasks. Employing a sequential partial unloading strategy, individual tasks are... Dynamically divided into local calculation ratios Direct uninstallation ratio and relay unloading ratio And it satisfies the completeness constraint:

[0014] in, All are non-negative values.

[0015] Furthermore, the objective function constructed for the system utility maximization problem is: The system utility maximization problem is modeled as follows:

[0016] in and To balance the weighting coefficients for perceived quality and latency performance, constraints are imposed. Ensure that each task is completed within the deadline; and Ensure the legality of the uninstallation ratio; Ensure that the total compute load offloaded to RSU does not exceed the system's aggregate computing power limit. ,in express Available computing resources within a unit time window; and To ensure the non-negativity of power allocation and the total amount constraint, For autonomous vehicles The quality of service experience For the task Normalized delay.

[0017] Furthermore, the Markov decision process consists of quaternions. Define, where: state space Time step status It is composed of global context information and the private features of each vehicle, specifically including the location of RSU and ABS, the real-time load of each RSU, the location of each AV, the distance vector from each AV to RSU and ABS, the current task attributes of each AV and the task queue statistical features. Action space The actions of each AV It includes two components: offloading action and power allocation action; the offloading action includes local calculation ratio, direct connection offloading ratio and relay offloading ratio, and the power allocation action includes the allocation ratio of sensing power, communication power and local calculation power. award Immediate reward signals aligned with the goal of maximizing system utility; Discount factor : Discount factor used to calculate cumulative returns.

[0018] Furthermore, the reinforcement learning reward function is time step Instant rewards Defined as the weighted sum of the current mission rewards for all vehicles, specifically:

[0019] in Weighting coefficients are applied to task priorities, corresponding to safety-critical tasks. Non-critical tasks correspond , and A coefficient for balancing perceived quality and latency performance. For autonomous vehicles The quality of service experience For the task Normalized delay; The time-delay excitation term is expressed as:

[0020] in This is the excitation amplitude coefficient; For the speed bonus item, the expression is:

[0021] To prevent the strategy from getting stuck at a local optimum that is exactly in time; The low-perception power penalty term is expressed as follows:

[0022] in This is the lower limit threshold for sensing accuracy. This is the penalty coefficient.

[0023] Furthermore, combining the reinforcement learning reward function, the Proximal Policy Optimization (PPO) algorithm is used to solve the Markov decision process. Through iterative training using an Actor-Critic dual-network architecture, the optimal task offloading ratio and power allocation results are obtained, including: The advantage function is calculated using the generalized advantage estimation (GAE) method, and its expression is:

[0024] in As a discount factor, GAE attenuation coefficient, For time-series difference errors, GAE calculates the error by episode, and after the calculation is completed, the dominance value is batch normalized. The Actor network updates by maximizing the pruned agent objective function, which is expressed as:

[0025] in This represents the probability ratio between the old and new strategies. For the truncation range hyperparameter, The entropy of the current strategy, The entropy coefficient is linearly decaying with the training progress; The Critic network updates independently by minimizing a truncated value loss function, the expression of which is:

[0026] in For Smooth L1 loss, To achieve the target return value, For the truncated value estimate; After multiple rounds of iterative training until reward convergence, the optimal joint offloading and power allocation strategy is obtained. After training, the task allocation ratio and power allocation scheme of all vehicles in the scenario are output in real time based on global observation, and centralized joint scheduling is completed.

[0027] Secondly, the present invention provides an integrated sensor-computing resource allocation system for air-ground collaboration in mixed traffic scenarios, comprising: Basic model construction module: Based on the integrated sensing, sensing and computing (ISCC) model for air-ground cooperative hybrid traffic scenarios, a power-constrained perception model, a millimeter-wave beamforming-based air-ground cooperative communication model, and a multi-layer offloading architecture-based computing model are constructed to complete the construction of the integrated sensing, sensing and computing basic model. The optimization problem construction module uses the existing perception model, air-ground cooperative communication model, and computational model to establish a system utility maximization problem. By jointly optimizing the task offloading strategy and power allocation scheme, the overall system utility is maximized while satisfying the latency constraints of each task. The system utility maximization problem is modeled as a Markov decision process. Reward function design module: used to design reinforcement learning reward functions aligned with the goal of maximizing system utility, integrating weighted and penalized mechanisms based on task priority, perceived quality, and latency constraints; The strategy solving module is used to solve the Markov decision process by combining the reinforcement learning reward function and the proximal policy optimization (PPO) algorithm. Through iterative training using the Actor-Critic dual network architecture, the optimal task offloading ratio and power allocation scheme are obtained, and the joint scheduling of the integrated sensory computing resources is completed.

[0028] Thirdly, the present invention can also provide an integrated sensing and computing system for air-ground coordination in mixed traffic scenarios, within a square area enclosed by roads in a city, including... An autonomous vehicle with integrated sensing and computing capabilities is randomly distributed on a two-way lane and maintains a constant speed. A Remote Unit (RSU) is fixedly deployed at each of the four corners of the inner side of the road. A height of [missing information] is introduced at the center of a square area. The ABS provides signal coverage and task relay services. Non-connected vehicles are also randomly distributed within the square area. These serve as dynamic perception targets for autonomous vehicles and do not participate in the communication network. The location information of non-connected vehicles needs to be obtained in real time by autonomous vehicles through onboard sensors. The above-mentioned integrated resource allocation method of air-ground collaboration in the mixed traffic scenario is used for resource allocation.

[0029] Compared with existing technologies, this invention has at least the following beneficial effects: This invention proposes an integrated air-ground communication, sensing, and computing resource allocation method for mixed traffic scenarios. By introducing an ABS-assisted air-ground collaborative architecture, a multi-layer offloading model is established, covering vehicle local, directly connected near-end RSUs, and remote low-load RSUs relayed via ABS. This effectively expands the service range and balances the load pressure among RSUs. Simultaneously, an integrated communication, sensing, and computing resource model is constructed, dynamically dividing the transmit power into three parts: communication, sensing, and computing. Combining channel status and RSU load feedback, the high-altitude coverage characteristics of ABS are used to achieve collaborative scheduling of task traffic among different nodes. Furthermore, the task offloading ratio and power allocation are obtained through a PPO-based algorithm, minimizing task processing latency and maximizing user experience quality while meeting the maximum tolerable latency. Attached Figure Description

[0030] Figure 1 This is a schematic diagram of a scenario disclosed in an embodiment of the present invention.

[0031] Figure 2 This is a flowchart of the integrated sensor-computer resource allocation method for air-ground collaboration in a mixed traffic scenario proposed in this invention.

[0032] Figure 3 This invention provides a comparison of the proposed method with other benchmark strategies in terms of latency metrics.

[0033] Figure 4 This is a comparison of the method proposed in this invention with other benchmark strategies in terms of QoE metric. Detailed Implementation

[0034] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0035] refer to Figure 1 and Figure 2 This embodiment discloses a method for resource allocation integrating sensing and computing in a hybrid traffic scenario involving air-ground collaboration, specifically including the following steps: Step 1: Construct an air-ground cooperative hybrid transportation scenario model based on ISCC This embodiment considers a scenario such as... Figure 1 shown A square area. The scene contains... Autonomous vehicles (AVs) with integrated sensing and computing capabilities are randomly distributed along a two-way road and maintain a constant speed. One Remote Unit (RSU) is fixedly deployed at each of the four corners of the inner side of the square road. Furthermore, a height of [missing information] is introduced at the very center of the square area. The ABS provides wide-area signal coverage and mission relay services. Non-connected vehicles (NCVs) are also randomly distributed within the area, serving as dynamic perception targets for the AV. These NCVs do not participate in the communication network, but their location information must be acquired in real-time by the AV through onboard sensors to ensure driving safety.

[0036] In the above scenario model, the tasks generated by AV have three optional collaborative processing paths: 1. Local processing path: the task is directly executed by the vehicle's onboard terminal; 2. Direct offloading path: the task is directly offloaded and sent to a nearby RSU for edge computing; 3. Relay offloading path: when the near-end RSU is overloaded due to traffic tidal effects or a surge in random tasks, the task is relayed via ABS to other far-end RSUs in the area that are under low load. The system achieves multi-level load balancing under air-ground collaboration by dynamically adjusting the task allocation ratio of each path.

[0037] Step 2: Construct a power-constrained sensing model In this embodiment, each autonomous vehicle with integrated sensing and computing capabilities... Limited by its own hardware power consumption and battery capacity, its total transmit power budget is set to This power is dynamically divided into three parts: sensing power. Communication power and calculate power Satisfying the constraints:

[0038] in, This formula defines the physical boundary of system resource allocation, meaning that the system must, under limited energy supply, dynamically adjust the power ratio to balance the accuracy of environmental perception, the rate of information transmission, and the response speed of local computing in real time.

[0039] Based on the power allocation scheme described above, a perception model is first constructed to achieve real-time monitoring of the traffic environment. (Autonomous vehicles) Utilizing allocated sensing power The vehicle-mounted sensors detect non-connected vehicles. This embodiment introduces perception accuracy. To quantify perceived quality, it shows an exponential positive correlation with the perceived power allocation value:

[0040] in, This is the sensitivity coefficient of the sensor, reflecting the response efficiency of the onboard sensing hardware to a unit power input. Higher sensing power results in higher sensing accuracy, but also higher energy consumption, which correspondingly reduces the resources available for communication and computing.

[0041] Step 3: Construct an air-to-ground cooperative communication model based on millimeter-wave beamforming. Having clarified the perception overhead, this embodiment constructs a communication model based on the millimeter-wave (mmWave) band. The wireless channel is modeled as a superposition of large-scale path loss, log-normal shadowing fading, and small-scale Rayleigh fading. The signal at a distance of [missing information] is calculated using a free-space path loss model. Propagation loss (unit: km) (unit: dB):

[0042] in (Unit: MHz) represents the carrier frequency. This is combined with the small-scale Rayleigh fading coefficient, which follows a complex Gaussian distribution. Complex channel coefficients Represented as:

[0043] in For the log-normal shaded component, For its standard deviation, take for ground links For over-the-air links This is to reflect the stronger multipath shadowing effect in the 28 GHz band terrestrial non-line-of-sight channel in urban areas. For nodes With nodes The distance between them is Free space path loss at km. The Rayleigh fading coefficient is a small-scale distribution that follows a cyclic symmetric complex Gaussian distribution.

[0044] Based on the above channel gain and allocated communication power autonomous vehicles With roadside units Direct transmission rate between Following Shannon's formula:

[0045] in It's bandwidth. For autonomous vehicles With roadside units Complex channel coefficients between It is the total noise power, derived from the noise power spectral density. Determined by the product of bandwidth:

[0046] The total noise power of the ABS receiver is similarly... .

[0047] For the ABS relay path, the transmission process is divided into two stages: access link AV→ABS and return link ABS→RSU. The transmission rates of the two links are as follows:

[0048]

[0049] in For ABS link bandwidth, The noise power of the access link (ABS receiver). The noise power of the backhaul link (RSU receiver). This represents the transmission power of the ABS.

[0050] Step 4: Construct a computational model based on a multi-layered offloading architecture Based on the perception and communication model, this embodiment constructs a task computation model. Autonomous vehicles Real-time generated computational tasks consist of triples The definitions represent the task data volume, required computation cycle, and maximum tolerable latency, respectively. To reflect the latency sensitivity of different tasks, the system introduces a task priority classification: safety-critical tasks with strict low-latency requirements and non-critical tasks.

[0051] The system adopts a continuous partial unloading strategy, which unloads individual tasks. Dynamically divided into local calculation ratios Direct uninstallation ratio and relay unloading ratio And it satisfies the completeness constraint:

[0052] in, vehicles Processing tasks The proportion of tasks allocated to local computing, direct RSU unloading, and ABS relay unloading at any given time. .

[0053] To determine the target RSU for direct and relay paths, the system selects the optimal node by maximizing the path score. The path score is defined as the reciprocal of the estimated end-to-end delay, taking into account transmission delay, computation delay, and link quality. Candidate nodes for direct paths... The rating is:

[0054] For the task The amount of data, For estimation based on the current channel state arrive direct transmission rate, For RSU Available computing power For the task The required computational load, relayed via ABS to... The rating is:

[0055] in This represents an estimate based on the current channel and load conditions. To prevent small constants from being divided by zero, For path loss estimation The access link transmission rate to ABS ABS based on path loss estimation The backhaul link transmission rate, For RSU Available computing power.

[0056] The final selected target RSUs are as follows:

[0057] and forced To avoid resource conflicts, This is the set of indices for all RSUs.

[0058] If you choose to compute locally, the computation latency is defined as follows:

[0059] in This indicates the vehicle's local computing power, which is affected by the allocation of computing power. For vehicles Processing tasks The proportion of tasks allocated to local computation at any given time. From autonomous vehicles Unloading to roadside unit The transmission delay is:

[0060] In roadside units The computation delay is:

[0061] in It is a roadside unit Its dynamic computing power is determined by its current load level. Decide:

[0062] That is RSU's maximum computing power. The amount of unloaded tasks increases dynamically and decreases naturally over time.

[0063] If the task is offloaded to a low-load RSU via the ABS relay, the transmission delay includes two hops:

[0064] Ultimately, because the three paths of local computation, direct unloading, and relay unloading are executed in parallel, the task... The total completion latency depends on the slowest execution path:

[0065] Only activation paths with non-zero uninstallation rates are included in the maximum value calculation. For vehicles Execute tasks locally The computational delay, For vehicles The task The direct offloading portion is transferred to the RSU. Transmission delay, For RSU Processing tasks The computational latency of the direct-connection unloading portion. For the task Forwarded to RSU via ABS relay The cumulative transmission delay of the two hops, For RSU Processing tasks The computational delay for the relay offloading section. The system needs to dynamically adjust the power. Relative to uninstallation ratio ,make sure To meet the requirements for business service quality.

[0066] Step 5: Constructing a sensory-computation integrated air-ground collaborative system to maximize its utility. After constructing the individual models for the synesthetic calculation described above, this step establishes a system utility maximization problem based on this, through a task offloading strategy. With power distribution scheme Joint optimization is performed to maximize the overall system utility while satisfying the latency constraints of each task. System utility is comprehensively measured from two dimensions: first, the quality of user experience (QoE), which reflects perceived performance; and second, the degree to which the latency of each task is satisfied relative to the deadline.

[0067] autonomous vehicles The quality of service experience is defined as:

[0068] in From sensing power Decide, This is the sensor sensitivity coefficient. For the task The percentage of timeouts. When the task is completed on time. When a timeout occurs, QoE decreases exponentially with the degree of timeout, penalizing delay defaults in a continuously differentiable manner.

[0069] To standardize the units of measurement and measure the degree of delay satisfaction, define the task. The normalized delay is:

[0070] in To the maximum tolerable delay, The smaller the value, the better the latency performance.

[0071] Based on the above indicators, the system utility maximization problem can be modeled as follows:

[0072] in and Weighting coefficients to balance perceived quality and latency performance. Constraints Ensure that each task is completed within the deadline; and Ensure the legality of the uninstallation ratio; Ensure that the total compute load offloaded to RSU does not exceed the system's aggregate computing power limit. ,in express Available computing resources within a unit time window; and Ensure the non-negativity of power allocation and total power constraints. For autonomous vehicles The quality of service experience For the task Normalized delay.

[0073] The aforementioned optimization problem involves dynamic channel conditions, time-varying RSU load, and coupled decisions regarding offloading and power, which traditional optimization methods struggle to solve in real time. Therefore, it is modeled as a Markov Decision Process (MDP). The solution is obtained using deep reinforcement learning.

[0074] state space Time step status It is composed of global context information and the private characteristics of each vehicle:

[0075] in and These are the locations of the RSU and ABS, respectively. Real-time load for each RSU For vehicles Location, for Distance vectors to each RSU and ABS This provides the current task attributes and task queue statistical characteristics.

[0076] Action space Each vehicle action It contains two components: Uninstallation action: Sampling from the Dirichlet distribution naturally satisfies the constraints. , ; Power distribution action: Similarly, the proportional vector is output using a Dirichlet distribution to ensure the constraint. , The structure automatically satisfies this requirement.

[0077] award Reward signal The design is aligned with the goal of maximizing system utility, as defined in step six.

[0078] Step Six: Design a reinforcement learning reward function oriented towards maximizing system utility The reward function is the core mechanism that guides the agent to converge to the optimal policy. To align the optimization objective of reinforcement learning with the system utility maximization problem established in step five, and to effectively punish delay-related violations, the following reward function is designed in this step.

[0079] Time step Instant rewards Defined as the weighted sum of the current mission rewards for all vehicles:

[0080] in Weighting coefficients are applied to task priorities, corresponding to safety-critical tasks. Non-critical tasks correspond This allows the gains and losses of high-priority tasks to have a greater impact on strategy updates. and This is a balance coefficient between perceived quality and latency performance. For autonomous vehicles The quality of service experience For the task Normalized delay.

[0081] To further differentiate between on-time completion and overtime completion, a time delay incentive term is introduced. :

[0082] in This is the excitation amplitude coefficient. This function is in... The gradient is continuously zero-crossing at each point. When the time is completed on time, a positive excitation is output, and when the timeout occurs, it naturally turns into a negative penalty. Both sides are continuously differentiable, avoiding the gradient discontinuity problem caused by hard threshold.

[0083] To incentivize agents to continuously reduce execution latency while ensuring timely completion, a speed reward is introduced. :

[0084] Lower latency results in higher rewards, providing a continuous descent gradient and preventing the policy from getting stuck in a local optimum that is just right.

[0085] To prevent the agent from over-compressing sensing power, a low sensing power penalty term is introduced. :

[0086] in This is the lower limit threshold for sensing accuracy. The penalty coefficient ensures that the agent maintains basic perception quality while optimizing latency.

[0087] The above reward mechanism combines QoE exponential reduction, tanh delay incentive, continuous velocity reward and low perception penalty to encode the two dimensions of maximizing perceived quality and satisfying delay constraints in the optimization objective into the reward signal, providing an effective learning signal for subsequent policy optimization of the PPO algorithm.

[0088] Step 7: Solving the joint offloading and power allocation problem based on the near-end policy optimization algorithm For the MDP problem established in step five, this step uses the Proximal Policy Optimization (PPO) algorithm for solution. PPO is a deep reinforcement learning method based on policy gradients, which achieves efficient policy iteration while ensuring training stability by limiting the magnitude of each policy update.

[0089] The algorithm employs an Actor-Critic dual-network architecture. (Actor network) In the current state As input, output the Dirichlet distribution concentration parameters for the unloading action and the power distribution action, respectively. And sample the continuous motion from it. Critic Network The cumulative reward of the current state is estimated to provide a baseline for policy updates. Both networks employ a multi-layer fully connected structure with ReLU activation function and use independent Adam optimizers for parameter updates.

[0090] To balance the bias and variance of the return estimates, the generalized advantage estimation (GAE) method is used to calculate the advantage function:

[0091] in As a discount factor, GAE attenuation coefficient, This represents the temporal difference (TD) error. GAE is calculated in episode groups to ensure trajectory integrity. After calculation, the dominance values ​​are batch normalized to reduce training variance.

[0092] Actor networks maximize the objective function of the pruned agent. Update:

[0093] in This represents the probability ratio between the old and new strategies. To truncate the range hyperparameters, the truncation mechanism limits the magnitude of a single policy update, preventing the policy from getting trapped in local optima. The entropy of the current strategy, The entropy coefficient is linearly decaying with the training progress, encouraging full exploration in the early stage of training and gradually converging to a deterministic strategy in the later stage of training.

[0094] The Critic network minimizes the truncated value loss function. Independent updates:

[0095] in Using Smooth L1 loss makes it more robust to outliers; To achieve the target return value; Limit the range of single-step changes in value estimation to prevent excessive Critic updates; After multiple rounds of iterative training, the agent learns the optimal joint offloading and power allocation strategy for all autonomous vehicles under dynamic channel conditions, time-varying RSU loads, and different task priority scenarios. Once training is complete, the agent, based on global observations including global RSU load, infrastructure location, and the status of all vehicles, outputs in real-time the task allocation ratio and power allocation scheme for all vehicles in the scenario, achieving centralized joint scheduling decisions. Ultimately, while meeting the deadline constraints of each task, it effectively improves the system's perception quality and task processing efficiency, balances the computational load among RSUs, and enhances the overall service capability of the vehicle-to-everything (V2X) system in complex dynamic environments.

[0096] To verify the effectiveness of this invention, comparative simulation experiments were conducted, comparing the proposed PPO-based joint offloading and power allocation method with four benchmark strategies. The four benchmark strategies are: uniform random strategy (offloading ratio and power are uniformly and randomly sampled), all local strategy (all tasks are calculated locally only on the vehicle), all direct-connect RSU strategy (all tasks are offloaded to RSU processing via direct path), and all ABS relay strategy (all tasks are relayed via ABS before processing).

[0097] The comparison results are as follows Figure 3 , Figure 4 As shown, the proposed PPO method significantly outperforms the four benchmark strategies in both system latency and QoE. Regarding system latency, the PPO method is far lower than all benchmark strategies, especially compared to all ABS relay strategies, where the latency reduction is most significant. This is mainly due to PPO's ability to dynamically select the optimal offloading path based on real-time channel conditions and RSU load; all local strategies, limited by onboard computing power, also exhibit significantly higher latency than the PPO method. In terms of QoE, the PPO method achieves the highest QoE score through reasonable allocation of sensing power, showing a particularly significant improvement compared to all ABS relay strategies, fully demonstrating the superiority of intelligent power allocation in dynamic scenarios. In summary, the proposed method effectively improves the overall system utility while meeting latency constraints, validating the effectiveness of the designed reinforcement learning framework.

[0098] Based on the technical concept of the above-mentioned method in this application, a sensor-computing integrated resource allocation system for air-ground collaboration in a mixed traffic scenario is also provided, comprising: Basic model construction module: Based on the integrated sensing, sensing and computing (ISCC) model for air-ground cooperative hybrid traffic scenarios, a power-constrained perception model, a millimeter-wave beamforming-based air-ground cooperative communication model, and a multi-layer offloading architecture-based computing model are constructed to complete the construction of the integrated sensing, sensing and computing basic model. The optimization problem construction module uses the existing perception model, air-ground cooperative communication model, and computational model to establish a system utility maximization problem. By jointly optimizing the task offloading strategy and power allocation scheme, the overall system utility is maximized while satisfying the latency constraints of each task. The system utility maximization problem is modeled as a Markov decision process. Reward function design module: used to design reinforcement learning reward functions aligned with the goal of maximizing system utility, integrating weighted and penalized mechanisms based on task priority, perceived quality, and latency constraints; The strategy solving module is used to solve the Markov decision process by combining the reinforcement learning reward function and the proximal policy optimization (PPO) algorithm. Through iterative training using the Actor-Critic dual network architecture, the optimal task offloading ratio and power allocation scheme are obtained, and the joint scheduling of the integrated sensory computing resources is completed.

[0099] This invention proposes an integrated resource allocation method for sensing, communication, and computing based on an air-ground cooperative architecture in mixed traffic scenarios. Addressing the challenges of long-term coexistence of autonomous and non-connected vehicles in existing vehicle-to-everything (V2X) systems, uneven spatiotemporal load distribution of roadside units (RSUs), and difficulties in optimizing the coupling of communication, sensing, and computing resources, this invention constructs an ABS-assisted air-ground cooperative multi-layer offloading architecture. Tasks are dynamically divided into three parallel processing paths: local computing, direct RSU offloading, and relaying to low-load RSUs via ABS. This effectively expands service coverage and achieves dynamic load balancing among RSUs. Simultaneously, this invention dynamically divides the total vehicle power budget into three parts: sensing, communication, and computing. Sensing power drives onboard sensors to perform real-time detection of non-connected vehicles, achieving deep integration of sensing, communication, and computing functions on the same hardware resource. This invention employs a PPO-based centralized reinforcement learning algorithm, introducing generalized advantage estimation and decay entropy regularization mechanisms to jointly optimize the task offloading ratio and three-dimensional power allocation. While meeting the maximum tolerable latency constraints for each task, this maximizes system perception quality and task processing efficiency, improving the overall resource utilization and service quality of the V2X system in mixed traffic scenarios.

[0100] The above content is only for illustrating the technical concept of the present invention and should not be construed as limiting the scope of protection of the present invention. Any modifications made to the technical solution based on the technical concept proposed in this invention shall fall within the scope of protection of the claims of this invention.

Claims

1. A method for resource allocation integrating sensing and computation in air-ground collaboration under mixed traffic scenarios, characterized in that, Includes the following steps: Based on the integrated air-ground cooperative hybrid transportation scenario of sensing, communication and computing, a power-constrained perception model, a millimeter-wave beamforming-based air-ground cooperative communication model, and a multi-layer offloading architecture-based computing model are constructed respectively. Based on the constructed perception model, air-ground cooperative communication model, and computational model, a system utility maximization problem is established. By jointly optimizing the task offloading strategy and power allocation scheme, the overall system utility is maximized while satisfying the latency constraints of each task. The system utility maximization problem is modeled as a Markov decision process. The design of the reinforcement learning reward function is aligned with the goal of maximizing system utility. The reinforcement learning reward function is designed with weights based on task priority, perceived quality, and latency satisfaction, and penalizes task timeout behavior. By combining the reinforcement learning reward function, the Proximal Policy Optimization (PPO) algorithm is used to solve the Markov decision process. Through iterative training using the Actor-Critic dual-network architecture, the optimal task offloading ratio and power allocation results are obtained.

2. The integrated sensor-computation resource allocation method for air-ground collaboration in a mixed traffic scenario as described in claim 1, characterized in that, The air-ground cooperative hybrid traffic scenario includes autonomous vehicles with integrated sensing and computing capabilities, non-connected vehicles, multiple roadside units (RSUs), and an airborne base station; the computing tasks generated by the AV have a three-level cooperative processing path: local processing, direct offloading to the near-end RSU, and offloading to the far-end low-load RSU via ABS relay. Randomly distributed within the two-way lane area A number of autonomous vehicles are deployed, with one RSU fixed at each of the four corners of the area and a height of [missing information] at the center of the area. The airborne base station, with non-connected vehicles randomly distributed within the area as dynamic sensing targets for AV; The three-level collaborative processing path specifically includes: the local processing path, where tasks are directly executed by the AV vehicle terminal; Direct offloading path: The task is directly offloaded and sent to the RSU near the AV for edge computing; Relay offloading path: When the load of the near-end RSU exceeds a preset threshold, the task is relayed to other remote RSUs with low load in the area via the ABS relay.

3. The integrated sensor-computation resource allocation method for air-ground collaboration in a mixed traffic scenario as described in claim 1, characterized in that, In power-constrained perception models, the perception accuracy of autonomous vehicles is determined by the perception power, specifically: in, For the first The perception accuracy of the AV unit. This is the sensor sensitivity coefficient. For the first The sensing power allocated to each AV unit.

4. The integrated sensor-computation resource allocation method for air-ground collaboration in a mixed traffic scenario as described in claim 1, characterized in that, The computational model based on the multi-layer offloading architecture is as follows: autonomous vehicles Real-time generated computational tasks consist of triples Define, where For task data volume, The required computation period for the task The maximum tolerable latency for the task; To reflect the different sensitivity of tasks to latency, the system introduces a task priority classification: safety-critical tasks with strict low-latency requirements and non-critical tasks. Employing a sequential partial unloading strategy, individual tasks are... Dynamically divided into local calculation ratios Direct uninstallation ratio and relay unloading ratio And it satisfies the completeness constraint: in, All are non-negative values.

5. The integrated sensor-computation resource allocation method for air-ground collaboration in a mixed traffic scenario as described in claim 1, characterized in that, The objective function constructed for the system utility maximization problem is: The system utility maximization problem is modeled as follows: in and To balance the weighting coefficients for perceived quality and latency performance, constraints are imposed. Ensure that each task is completed within the deadline; and Ensure the legality of the uninstallation ratio; Ensure that the total compute load offloaded to RSU does not exceed the system's aggregate computing power limit. ,in express Available computing resources within a unit time window; and To ensure the non-negativity of power allocation and the total amount constraint, For autonomous vehicles The quality of service experience For the task Normalized delay.

6. The integrated sensor-computation resource allocation method for air-ground collaboration in a mixed traffic scenario as described in claim 1, characterized in that, Markov decision processes consist of quaternions Define, where: state space Time step status It is composed of global context information and the private features of each vehicle, specifically including the location of RSU and ABS, the real-time load of each RSU, the location of each AV, the distance vector from each AV to RSU and ABS, the current task attributes of each AV and the task queue statistical features. Action space The actions of each AV It includes two components: offloading action and power allocation action; the offloading action includes local calculation ratio, direct connection offloading ratio and relay offloading ratio, and the power allocation action includes the allocation ratio of sensing power, communication power and local calculation power. award Immediate reward signals aligned with the goal of maximizing system utility; Discount factor : Discount factor used to calculate cumulative returns.

7. The integrated sensor-computation resource allocation method for air-ground collaboration in a mixed traffic scenario as described in claim 1, characterized in that, The reward function for reinforcement learning is time step. Instant rewards Defined as the weighted sum of the current mission rewards for all vehicles, specifically: in Weighting coefficients are applied to task priorities, corresponding to safety-critical tasks. Non-critical tasks correspond , and A coefficient for balancing perceived quality and latency performance. For autonomous vehicles The quality of service experience For the task Normalized delay; The time-delay excitation term is expressed as: in This is the excitation amplitude coefficient; For the speed bonus item, the expression is: To prevent the strategy from getting stuck at a local optimum that is exactly in time; The low-perception power penalty term is expressed as follows: in This is the lower limit threshold for sensing accuracy. This is the penalty coefficient.

8. The integrated sensor-computation resource allocation method for air-ground collaboration in a mixed traffic scenario as described in claim 1, characterized in that, Combining the reinforcement learning reward function, the Proximal Policy Optimization (PPO) algorithm is used to solve the Markov decision process. Through iterative training using an Actor-Critic dual-network architecture, the optimal task offloading ratio and power allocation results are obtained, including: The advantage function is calculated using the generalized advantage estimation (GAE) method, and its expression is: in As a discount factor, GAE attenuation coefficient, For time-series difference errors, GAE calculates the error by episode, and after the calculation is completed, the dominance value is batch normalized. The Actor network updates by maximizing the pruned agent objective function, which is expressed as: in This represents the probability ratio between the old and new strategies. For the truncation range hyperparameter, The entropy of the current strategy, The entropy coefficient is linearly decaying with the training progress; The Critic network updates independently by minimizing a truncated value loss function, the expression of which is: in For Smooth L1 loss, To achieve the target return value, For the truncated value estimate; After multiple rounds of iterative training until reward convergence, the optimal joint offloading and power allocation strategy is obtained. After training, the task allocation ratio and power allocation scheme of all vehicles in the scenario are output in real time based on global observation, and centralized joint scheduling is completed.

9. A sensor-computing integrated resource allocation system for air-ground collaboration in a mixed traffic scenario, characterized in that, include: Basic model construction module: Based on the integrated sensing, sensing and computing (ISCC) model for air-ground cooperative hybrid traffic scenarios, a power-constrained perception model, a millimeter-wave beamforming-based air-ground cooperative communication model, and a multi-layer offloading architecture-based computing model are constructed to complete the construction of the integrated sensing, sensing and computing basic model. The optimization problem construction module uses the existing perception model, air-ground cooperative communication model, and computational model to establish a system utility maximization problem. By jointly optimizing the task offloading strategy and power allocation scheme, the overall system utility is maximized while satisfying the latency constraints of each task. The system utility maximization problem is modeled as a Markov decision process. Reward function design module: used to design reinforcement learning reward functions aligned with the goal of maximizing system utility, integrating weighted and penalized mechanisms based on task priority, perceived quality, and latency constraints; The strategy solving module is used to solve the Markov decision process by combining the reinforcement learning reward function and the proximal policy optimization (PPO) algorithm. Through iterative training using the Actor-Critic dual network architecture, the optimal task offloading ratio and power allocation scheme are obtained, and the joint scheduling of the integrated sensory computing resources is completed.

10. A sensor-computing integrated system for air-ground coordination in a mixed traffic scenario, characterized in that, Within a city, the square area enclosed by roads includes... An autonomous vehicle with integrated sensing and computing capabilities is randomly distributed on a two-way lane and maintains a constant speed. A Remote Unit (RSU) is fixedly deployed at each of the four corners of the inner side of the road. A height of [missing information] is introduced at the center of a square area. The ABS provides signal coverage and task relay services. Non-connected vehicles are also randomly distributed within the square area. These serve as dynamic perception targets for autonomous vehicles and do not participate in the communication network. The location information of non-connected vehicles needs to be obtained in real time by the autonomous vehicle through onboard sensors. Resource allocation is carried out using the integrated air-ground communication and computing resource allocation method for hybrid traffic scenarios as described in any one of claims 1-7.

Images